home *** CD-ROM | disk | FTP | other *** search
Text File | 1999-04-09 | 41.1 KB | 1,390 lines |
- Linux SMP FAQ
- David MentrΘ, David.Mentre@irisa.fr
- v0.47, 20 november 1998
-
- This FAQ review main issues (and I hope solutions) related to SMP con
- figuration under Linux.
- ______________________________________________________________________
-
- Table of Contents
-
-
- 1. Introduction
-
- 2. What's new ?
-
- 3. Questions related to any architectures
-
- 3.1 Kernel side
- 3.2 User side
- 3.3 SMP Programming
- 3.3.1 Parallelization methods
- 3.3.2 The C Library
- 3.3.3 Languages, Compilers and debuggers
- 3.3.4 Other libraries
- 3.3.5 Other points about SMP Programming
-
- 4. Intel architecture specific questions
-
- 4.1 Why it doesn't work on my machine?
- 4.2 Possible causes of crash
- 4.3 Motherboard specific information
- 4.3.1 Motherboards with known problems
- 4.3.2 Motherboards with NO known problems
- 4.4 How to obtain maximum performance ? (not strictly SMP related)
- 4.4.1 From disks
-
- 5. Useful pointers
-
- 5.1 Various
- 5.2 Multithreaded programs and library
- 5.3 SMP specific patches
- 5.4 Parallelizing/Optimizing Compilers for 586/686 machines (
-
- 6. Glossary
-
- 7. List of contributors
-
-
-
- ______________________________________________________________________
-
- 1. Introduction
-
- Linux can work on SMP (Symetric Multi-Processors) machines. SMP
- support has started with the 2.0 family and has been improved in the
- 2.1 (future 2.2) saga.
-
-
- FAQ maintained by David MentrΘ (David.Mentre@irisa.fr). The latest
- edition of this FAQ can be found at
-
- ╖ http://www.irisa.fr/prive/mentre/smp-faq/ (France)
-
- ╖ http://www.phy.duke.edu/brahma/smp-faq/ (USA)
-
-
- If you want to contribute to this FAQ, I would prefer a diff against
- the SGML version <http://www.irisa.fr/prive/mentre/smp-faq/smp-
- faq.sgml> of this document, but any remarks (in plain text) will be
- greatly appreciated. If you send me an email about this FAQ, please
- include a tag like [Linux SMP FAQ] in the Subject: field of your e-
- mail. It helps me automatically sort mails (and you will have a faster
- reply ;)).
-
-
- This FAQ is an improvement of a first draft made by Chris Pirih.
-
-
- All information contained in this FAQ is provided "as is." All
- warranties, expressed, implied or statutory, concerning the accuracy
- of the information of the suitability for any particular use are
- hereby specifically disclaimed. While every effort has been taken to
- ensure the accuracy of the information contained in this FAQ, the
- authors assume(s) no responsibility for errors or omissions, or for
- damages resulting from the use of the information contained herein.
-
-
- 2. What's new ?
-
-
-
- v0.47, 20 november 1998
-
- ╖ Added that 2.0.36 as the MTRR patch (related to the BogoMips
- problem)
-
-
- v0.46, 10 november 1998
-
- ╖ Update about Epox KP6-LS motherboards
-
-
- v0.45, 25 october 1998
-
- ╖ Corrected an error regarding /proc/stat file
-
- ╖ Added a pointer to CESDIS Ethernet Linux Drivers site
-
-
- v0.44, 14 october 1998
-
- ╖ Updated the link to the web page: Motherboards rumored to run
- Linux SMP
-
- ╖ Added Jakob explanation how to time SMP systems with 2.0 kernels
-
-
- v0.43, 9 september 1998
-
- ╖ Updated first question in section 3.1
-
- ╖ Updated mt-Mesa link: multi-threaded is now included as
- experimental in the Mesa distribution
-
-
- v0.42, 2 september 1998
-
- ╖ Minor cosmetic update in sect 3.3
-
- ╖ Two links (multithreaded Mesa and SMP performance) marked
- outdated
-
- ╖ Updated the item about threads and exceptions in C++ (sect 3.3)
-
-
- v0.41, 1 september 1998
-
- ╖ Added a major section: "3.3 SMP Programming" written by Jakob
- ╪stergaard
-
- ╖ moved some item of section "3.2 User side" in sect 3.3
-
-
- v0.40, 27 august 1998
-
- ╖ Updated section 3.1, item 7: processor affinity
-
-
- v0.39, 27 august 1998
-
- ╖ Updated needed Award BIOS version for Tyan motherboards (hASCII)
-
- ╖ Added an item on IRQ in the crash section (me and hASCII)
-
- ╖ Added good support of Asus P2B-DS (Ulf Rompe)
-
- ╖ Added another smp-list archive in pointer section (Hank
- Leininger)
-
-
- v0.38, 8 august 1998
-
- ╖ Added a pointer to the Linux Threads FAQ
-
-
- v0.37, 30 July 1998
-
- ╖ Emil Briggs is working on parallel plugins for Gimp (see "Is
- there any threaded programs or library?", sect. "User side")
-
-
- v0.36, 26 July 1998
-
- ╖ Thanks to Jakob ╪stergaard, two changes in "Possible causes of
- Crash"
-
- ╖ Changed 2.0.33 to 2.0.35 (latest stable)
-
- ╖ Added a "BIOS related causes of failure"
-
-
- v0.35, 14 July 1998
-
- ╖ Added N440BX Server Board in Motherboards with NO problems
-
- ╖ Added a succes story for GigaByte motherboard with BIOS upgrade
-
- ╖ Added a "How to obtain maximum performance ?" section (waiting
- for your contributions ;)
-
-
- v0.34, 10 june 1998
-
- ╖ Added a "Parallelizing/Optimizing Compilers for 586/686
- machines" section in section "Useful Pointers", thanks to Sumit
- Roy
-
-
- ╖ Corrected a mispelling, "Asus P/I-UP5" is in fact "Asus P/I-
- P65UP5"
-
-
- v0.33, 3 june 1998
-
- ╖ Yet another success story for a GigaByte DLX Motherboard.
-
- ╖ A tip for Tyan motherboards, disable the "DRAM Fast Leadoff"
- BIOS option
-
-
- v0.32, 27 may 1998
-
- ╖ Asus P/I-UP5 added in the motherboard-with-NO-problem section
-
-
- v0.31, 18 may 1998
-
- ╖ Elitegroup P6LX2-A works with 2.1.100 and 101
-
- ╖ Bugs should be reported to linux-smp@vger.rutgers.edu
-
-
- v0.30, 12 may 1998
-
- ╖ SuperMicro is now in the motherboard-with-NO-problem section
-
-
- v0.29, 11 may 1998
-
- ╖ A success story for a GigaByte 686 motherboard with 2.1.101
-
- ╖ Added a new item in the "User Side" section: "Is there any
- threaded programs or library?"
-
- ╖ OpenGL Mesa library is beeing multithreaded. Cool! See the new
- section for details.
-
-
- v0.28, 09 may 1998
-
- ╖ A US mirror of this FAQ is now available (see Introduction)
-
- ╖ Merge of the two confusing Gigabyte 686 entries
-
-
- v0.27, 05 may 1998
-
- ╖ New info for the Adaptec and TekRam drivers
-
- ╖ Micronics W6-LI motherboard works under SMP
-
-
-
-
- 3. Questions related to any architectures
-
-
- 3.1. Kernel side
-
-
- 1. Does Linux support multi-threading? If I start two or more
- processes, will they be distributed among the available CPUs?
-
- Yes. Processes and kernel-threads are distributed among processors.
- 2. What kind of architectures are supported in SMP?
-
-
- From Alan Cox:
- SMP is supported in 2.0 on the hypersparc (SS20, etc.) systems
- and Intel 486, Pentium or higher machines which are Intel
- MP1.1/1.4 compliant. Richard Jelinek adds: right now, systems
- have been tested up to 4 CPUs and the MP standard (and so Linux)
- theoretically allows up to 16 CPUs.
-
- SMP support for UltraSparc, SparcServer, Alpha and PowerPC
- machines is in progress in 2.1.x.
-
-
- From Ralf BΣchle:
- MIPS, m68k and ARM does not support SMP; the latter two probly
- won't ever.
-
- That is, I'm going to hack on MIPS-SMP as soon as I get a SMP
- box ...
-
-
- 3. How do I make a Linux SMP kernel?
-
- Uncomment the SMP=1 line in the main Makefile
- (/usr/src/linux/Makefile).
-
- AND
-
- enable "RTC support" (from Robert G. Brown). Note that inserting
- RTC support actually doesn't afaik prevent drift, but according to
- a discussion [Robert G. Brown] remember from a year ago or so it
- can prevent lockup when the clock is read at boot time. A note from
- Richard Jelinek says also that activating the Enhanced RTC is
- necessary to get the second CPU working (identified) on some
- original Intel Mainboards.
-
- AND
-
- do NOT enable APM! APM and SMP are not compatible, and your system
- will almost certainly (or at least probably ;)) crash under boot if
- APM is enabled (Jakob Oestergaard). Alan Cox confirms this : 2.1.x
- turns APM off for SMP boxes. Basically APM is undefined in the
- presence of SMP systems, and anything could occur.
-
-
-
- You must rebuild all your kernel and kernel modules when changing
- to and from SMP mode. Remember to make modules and make
- modules_install (from Alan Cox).
-
-
- 4. How do I make a Linux non-SMP kernel?
-
- Comment the SMP=1 line in the Makefile (and not set SMP to 0).
-
-
-
- You must rebuild all your kernel and kernel modules when changing
- to and from SMP mode. Remember to make modules and make
- modules_install.
-
-
-
- 5. How can I tell if it worked?
-
- cat /proc/cpuinfo
-
-
-
-
- Typical output (dual PentiumII):
-
- ______________________________________________________________________
- processor : 0
- cpu : 686
- model : 3
- vendor_id : GenuineIntel
- stepping : 3
- fdiv_bug : no
- hlt_bug : no
- fpu : yes
- fpu_exception : yes
- cpuid : yes
- wp : yes
- flags : fpu vme de pse tsc msr pae mce cx8 apic 11 mtrr pge mca cmo
- v mmx
- bogomips : 267.06
-
- processor : 1
- cpu : 686
- model : 3
- vendor_id : GenuineIntel
- stepping : 3
- fdiv_bug : no
- hlt_bug : no
- fpu : yes
- fpu_exception : yes
- cpuid : yes
- wp : yes
- flags : fpu vme de pse tsc msr pae mce cx8 apic 11 mtrr pge mca cmo
- v mmx
- bogomips : 267.06
- ______________________________________________________________________
-
-
-
-
- 6. What is the status of converting the kernel toward finer grained
- locking and multithreading?
-
- 2.1.x has signal handling, interrupts and some I/O stuff fine grain
- locked. The rest is gradually migrating. All the scheduling is SMP
- safe
-
-
- 7. Does Linux SMP support processor affinity?
-
-
-
- Standard kernel
- No and Yes. There is no way to force a process onto specific
- CPU's but the linux scheduler has a processor bias for each
- process, which tends to keep processes tied to a specific CPU.
-
-
- Patch
- Yes. Look at PSET - Processor Sets for the Linux kernel
- <http://isunix.it.ilstu.edu/~thockin/pset/>:
-
- The goal of this project is to make a source compatible
- and functionally equivalent version of pset (as defined
- by SGI - partially removed from their IRIX 6.4 kernel)
- for Linux. This enables users to determine which
- processor or set of processors a process may run on.
- Possible uses include forcing threads to seperate proces
- sors, timings, security (a `root' only CPU?) and probably
- more.
-
-
-
-
- 8. Where should one report SMP bugs to?
-
- Please report bugs to linux-smp@vger.rutgers.edu.
-
-
-
-
- 3.2. User side
-
-
- 1. Do I really need SMP?
-
- If you have to ask, you probably don't. :)
-
-
- 2. How does one display mutiple cpu performance?
-
- Thanks to Samuel S. Chessman, here is some useful utilities:
-
- Character based:
- http://www.cs.inf.ethz.ch/~rauch/procps.html
-
- Basically, it's procps v1.12.2 (top, ps, et. al.) and some
- patches to support SMP.
-
-
- Graphic:
- xosview-1.5.1 supports SMP. And kernels above 2.1.85 (included)
- the cpuX entry in /proc/stat file.
-
- The official homepage for xosview is:
- http://lore.ece.utexas.edu/~bgrayson/xosview.html
-
- The various forissier's kernel patches are at: http://www-
- isia.cma.fr/~forissie/smp_kernel_patch/
-
-
- 3. How can I enable more than 1 process for my kernel compile?
-
- use:
-
- ___________________________________________________________________
- # make [modules|zImage|bzImages] MAKE="make -jX"
- where X=max number of processes.
- WARNING: This won't work for "make dep".
- ___________________________________________________________________
-
-
-
- With a 2.1.x like kernel, see also the file
- /usr/src/linux/Documentation/smp for specific instruction.
-
- BTW, since running multiple compilers allows a machine with sufficient
- memory to use use the otherwise wasted CPU time during I/O caused
- delays make MAKE="make -j 2" -j 2 actually even helps on uniprocessor
- boxes (from Ralf BΣchle).
-
-
- 4. Why the time given by the time command is false ? (from Joel
- Marchand)
-
- In the 2.0 series, the result given by the time command is false.
- The sum user+system is right *but* the spreading between user and
- system time is false.
-
- More precisely: "The explanation is, that all time spent in
- processors other than the boot cpu is accounted as system time. If
- you time a program, add the user time and the system time, then you
- timing will be almost right, except for also including the system
- time that is correctly accounted for" (Jakob ╪stergaard).
-
- This bug in corrected in 2.1 series.
-
-
- 5. How will my application perform under SMP?
-
- Look at SMP Performance of Linux [OUTDATED]
- <http://www.interlog.com/~mackin/linux-smp.html> which gives useful
- hints how to bench a specific machine (from a post made by Cameron
- MacKinnon).
-
-
-
-
-
-
- 3.3. SMP Programming
-
- Section made by Jakob ╪stergaard.
-
- This section is intended to outline what works, and what doesn't, when
- it comes to programming multi-threaded software for SMP Linux.
-
-
- 3.3.1. Parallelization methods
-
-
- 1. POSIX Threads
-
- 2. PVM / MPI Message Passing Libraries
-
- 3. fork() -- Multiple processes
-
- Since both fork() and PVM/MPI processes usually does not share memory,
- but either communicate by means of IPC or a messaging API, they will
- not be described further in this section. They are not very specific
- to SMP, since they are used just as much - or more - on uniprocessor
- computers, and clusters thereof.
-
- Only POSIX Threads provide us with multiple threads sharing ressources
- like - especially - memory. The exact same thing that makes a SMP
- machine special, by allowing many processors to share their memory. To
- use both (or more ;) processors of an SMP, use a kernel-thread
- library. A good library is the LinuxThreads, a pthread library made by
- Xavier Leroy <http://pauillac.inria.fr/~xleroy/linuxthreads/> which is
- now integrated with glibc2 (aka libc6).
-
- Multi-threading has never been really popular in the UN*X world
- though. For some reason, applications requiring multiple processes or
- threads, has mostly been written using fork(). Therefore, when using
- the thread approach, one runs into problems of incompatible (not
- thread-ready) libraries, compilers, and debuggers. GNU/Linux is no
- exception to this. Hopefully the next few sections will sched a little
- light over what is currently possible, and what is not.
- 3.3.2. The C Library
-
- Older C libraries are not thread-safe. It is very important that you
- use GNU LibC (glibc), also known as libc6. Earlier versions are
- ofcourse possible to use, but it will cause you much more trouble than
- upgrading your system will, well probably :)
-
- If you want to use GDB to debug your programs, you should get the
- patches or SRPMS available for glibc at
- http://odin.appliedtheory.com/debug_thread_rpms/RedHat_5.1/ which will
- allow GDB to work with threads.
-
- It is not nessecary to patch glibc in any way just to make it work
- with threads. If you do not need to debug the software (this could be
- true for all machines that are not development workstations), there is
- no need to patch glibc.
-
-
- 3.3.3. Languages, Compilers and debuggers
-
- There is a wealth of programming languages available for GNU/Linux,
- and many of them can be made to use threads one way or the other (some
- languages like Ada and Java even have threads as primitives in the
- language).
-
- This section will, however, currently only describe C and C++. If you
- have experience in SMP Programming with other languages, please
- enlighten us.
-
- GNU C and C++, as well as the EGCS C and C++ compilers work with the
- thread support from the standard C library (glibc). There are however
- a few issues:
-
-
- 1. When compiling C or C++, use the -D_REENTRANT define in the
- compiler command line. This is necessary to make certain error-
- handling functions work.
-
- 2. When using C++, If two threads throw exceptions concurrently, the
- program will segfault. The compiler does not generate thread-safe
- exception code.
-
- The workaround is to put a
- pthread_mutex_lock(&global_exception_lock) in the constructor(s) of
- every class you throw(), and to put the corresponding
- pthread_mutex_unlock(...) in the destructor. It's ugly, but it
- works. This solution was given by Markus Ferch.
-
- The GNU Debugger GDB as of version 4.17, is incapable of handling
- threads, at least in any sensible way. There is however a patched
- version available at
- http://odin.appliedtheory.com/debug_thread_rpms/RedHat_5.1/, for
- RedHat Linux at least. It's probably quite easy to use the standalone
- patches on any other GNU/Linux distribution. Note that you must also
- patch your glibc.
-
- Note that core-dumps are of no use, when using multiple threads.
- Somehow the core dump is attached to one of the currently running
- threads, and not to the program as a whole. Therefore, whenever you
- are debugging anything, run it from the debugger.
-
- Hint: If you have a thread running haywire, like eating 100% CPU time,
- and you cannot seem to figure out why, here is a nice way to find out
- what's going on: Run the program straight from the shell, no GDB. Make
- the thread go haywire. Use top to get the PID of the process. Run GDB
- like gdb program pid. This will make GDB attach itself to the process
- with the PID you specified, and stop the thead. Now you have a GDB
- session with the offending thread, and can use bt and the likes to see
- what is happening.
-
-
- 3.3.4. Other libraries
-
- ElectricFence: This library is not thread safe. It should however be
- possible to make it work in SMP environments, by inserting mutex locks
- in the ElectricFence code.
-
-
-
- 3.3.5. Other points about SMP Programming
-
-
- 1. Where can I found more information about parallel programming?
-
- Look at the Linux Parallel Processing HOWTO
- <http://yara.ecn.purdue.edu/~pplinux/PPHOWTO/pphowto.html>
-
- Lots of useful information can be found at Parallel Processing
- using Linux <http://yara.ecn.purdue.edu/~pplinux/>
-
- Look also at the Linux Threads FAQ <http://linas.org/linux/threads-
- faq.html>
-
-
- 2. Is there any threaded programs or library?
-
- Yes. For programs, you should look at: Multithreaded programs on
- linux <http://www.e.kth.se/~e94_bek/mthread.html> (I love
- hyperlinks, did you know that ? ;))
-
- As far as library are concerned, there is:
-
-
- OpenGL Mesa library
- Thanks to David Buccarelli, Andreas Schiffler and Emil Briggs,
- it exists in a multithreaded version (right now [1998-05-11],
- there is a working version that provides speedups of 5-30% on
- some OpenGL benchmarks). The multithreaded stuff is now included
- in the regular Mesa distribution as an experimental option. For
- more information, look at the Mesa library
- <http://www.ssec.wisc.edu/~brianp/Mesa.html>
-
-
- BLAS
- Pentium Pro Optimized BLAS and FFTs for Intel Linux
- <http://www.cs.utk.edu/~ghenry/distrib/>
-
- Multithreaded BLAS routines are not available right now, but a
- dual proc library is planned for 1998-05-27, see Blas News
- <http://www.cs.utk.edu/~ghenry/distrib/blasnews> for details.
-
-
- The GIMP
- Emil Briggs, the same guy who is involved in multithreaded Mesa,
- is also working on multithreaded The GIMP plugins. Look at
- http://nemo.physics.ncsu.edu/~briggs/gimp/index.html for more
- info.
-
-
-
-
-
- 4. Intel architecture specific questions
-
-
- 4.1. Why it doesn't work on my machine?
-
-
- 1. Can I use my Cyrix/AMD/non-Intel CPU in SMP?
-
- Short answer: no.
-
- Long answer: Intel claims ownership to the APIC SMP scheme, and
- unless a company licenses it from Intel they may not use it. There
- are currently no companies that have done so. (This of course can
- change in the future) FYI - Both Cyrix and AMD support the non-
- proprietary OpenPIC SMP standard but currently there are no
- motherboards that use it.
-
-
- 2. Why doesn't my old Compaq work?
-
- Put it into MP1.1/1.4 compliant mode.
-
-
- 3. Why doesnt my ALR work?
-
- From Robert Hyatt : ALR Revolution quad-6 seems quite safe, while
- some older revolution quad machines without P6 processors seem
- "iffy"...
-
-
- 4. Why does SMP go so slowly? or Why does one CPU show a very low
- bogomips value while the first one is normal?
-
- From Alan Cox: If one of your CPU's is reporting a very low
- bogomips value the cache is not enabled on it. Your vendor probably
- provides a buggy BIOS. Get the patch to work around this or better
- yet send it back and buy a board from a competent supplier.
-
- Another work-around is to use the 2.0.36 kernel. It contains the
- MTRR patch which should solve this problem (select option "Handle
- buggy SMP BIOSes with bad MTRR setup" in the "General setup" menu).
-
-
- 5. I've heard IBM machines have problems
-
-
- Some IBM machines have the MP1.4 bios block in the EBDA, allowed
- but not supported by <2.1.80. Please update to the right kernel.
-
- There is an old 486SLC based IBM SMP box. Linux/SMP requires
- hardware FPU support.
-
-
- 6. Is there any advantage of Intel MP 1.4 over 1.1 specification?
-
- Nope (according to Alan :) ), 1.4 is just a stricker specs of 1.1.
-
-
- 7. Why does the clock drift so rapidly when I run linux SMP?
-
-
- This is known problem with IRQ handling and long kernel locks in
- the 2.0 series kernels. Consider upgrading to a later 2.1 kernel
- (not garenteed to work).
-
- From Jakob Oestergaard: Or, consider running xntpd. That should
- keep your clock right on time. (I think that I've heard that
- enabling RTC in the kernel also fixes the clock drift. It works for
- me! but I'm not sure whether that's general or I'm just being
- lucky)
-
-
-
- 8. Why are my CPU's numbered 0 and 2 instead of 0 and 1 (or some other
- odd numbering)?
-
- The CPU number is assigned by the MB manufacturer and doesn't mean
- anything. Ignore it.
-
-
-
- 9. My SMP system is locking up all the time. Black screen, nothing in
- the logs. Help!
-
- If you're running a 2.0 kernel, consider upgrading to later 2.0.32+
- kernels or apply Leonard Zubkoff's deadlock patch. If you still
- have deadlocks, apply Ingo Molnar's deadlock detection patch and
- post the results (against your System.map) to linux-smp or linux-
- kernel. You might also consider running a 2.1 kernel.
-
-
-
-
-
- 4.2. Possible causes of crash
-
- You'll find in this section some possible reasons for a crash of an
- SMP machine (credits are due to Jakob ╪stergaard for this part). As
- far as I (david) know, theses problems are Intel specific.
-
-
-
- ╖ Cooling problems
-
- From Ralf BΣchle: [Related to case size and fans] It's important
- that the air is flowing. It of course can't where cables etc. are
- preventing this like in too small cases. On the other side I've
- seen oversized cases causing big problems. There are some tower
- cases on the market that actually are worse for cooling than
- desktops. In short, the right thing is thinking about aerodynamics
- in the case. Extra cases for hot peripherals are usefull as well.
-
-
-
- ╖ Bad memory
-
- Don't buy too cheap RAM and don't use mixed RAM modules on a
- motherboard that is picky about it.
-
- Especially Tyan motherboards are known to be picky about RAM speed
- (see the Tyan paragraph below for a possible solution).
-
-
-
- ╖ Bad combination of different stepping CPUs
-
- Check /proc/cpuinfo to see that your CPUs are same stepping.
-
-
- ╖ You are running 2.0.35 aren't you ?
-
- If you run 2.0.31 or 2.1.xx you can't be sure that SMP is stable.
- 2.0.35 is the right kernel for a production system. 2.1.xx kernels
- perform better, but they are development releases and should NOT be
- considered stable!
-
-
- ╖ If your system is unstable, then DON'T overclock it!
-
- ...and even if it is stable, DON'T overclock.
-
- From Ralf BΣchle: Overclocking causes very subtile problems. I
- have a nice example, one of my overclocked old machines
- misscomputes a couple of pixels of a 640 x 400 fractal. The
- problem is only visible when comparing them using tools. So better
- say never, nuncas, jamais, niemals overclock.
-
-
-
- ╖ 2.0.x kernel and fast ethernet (from Robert G. Brown)
-
- 2.0.X kernels on high performance fast ethernet systems have
- significant (and known) problems with a race/deadlock condition in
- the networking interrupt handler.
-
- The solution is to get the latest 100BT development drivers from
- CESDIS Linux Ethernet device drivers site
- <http://cesdis.gsfc.nasa.gov/linux/drivers/> (ones that define
- SMPCHECK).
-
-
- ╖ A bug in the 440FX chipset (from Emil Briggs)
-
- If you had a system using the 440FX chipset then your problem with
- the lockups was possibly due to a documented errata in the chipset.
- Here is a reference
-
- References: Intel 440FX PCIset 82441FX (PMC) and 82442FX (DBX)
- Specification Update. pg. 13
-
- http://www.intel.com/design/pcisets/specupdt/297654.htm
-
- The problem can be fixed with a BIOS workaround (Or a kernel patch)
- and in fact David Wragg wrote a patch that's included with Richard
- Gooch's MTTR patch. For more information and a fix look here:
-
- http://nemo.physics.ncsu.edu/~briggs/vfix.html
-
-
- ╖ DONT run emm386.exe before booting linux SMP
-
- From Mark Duguid, dumb rule #1 with W6LI motherboards. ;)
-
-
- ╖ If the machine reboots/freezes after a while, there can be two good
- BIOS + memory related reasons (from Jakob ╪stergaard)
-
- ╖ If the BIOS has settings like "memory hole at 16M" and/or "OS/2
- memory > 64MB", try disabling them both. Linux does not always
- react well with theese options.
-
- ╖ If you have more than 64 MB of memory in the machine, and you
- specified the exact number manually in the LILO configuration, you
- should specify one MB less than you actually have in the machine.
- If you have 128 MB, you lilo.conf line looks like:
- append="mem=127M"
-
-
- ╖ Be aware of IRQ related problems
-
- Sometime, some cards are not recognized or can trigger IRQ
- conflicts. Try shuffle cards on slots in different ways.
-
- Contributed by hASCII : removing an " append="hisax=9,2,3"" line in
- lilo.conf allowed using a kernel from the 2.1.xx series with
- activated ISDN + Hisax support. Kernels from the 2.0.xx series
- doesn't make problems like this.
-
-
-
- Some hardware is also known to cause problems. This includes:
-
- ╖ Adaptec SCSI controllers
-
- Latest news (05 may 1998):
-
- The latest version of this driver (5.0.13 at this writting) should
- be SMP safe (both in 2.0 and 2.1 kernels). Doug had verified it
- through code review and intensive tests (on a dual PII system).
-
- This said, some machines still have problems. Doug has kindly given
- some explanations:
-
- ╖ Doug: The simple fact of the matter is that this version of the
- aic7xxx driver places a larger demand on DMA transfers than the old
- driver did, especially for for smaller DMA transactions. So, the
- point should be made that this version of the driver *is* more
- demanding on your system than either the 4.1 or 4.1.1 driver, and
- that certain marginal systems may get hit by this problem (such as
- my own P133 system is).
-
- ╖ In 2.1 kernels, they are still problems (with IRQ) with the kernel
- (2.1.99) and they are not related to this specific driver. People
- are working on this.
-
-
- ╖ 3Com 3c905 cards
-
- Some work, some don't. Try disabling busmastering if your system is
- unstable.
-
-
-
-
- 4.3. Motherboard specific information
-
- Please note: Some more specific information can be found with the list
- of Motherboards rumored to run Linux SMP <http://www.nlug.org/smp/>
-
-
- 4.3.1. Motherboards with known problems
-
-
- ╖ GA686DLX (Andrew Crane)
-
- Same BIOS related BogoMIPS problem as other motherboards.
-
- Solution from Alan Cox: Congratulations, send the bill for your
- hair damage to the supplier. You have yet another SMP box with
- faulty bios. There is a patch for 2.0.x on www.uk.linux.org and
- there are people working on generic MTRR handling for 2.1.x
-
- From Claus-Justus Heine: I'm able to boot above MB with 2.1.90 +
- Ingo's apic + Richard's mtrr patches. From David MentrΘ: the kernel
- > 2.1.99 have the MTRR patch included and Florian Hinzmann told me
- that with 2.1.101, both CPUs are detected fine.
-
- David Maslen tells: Fortunately I suspect these problems have been
- fixed, and would like to report my success so that others won't be
- put off what seems to be a nice motherboard. [...] My system has
- been running for about 3 days without a lockup (72hrs) Kernel
- 2.1.103 compiled easily. /proc/cpuinfo says the cpus are both
- there. This was also true of 2.0.33.
-
- Other possible solution: BIOS upgrade (see http://www.giga-
- byte.com).
-
- (14 July 1998)Volker Reichelt confirms that with a BIOS upgrade
- from v1.14 to v1.20 everything seems to work fine.
-
-
- ╖ EPoX KP6-LS (Christopher Allen Wing, 16 march 1998)
-
- It appears to have the same BIOS related BogoMIPS problem as other
- motherboards. (one CPU only gives about 3 BogoMIPS, the other gives
- the full amount) All 2.0.x kernels lock up soon after booting, late
- 2.1.x kernels run slowly but don't seem to lock up. There is no
- BIOS upgrade available (yet). I wrote the manufacturer but have not
- received a reply.
-
- (update, 10 november 1998) First of all, there are 2 revisions on
- the KP6-LS: (1) Revision 2 (production release, has a power-leak on
- the second processor connector - To fix this problem, contact Epox
- support and they will guide you for further assistence. - For some
- boards it is needed to replace the entire board. Cause the leak is
- corrected in the middle of this revision). (2) Revision 3
- (production release). For both the revisions is a patch avaliable
- from their site (http://www.epox.com/support/bios.html). Please
- make sure that Epox has been contacted about the bios upgrade
- before applying. After applying the patch, kernel 2.0.35 and all
- other expirimental kernels from 2.1.110 up to 2.1.125 report a
- correct BogoMIPS index. (Niels Ammerlaan)
-
-
- ╖ Tyan
-
- Tyan motherboards are known to be picky about RAM speed (Jakob
- Oestergaard).
-
- From Doug Ledford about the onboard aic-7895 SCSI controller (for
- which he wrote the driver): "BTW, make sure you have at least BIOS
- version 1.16 on that Tyan motherboard. The 1.15 and below BIOS
- versions have a bug relating to IRQ allocation for the 7895 SCSI
- controller" (submitted by Szakacsits Szabolcs).
-
- (27 august 1998) From hASCII: If you have a tyan with award bios,
- the newest version you can get is 1.02.
-
- But, as motherboard problems are more close to grayshade than black
- and white ;), Richard Jelinek (from S.u.S.E.) tells that they
- sell/sold several Tyan Titan Pro (Dual-PPro) Motherboards. They
- work perfectly with SMP.
-
- (3 june 1998) maybe an explanation found by Leonard Zubkoff and
- told tp me by Sean Reifschneider: The Tyan Titan Pro motherboards
- with 3.03 BIOS version (dated 10/31/96) has an option labled "DRAM
- Fast Leadoff", which MUST be disabled.
-
-
-
- ╖ MS-6114
-
- More details for this motherboard at
- http://www.msi.com.tw/product/6114/6114.htm
- <http://www.msi.com.tw/product/6114/6114.htm>
-
- Solution: BIOS upgrade
-
- Somebody experienced solid hangs (nothing in the log files) under
- constant load of about 5 running processes within less than 12
- hours with AMI BIOS v1.1. v1.4b3 runs without problems.
-
-
-
-
-
- 4.3.2. Motherboards with NO known problems
-
-
- ╖ AIR P6NDP and P6NDI (Leonard N. Zubkoff)
-
- My primary production machine is based on an AIR P6NDP and one of
- my test machines uses a P6NDI. Both seem to be fine motherboards
- in my experience. The P6NDI BIOS is a little conservative in its
- programming of the Natoma chipset for 50ns EDO, but a minor tweek
- to one register in rc.local took care of that.
-
-
- ╖ AIR 54CDP (Chris Mauritz)
-
- You can also list the following motherboard as working with no
- problems:
-
- AIR 54CDP motherboard / EISA/PCI / onboard aic7870 / dual P120 /
- Redhat 5.0 (2.0.32 and 2.0.33 kernels)
-
- However, Jon Lewis adds the following comments: Has Chris tried
- putting multiple PCI cards in these motherboards...say a SCSI card
- and a network card? Due to BIOS brain damage, the 54CDP insists on
- having all PCI devices share a single IRQ. AIR lead me on for 2
- weeks and then said "there have been / will be no updates for your
- board". This means I can't have Tulip network cards and PCI SCSI
- without hacking the drives (which seems to work, though I was
- cautioned not to do it).
-
-
- ╖ HP XU 6/200 (Jean-Francois Micouleau)
-
- Works with 2.0 and 2.1 kernels. Some problems under high network
- load with 2.0.x kernel. Works under 2.1.78 with Ingo Molnar IO-APIC
- patch.
-
-
- ╖ Elitegroup P6FX2-A (Benedikt Heinen)
-
- Had this mainboard running with ONE PPro on it for several months,
- and since about a year, it's running without problems with TWO PPro
- 200MHz. The only crashes this machine ever experienced were before
- Leonard Zubkoff's deadlock-patches for Linux 2.0.30... ;)
-
- Elitegroup P6FX2-A / ISA/PCI / Dual PPro200 / Debian "hamm"
-
-
- ╖ Elitegroup P6LX2-A (Ulf Nielsen)
-
- This motherboard works with a post 2.1.99 kernel (100 and 101
- tested). All kernels pre 2.1.100 does not work on P6LX2-A.
-
-
- ╖ QDI P6I440LX/DP "Legend IV" (Tony Kocurko)
-
- The QDI P6I440LX/DP "Legend IV" dual Pentium II motherboard now
- boots properly with Linux-SMP. As of 0326 hours on 15 April, there
- is a necessary BIOS upgrade to version 1.6 at the Canadian site
- (http://www.qdi.ca). Previously, the Legend IV would bring up the
- first processor normally, but the /proc/cpuinfo file would show the
- second processor barely alive. The BIOS update seems to have
- corrected the problem.
-
-
- ╖ Asus P2L97-DS
-
- ASUS P2L97-DS works great under Linux-SMP (from Richard Jelinek).
-
-
- ╖ Asus P/I-P65UP5 (Jukka Tainio)
-
- I have run it with both 2x233 MMX Pentium and 2x200 PPRO. Both
- boards are in heavy www-proxy use and work without problems. Kernel
- version is 2.0.33.
-
-
- ╖ Asus P2B-DS (Ulf Rompe)
-
- Asus P2B-DS is working without problems. It runs with two PII-266,
- the onboard Adaptec 7890 (Patch from
- ftp://ftp.dialnet.net/pub/linux/aic7xxx/ required) in U2W mode, an
- Elsa Victory Erazor AGP an some obscure other hardware.
-
- The only thing missing is support for the new SMP-aware APM, but
- that is a general Linux problem.
-
-
-
- ╖ Micronics W6-LI
-
- From Mark Garlanger: 2.0.33 is very stable on the board, and the
- 2.1.xx kernels are improving. People have also been able to run
- other OSes including Windows NT(yuck...), FreeBSD-SMP, Rhapsody.
-
-
- ╖ SuperMicro
-
- Thomas Schenk is running about 80 SMP machines with SuperMicro
- motherboards without any problem. Kernel used: 2.0.33. No "bad
- bogomips" problem for him (was I [david] wrong about this ?).
-
-
- ╖ N440BX Server Board
-
- Moni Hollmann using a N440BX Server Board unter Linux (SMP) and no
- Problems occured so far. RAM: 256 MB, 2x Pentium/II- 350
- (Deschutes).
-
-
-
- 4.4. How to obtain maximum performance ? (not strictly SMP related)
-
-
-
-
-
- 4.4.1. From disks
-
-
- Some useful information pointed out by Robert G. Brown, one of the
- bigest contributor to this FAQ.
-
-
- Date: Wed, 8 Jul 1998 09:12:41 +1000 (EST)
- From: "Andrew Mc.Ghee" <mcghee@mech.uq.edu.au>
-
- I believe there was work done on this at - (I've got the report sitting in
- front of me, but there is no http reference, must have used UNIX netscape
- to print it, but it was done by Eric Hendriks at cesdis.gsfc.nasa.gov)
-
- Distilling it down, the paper presents
-
- Interface Disk Arrangement Cpu Usage Throughput
- =============================================================
- IDE-PIO 4 Disks / 2 Channels 98% read 7.53 MB/s
- 58% write 4.84 MB/s
- IDE-DMA 4 Disks / 2 Channels 34% read 11.78 MB/s
- 57% write 9.21 MB/s
- IDE-DMA 2 Disks / 2 Channels 31% ----- 9.87 MB/s
- SCSI 4 Disks / 1 Channel 53% read 13.41 MB/s
- 92% write 11.93 MB/s
- SCSI 4 Disks / 2 Channels 77% read 21.99 MB/s
- 97% write 13.44 MB/s
-
- The test where done using RAID0 via the md system, but of course the test
- involved using a large, single contiguous data transfer.
-
- Andrew Mc.Ghee
- Mechanical Engineering
- University of Queensland
-
-
-
-
- 5. Useful pointers
-
-
- 5.1. Various
-
-
- ╖ Parallel Processing using Linux
- <http://yara.ecn.purdue.edu/~pplinux/>
-
- ╖ Linux Parallel Processing HOWTO
- <http://yara.ecn.purdue.edu/~pplinux/PPHOWTO/pphowto.html>
-
- ╖ (outdated) Linux SMP home page
- <http://www.uk.linux.org/SMP/title.html>
-
- ╖ linux-smp mailing list
-
- To subscribe, send subscribe linux-smp in the message body at
- majordomo@vger.rutgers.edu
-
- To unsubscribe, send unsubscribe linux-smp in the message body at
- majordomo@vger.rutgers.edu
-
- Linux SMP archives <http://www.linuxhq.com/lnxlists/linux-smp/>
-
- Linux SMP archives at progressive-comp.com <http://www.progressive-
- comp.com/Lists/?l=linux-smp&r=1&w=2#linux-smp>
-
- ╖ pthread library made by Xavier Leroy
- <http://pauillac.inria.fr/~xleroy/linuxthreads/>
-
- ╖ Motherboards rumored to run Linux SMP <http://www.nlug.org/smp/>
-
- ╖ procps <http://www.cs.inf.ethz.ch/~rauch/procps.html>
-
- ╖ xosview <http://lore.ece.utexas.edu/~bgrayson/xosview.html>
-
- ╖ SMP Performance of Linux [OUTDATED]
- <http://www.interlog.com/~mackin/linux-smp.html>
-
- ╖ CESDIS Linux Ethernet device drivers site
- <http://cesdis.gsfc.nasa.gov/linux/drivers/>
-
-
-
- 5.2. Multithreaded programs and library
-
-
- ╖ Linux Threads FAQ <http://linas.org/linux/threads-faq.html>
-
- ╖ Multithreaded programs on linux
- <http://www.e.kth.se/~e94_bek/mthread.html>
-
- ╖ Pentium Pro Optimized BLAS and FFTs for Intel Linux
- <http://www.cs.utk.edu/~ghenry/distrib/> (not available right now,
- but a dual proc library is planned for 5/27/98, see Blas News
- <http://www.cs.utk.edu/~ghenry/distrib/blasnews> for details)
-
- ╖ Mesa library <http://www.ssec.wisc.edu/~brianp/Mesa.html> (with
- experimental multi-threading)
-
- ╖ Parallel plugins for The GIMP
- <http://nemo.physics.ncsu.edu/~briggs/gimp/index.html>
-
-
-
-
- 5.3. SMP specific patches
-
-
- ╖ Forissier kernel patches <http://www-
- isia.cma.fr/~forissie/smp_kernel_patch/>
-
- ╖ Patch for a bug in the 440FX chipset
- <http://nemo.physics.ncsu.edu/~briggs/vfix.html>
-
- ╖ MTRR patch (latest version: 1.9)
- <http://www.atnf.csiro.au/~rgooch/kernel-patches.html>
-
- ╖ PSET - Processor Sets for the Linux kernel
- <http://isunix.it.ilstu.edu/~thockin/pset/>
-
-
-
-
- 5.4. ( Sumit Roy ) Parallelizing/Optimizing Compilers for 586/686
- machines
-
-
- ╖ Pentium Compiler Group <http://www.goof.com/pcg/> creators of pgcc
-
- ╖ Absoft <http://www.absoft.com/> , Fortran 90 and Fortran 77
- compilers
-
- ╖ The Portland Group, Inc. <http://www.pgroup.com/>, supports the
- OpenMP <http://www.openmp.org> standard for Fortran parallelization
- on Linux
-
- ╖ Pacific-Sierra Research Corporation <http://www.psrv.com/>, has a
- free F90 compiler for Linux, as well as parallelizing compilers for
- SMP Linux
-
- ╖ Applied Parallel Research <http://s006.infomall.org/index.html>,
- currently have parallelizing compilers for WinNT
-
-
-
- 6. Glossary
-
-
- ╖ SMP Symetric Multi-Processors
-
- ╖ APIC Advanced Programmable Interrupt Controler
-
- ╖ thread A thread is a processor activity in a process. The same
- process can have multiple threads. Those threads share the process
- address space and can therefore share data.
-
- ╖ pthread Posix thread, threads defined by the Posix standard.
-
-
-
- 7. List of contributors
-
- Many thanks to those who help me to maintain this FAQ :
-
-
- ╖ Tigran A. Aivazian
-
- ╖ Niels Ammerlaan
-
- ╖ Emil Briggs
-
- ╖ Robert G. Brown
-
- ╖ Samuel S. Chessman
-
- ╖ Alan Cox
-
- ╖ Andrew Crane
-
- ╖ Mark Duguid
-
- ╖ Jocelyne Erhel
-
- ╖ Byron Faber
-
- ╖ Mark Garlanger
-
- ╖ hASCII
-
- ╖ Claus-Justus Heine
-
- ╖ Benedikt Heinen
-
- ╖ Florian Hinzmann
-
- ╖ Moni Hollmann
-
-
- ╖ Robert Hyatt
-
- ╖ Richard Jelinek
-
- ╖ Tony Kocurko
-
- ╖ Doug Ledford
-
- ╖ Hank Leininger
-
- ╖ Cameron MacKinnon
-
- ╖ Joel Marchand
-
- ╖ David Maslen
-
- ╖ Chris Mauritz
-
- ╖ Jean-Francois Micouleau
-
- ╖ Ulf Nielsen
-
- ╖ Jakob Oestergaard
-
- ╖ Ulf Rompe
-
- ╖ Jean-Michel Rouet
-
- ╖ Ralf BΣchle
-
- ╖ Volker Reichelt
-
- ╖ Sean Reifschneider
-
- ╖ Sumit Roy
-
- ╖ Thomas Schenk
-
- ╖ Szakacsits Szabolcs
-
- ╖ Jukka Tainio
-
- ╖ Simen Timian Thoresen
-
- ╖ El Warren
-
- ╖ Christopher Allen Wing
-
- ╖ Leonard N. Zubkoff
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-