PC World 1999 August

home *** CD-ROM | disk | FTP | other *** search

/ PC World 1999 August / PCWorld_1999-08_cd.bin / doc / FAQ / txt / SMP-FAQ < prev next >

Wrap

Text File | 1999-04-09 | 41.1 KB | 1,390 lines

Linux SMP FAQ David MentrΘ, David.Mentre@irisa.fr v0.47, 20 november 1998 This FAQ review main issues (and I hope solutions) related to SMP con figuration under Linux. ______________________________________________________________________ Table of Contents 1. Introduction 2. What's new ? 3. Questions related to any architectures 3.1 Kernel side 3.2 User side 3.3 SMP Programming 3.3.1 Parallelization methods 3.3.2 The C Library 3.3.3 Languages, Compilers and debuggers 3.3.4 Other libraries 3.3.5 Other points about SMP Programming 4. Intel architecture specific questions 4.1 Why it doesn't work on my machine? 4.2 Possible causes of crash 4.3 Motherboard specific information 4.3.1 Motherboards with known problems 4.3.2 Motherboards with NO known problems 4.4 How to obtain maximum performance ? (not strictly SMP related) 4.4.1 From disks 5. Useful pointers 5.1 Various 5.2 Multithreaded programs and library 5.3 SMP specific patches 5.4 Parallelizing/Optimizing Compilers for 586/686 machines ( 6. Glossary 7. List of contributors ______________________________________________________________________ 1. Introduction Linux can work on SMP (Symetric Multi-Processors) machines. SMP support has started with the 2.0 family and has been improved in the 2.1 (future 2.2) saga. FAQ maintained by David MentrΘ (David.Mentre@irisa.fr). The latest edition of this FAQ can be found at ╖ http://www.irisa.fr/prive/mentre/smp-faq/ (France) ╖ http://www.phy.duke.edu/brahma/smp-faq/ (USA) If you want to contribute to this FAQ, I would prefer a diff against the SGML version <http://www.irisa.fr/prive/mentre/smp-faq/smp- faq.sgml> of this document, but any remarks (in plain text) will be greatly appreciated. If you send me an email about this FAQ, please include a tag like [Linux SMP FAQ] in the Subject: field of your e- mail. It helps me automatically sort mails (and you will have a faster reply ;)). This FAQ is an improvement of a first draft made by Chris Pirih. All information contained in this FAQ is provided "as is." All warranties, expressed, implied or statutory, concerning the accuracy of the information of the suitability for any particular use are hereby specifically disclaimed. While every effort has been taken to ensure the accuracy of the information contained in this FAQ, the authors assume(s) no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein. 2. What's new ? v0.47, 20 november 1998 ╖ Added that 2.0.36 as the MTRR patch (related to the BogoMips problem) v0.46, 10 november 1998 ╖ Update about Epox KP6-LS motherboards v0.45, 25 october 1998 ╖ Corrected an error regarding /proc/stat file ╖ Added a pointer to CESDIS Ethernet Linux Drivers site v0.44, 14 october 1998 ╖ Updated the link to the web page: Motherboards rumored to run Linux SMP ╖ Added Jakob explanation how to time SMP systems with 2.0 kernels v0.43, 9 september 1998 ╖ Updated first question in section 3.1 ╖ Updated mt-Mesa link: multi-threaded is now included as experimental in the Mesa distribution v0.42, 2 september 1998 ╖ Minor cosmetic update in sect 3.3 ╖ Two links (multithreaded Mesa and SMP performance) marked outdated ╖ Updated the item about threads and exceptions in C++ (sect 3.3) v0.41, 1 september 1998 ╖ Added a major section: "3.3 SMP Programming" written by Jakob ╪stergaard ╖ moved some item of section "3.2 User side" in sect 3.3 v0.40, 27 august 1998 ╖ Updated section 3.1, item 7: processor affinity v0.39, 27 august 1998 ╖ Updated needed Award BIOS version for Tyan motherboards (hASCII) ╖ Added an item on IRQ in the crash section (me and hASCII) ╖ Added good support of Asus P2B-DS (Ulf Rompe) ╖ Added another smp-list archive in pointer section (Hank Leininger) v0.38, 8 august 1998 ╖ Added a pointer to the Linux Threads FAQ v0.37, 30 July 1998 ╖ Emil Briggs is working on parallel plugins for Gimp (see "Is there any threaded programs or library?", sect. "User side") v0.36, 26 July 1998 ╖ Thanks to Jakob ╪stergaard, two changes in "Possible causes of Crash" ╖ Changed 2.0.33 to 2.0.35 (latest stable) ╖ Added a "BIOS related causes of failure" v0.35, 14 July 1998 ╖ Added N440BX Server Board in Motherboards with NO problems ╖ Added a succes story for GigaByte motherboard with BIOS upgrade ╖ Added a "How to obtain maximum performance ?" section (waiting for your contributions ;) v0.34, 10 june 1998 ╖ Added a "Parallelizing/Optimizing Compilers for 586/686 machines" section in section "Useful Pointers", thanks to Sumit Roy ╖ Corrected a mispelling, "Asus P/I-UP5" is in fact "Asus P/I- P65UP5" v0.33, 3 june 1998 ╖ Yet another success story for a GigaByte DLX Motherboard. ╖ A tip for Tyan motherboards, disable the "DRAM Fast Leadoff" BIOS option v0.32, 27 may 1998 ╖ Asus P/I-UP5 added in the motherboard-with-NO-problem section v0.31, 18 may 1998 ╖ Elitegroup P6LX2-A works with 2.1.100 and 101 ╖ Bugs should be reported to linux-smp@vger.rutgers.edu v0.30, 12 may 1998 ╖ SuperMicro is now in the motherboard-with-NO-problem section v0.29, 11 may 1998 ╖ A success story for a GigaByte 686 motherboard with 2.1.101 ╖ Added a new item in the "User Side" section: "Is there any threaded programs or library?" ╖ OpenGL Mesa library is beeing multithreaded. Cool! See the new section for details. v0.28, 09 may 1998 ╖ A US mirror of this FAQ is now available (see Introduction) ╖ Merge of the two confusing Gigabyte 686 entries v0.27, 05 may 1998 ╖ New info for the Adaptec and TekRam drivers ╖ Micronics W6-LI motherboard works under SMP 3. Questions related to any architectures 3.1. Kernel side 1. Does Linux support multi-threading? If I start two or more processes, will they be distributed among the available CPUs? Yes. Processes and kernel-threads are distributed among processors. 2. What kind of architectures are supported in SMP? From Alan Cox: SMP is supported in 2.0 on the hypersparc (SS20, etc.) systems and Intel 486, Pentium or higher machines which are Intel MP1.1/1.4 compliant. Richard Jelinek adds: right now, systems have been tested up to 4 CPUs and the MP standard (and so Linux) theoretically allows up to 16 CPUs. SMP support for UltraSparc, SparcServer, Alpha and PowerPC machines is in progress in 2.1.x. From Ralf BΣchle: MIPS, m68k and ARM does not support SMP; the latter two probly won't ever. That is, I'm going to hack on MIPS-SMP as soon as I get a SMP box ... 3. How do I make a Linux SMP kernel? Uncomment the SMP=1 line in the main Makefile (/usr/src/linux/Makefile). AND enable "RTC support" (from Robert G. Brown). Note that inserting RTC support actually doesn't afaik prevent drift, but according to a discussion [Robert G. Brown] remember from a year ago or so it can prevent lockup when the clock is read at boot time. A note from Richard Jelinek says also that activating the Enhanced RTC is necessary to get the second CPU working (identified) on some original Intel Mainboards. AND do NOT enable APM! APM and SMP are not compatible, and your system will almost certainly (or at least probably ;)) crash under boot if APM is enabled (Jakob Oestergaard). Alan Cox confirms this : 2.1.x turns APM off for SMP boxes. Basically APM is undefined in the presence of SMP systems, and anything could occur. You must rebuild all your kernel and kernel modules when changing to and from SMP mode. Remember to make modules and make modules_install (from Alan Cox). 4. How do I make a Linux non-SMP kernel? Comment the SMP=1 line in the Makefile (and not set SMP to 0). You must rebuild all your kernel and kernel modules when changing to and from SMP mode. Remember to make modules and make modules_install. 5. How can I tell if it worked? cat /proc/cpuinfo Typical output (dual PentiumII): ______________________________________________________________________ processor : 0 cpu : 686 model : 3 vendor_id : GenuineIntel stepping : 3 fdiv_bug : no hlt_bug : no fpu : yes fpu_exception : yes cpuid : yes wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic 11 mtrr pge mca cmo v mmx bogomips : 267.06 processor : 1 cpu : 686 model : 3 vendor_id : GenuineIntel stepping : 3 fdiv_bug : no hlt_bug : no fpu : yes fpu_exception : yes cpuid : yes wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic 11 mtrr pge mca cmo v mmx bogomips : 267.06 ______________________________________________________________________ 6. What is the status of converting the kernel toward finer grained locking and multithreading? 2.1.x has signal handling, interrupts and some I/O stuff fine grain locked. The rest is gradually migrating. All the scheduling is SMP safe 7. Does Linux SMP support processor affinity? Standard kernel No and Yes. There is no way to force a process onto specific CPU's but the linux scheduler has a processor bias for each process, which tends to keep processes tied to a specific CPU. Patch Yes. Look at PSET - Processor Sets for the Linux kernel <http://isunix.it.ilstu.edu/~thockin/pset/>: The goal of this project is to make a source compatible and functionally equivalent version of pset (as defined by SGI - partially removed from their IRIX 6.4 kernel) for Linux. This enables users to determine which processor or set of processors a process may run on. Possible uses include forcing threads to seperate proces sors, timings, security (a `root' only CPU?) and probably more. 8. Where should one report SMP bugs to? Please report bugs to linux-smp@vger.rutgers.edu. 3.2. User side 1. Do I really need SMP? If you have to ask, you probably don't. :) 2. How does one display mutiple cpu performance? Thanks to Samuel S. Chessman, here is some useful utilities: Character based: http://www.cs.inf.ethz.ch/~rauch/procps.html Basically, it's procps v1.12.2 (top, ps, et. al.) and some patches to support SMP. Graphic: xosview-1.5.1 supports SMP. And kernels above 2.1.85 (included) the cpuX entry in /proc/stat file. The official homepage for xosview is: http://lore.ece.utexas.edu/~bgrayson/xosview.html The various forissier's kernel patches are at: http://www- isia.cma.fr/~forissie/smp_kernel_patch/ 3. How can I enable more than 1 process for my kernel compile? use: ___________________________________________________________________ # make [modules|zImage|bzImages] MAKE="make -jX" where X=max number of processes. WARNING: This won't work for "make dep". ___________________________________________________________________ With a 2.1.x like kernel, see also the file /usr/src/linux/Documentation/smp for specific instruction. BTW, since running multiple compilers allows a machine with sufficient memory to use use the otherwise wasted CPU time during I/O caused delays make MAKE="make -j 2" -j 2 actually even helps on uniprocessor boxes (from Ralf BΣchle). 4. Why the time given by the time command is false ? (from Joel Marchand) In the 2.0 series, the result given by the time command is false. The sum user+system is right *but* the spreading between user and system time is false. More precisely: "The explanation is, that all time spent in processors other than the boot cpu is accounted as system time. If you time a program, add the user time and the system time, then you timing will be almost right, except for also including the system time that is correctly accounted for" (Jakob ╪stergaard). This bug in corrected in 2.1 series. 5. How will my application perform under SMP? Look at SMP Performance of Linux [OUTDATED] <http://www.interlog.com/~mackin/linux-smp.html> which gives useful hints how to bench a specific machine (from a post made by Cameron MacKinnon). 3.3. SMP Programming Section made by Jakob ╪stergaard. This section is intended to outline what works, and what doesn't, when it comes to programming multi-threaded software for SMP Linux. 3.3.1. Parallelization methods 1. POSIX Threads 2. PVM / MPI Message Passing Libraries 3. fork() -- Multiple processes Since both fork() and PVM/MPI processes usually does not share memory, but either communicate by means of IPC or a messaging API, they will not be described further in this section. They are not very specific to SMP, since they are used just as much - or more - on uniprocessor computers, and clusters thereof. Only POSIX Threads provide us with multiple threads sharing ressources like - especially - memory. The exact same thing that makes a SMP machine special, by allowing many processors to share their memory. To use both (or more ;) processors of an SMP, use a kernel-thread library. A good library is the LinuxThreads, a pthread library made by Xavier Leroy <http://pauillac.inria.fr/~xleroy/linuxthreads/> which is now integrated with glibc2 (aka libc6). Multi-threading has never been really popular in the UN*X world though. For some reason, applications requiring multiple processes or threads, has mostly been written using fork(). Therefore, when using the thread approach, one runs into problems of incompatible (not thread-ready) libraries, compilers, and debuggers. GNU/Linux is no exception to this. Hopefully the next few sections will sched a little light over what is currently possible, and what is not. 3.3.2. The C Library Older C libraries are not thread-safe. It is very important that you use GNU LibC (glibc), also known as libc6. Earlier versions are ofcourse possible to use, but it will cause you much more trouble than upgrading your system will, well probably :) If you want to use GDB to debug your programs, you should get the patches or SRPMS available for glibc at http://odin.appliedtheory.com/debug_thread_rpms/RedHat_5.1/ which will allow GDB to work with threads. It is not nessecary to patch glibc in any way just to make it work with threads. If you do not need to debug the software (this could be true for all machines that are not development workstations), there is no need to patch glibc. 3.3.3. Languages, Compilers and debuggers There is a wealth of programming languages available for GNU/Linux, and many of them can be made to use threads one way or the other (some languages like Ada and Java even have threads as primitives in the language). This section will, however, currently only describe C and C++. If you have experience in SMP Programming with other languages, please enlighten us. GNU C and C++, as well as the EGCS C and C++ compilers work with the thread support from the standard C library (glibc). There are however a few issues: 1. When compiling C or C++, use the -D_REENTRANT define in the compiler command line. This is necessary to make certain error- handling functions work. 2. When using C++, If two threads throw exceptions concurrently, the program will segfault. The compiler does not generate thread-safe exception code. The workaround is to put a pthread_mutex_lock(&global_exception_lock) in the constructor(s) of every class you throw(), and to put the corresponding pthread_mutex_unlock(...) in the destructor. It's ugly, but it works. This solution was given by Markus Ferch. The GNU Debugger GDB as of version 4.17, is incapable of handling threads, at least in any sensible way. There is however a patched version available at http://odin.appliedtheory.com/debug_thread_rpms/RedHat_5.1/, for RedHat Linux at least. It's probably quite easy to use the standalone patches on any other GNU/Linux distribution. Note that you must also patch your glibc. Note that core-dumps are of no use, when using multiple threads. Somehow the core dump is attached to one of the currently running threads, and not to the program as a whole. Therefore, whenever you are debugging anything, run it from the debugger. Hint: If you have a thread running haywire, like eating 100% CPU time, and you cannot seem to figure out why, here is a nice way to find out what's going on: Run the program straight from the shell, no GDB. Make the thread go haywire. Use top to get the PID of the process. Run GDB like gdb program pid. This will make GDB attach itself to the process with the PID you specified, and stop the thead. Now you have a GDB session with the offending thread, and can use bt and the likes to see what is happening. 3.3.4. Other libraries ElectricFence: This library is not thread safe. It should however be possible to make it work in SMP environments, by inserting mutex locks in the ElectricFence code. 3.3.5. Other points about SMP Programming 1. Where can I found more information about parallel programming? Look at the Linux Parallel Processing HOWTO <http://yara.ecn.purdue.edu/~pplinux/PPHOWTO/pphowto.html> Lots of useful information can be found at Parallel Processing using Linux <http://yara.ecn.purdue.edu/~pplinux/> Look also at the Linux Threads FAQ <http://linas.org/linux/threads- faq.html> 2. Is there any threaded programs or library? Yes. For programs, you should look at: Multithreaded programs on linux <http://www.e.kth.se/~e94_bek/mthread.html> (I love hyperlinks, did you know that ? ;)) As far as library are concerned, there is: OpenGL Mesa library Thanks to David Buccarelli, Andreas Schiffler and Emil Briggs, it exists in a multithreaded version (right now [1998-05-11], there is a working version that provides speedups of 5-30% on some OpenGL benchmarks). The multithreaded stuff is now included in the regular Mesa distribution as an experimental option. For more information, look at the Mesa library <http://www.ssec.wisc.edu/~brianp/Mesa.html> BLAS Pentium Pro Optimized BLAS and FFTs for Intel Linux <http://www.cs.utk.edu/~ghenry/distrib/> Multithreaded BLAS routines are not available right now, but a dual proc library is planned for 1998-05-27, see Blas News <http://www.cs.utk.edu/~ghenry/distrib/blasnews> for details. The GIMP Emil Briggs, the same guy who is involved in multithreaded Mesa, is also working on multithreaded The GIMP plugins. Look at http://nemo.physics.ncsu.edu/~briggs/gimp/index.html for more info. 4. Intel architecture specific questions 4.1. Why it doesn't work on my machine? 1. Can I use my Cyrix/AMD/non-Intel CPU in SMP? Short answer: no. Long answer: Intel claims ownership to the APIC SMP scheme, and unless a company licenses it from Intel they may not use it. There are currently no companies that have done so. (This of course can change in the future) FYI - Both Cyrix and AMD support the non- proprietary OpenPIC SMP standard but currently there are no motherboards that use it. 2. Why doesn't my old Compaq work? Put it into MP1.1/1.4 compliant mode. 3. Why doesnt my ALR work? From Robert Hyatt : ALR Revolution quad-6 seems quite safe, while some older revolution quad machines without P6 processors seem "iffy"... 4. Why does SMP go so slowly? or Why does one CPU show a very low bogomips value while the first one is normal? From Alan Cox: If one of your CPU's is reporting a very low bogomips value the cache is not enabled on it. Your vendor probably provides a buggy BIOS. Get the patch to work around this or better yet send it back and buy a board from a competent supplier. Another work-around is to use the 2.0.36 kernel. It contains the MTRR patch which should solve this problem (select option "Handle buggy SMP BIOSes with bad MTRR setup" in the "General setup" menu). 5. I've heard IBM machines have problems Some IBM machines have the MP1.4 bios block in the EBDA, allowed but not supported by <2.1.80. Please update to the right kernel. There is an old 486SLC based IBM SMP box. Linux/SMP requires hardware FPU support. 6. Is there any advantage of Intel MP 1.4 over 1.1 specification? Nope (according to Alan :) ), 1.4 is just a stricker specs of 1.1. 7. Why does the clock drift so rapidly when I run linux SMP? This is known problem with IRQ handling and long kernel locks in the 2.0 series kernels. Consider upgrading to a later 2.1 kernel (not garenteed to work). From Jakob Oestergaard: Or, consider running xntpd. That should keep your clock right on time. (I think that I've heard that enabling RTC in the kernel also fixes the clock drift. It works for me! but I'm not sure whether that's general or I'm just being lucky) 8. Why are my CPU's numbered 0 and 2 instead of 0 and 1 (or some other odd numbering)? The CPU number is assigned by the MB manufacturer and doesn't mean anything. Ignore it. 9. My SMP system is locking up all the time. Black screen, nothing in the logs. Help! If you're running a 2.0 kernel, consider upgrading to later 2.0.32+ kernels or apply Leonard Zubkoff's deadlock patch. If you still have deadlocks, apply Ingo Molnar's deadlock detection patch and post the results (against your System.map) to linux-smp or linux- kernel. You might also consider running a 2.1 kernel. 4.2. Possible causes of crash You'll find in this section some possible reasons for a crash of an SMP machine (credits are due to Jakob ╪stergaard for this part). As far as I (david) know, theses problems are Intel specific. ╖ Cooling problems From Ralf BΣchle: [Related to case size and fans] It's important that the air is flowing. It of course can't where cables etc. are preventing this like in too small cases. On the other side I've seen oversized cases causing big problems. There are some tower cases on the market that actually are worse for cooling than desktops. In short, the right thing is thinking about aerodynamics in the case. Extra cases for hot peripherals are usefull as well. ╖ Bad memory Don't buy too cheap RAM and don't use mixed RAM modules on a motherboard that is picky about it. Especially Tyan motherboards are known to be picky about RAM speed (see the Tyan paragraph below for a possible solution). ╖ Bad combination of different stepping CPUs Check /proc/cpuinfo to see that your CPUs are same stepping. ╖ You are running 2.0.35 aren't you ? If you run 2.0.31 or 2.1.xx you can't be sure that SMP is stable. 2.0.35 is the right kernel for a production system. 2.1.xx kernels perform better, but they are development releases and should NOT be considered stable! ╖ If your system is unstable, then DON'T overclock it! ...and even if it is stable, DON'T overclock. From Ralf BΣchle: Overclocking causes very subtile problems. I have a nice example, one of my overclocked old machines misscomputes a couple of pixels of a 640 x 400 fractal. The problem is only visible when comparing them using tools. So better say never, nuncas, jamais, niemals overclock. ╖ 2.0.x kernel and fast ethernet (from Robert G. Brown) 2.0.X kernels on high performance fast ethernet systems have significant (and known) problems with a race/deadlock condition in the networking interrupt handler. The solution is to get the latest 100BT development drivers from CESDIS Linux Ethernet device drivers site <http://cesdis.gsfc.nasa.gov/linux/drivers/> (ones that define SMPCHECK). ╖ A bug in the 440FX chipset (from Emil Briggs) If you had a system using the 440FX chipset then your problem with the lockups was possibly due to a documented errata in the chipset. Here is a reference References: Intel 440FX PCIset 82441FX (PMC) and 82442FX (DBX) Specification Update. pg. 13 http://www.intel.com/design/pcisets/specupdt/297654.htm The problem can be fixed with a BIOS workaround (Or a kernel patch) and in fact David Wragg wrote a patch that's included with Richard Gooch's MTTR patch. For more information and a fix look here: http://nemo.physics.ncsu.edu/~briggs/vfix.html ╖ DONT run emm386.exe before booting linux SMP From Mark Duguid, dumb rule #1 with W6LI motherboards. ;) ╖ If the machine reboots/freezes after a while, there can be two good BIOS + memory related reasons (from Jakob ╪stergaard) ╖ If the BIOS has settings like "memory hole at 16M" and/or "OS/2 memory > 64MB", try disabling them both. Linux does not always react well with theese options. ╖ If you have more than 64 MB of memory in the machine, and you specified the exact number manually in the LILO configuration, you should specify one MB less than you actually have in the machine. If you have 128 MB, you lilo.conf line looks like: append="mem=127M" ╖ Be aware of IRQ related problems Sometime, some cards are not recognized or can trigger IRQ conflicts. Try shuffle cards on slots in different ways. Contributed by hASCII : removing an " append="hisax=9,2,3"" line in lilo.conf allowed using a kernel from the 2.1.xx series with activated ISDN + Hisax support. Kernels from the 2.0.xx series doesn't make problems like this. Some hardware is also known to cause problems. This includes: ╖ Adaptec SCSI controllers Latest news (05 may 1998): The latest version of this driver (5.0.13 at this writting) should be SMP safe (both in 2.0 and 2.1 kernels). Doug had verified it through code review and intensive tests (on a dual PII system). This said, some machines still have problems. Doug has kindly given some explanations: ╖ Doug: The simple fact of the matter is that this version of the aic7xxx driver places a larger demand on DMA transfers than the old driver did, especially for for smaller DMA transactions. So, the point should be made that this version of the driver *is* more demanding on your system than either the 4.1 or 4.1.1 driver, and that certain marginal systems may get hit by this problem (such as my own P133 system is). ╖ In 2.1 kernels, they are still problems (with IRQ) with the kernel (2.1.99) and they are not related to this specific driver. People are working on this. ╖ 3Com 3c905 cards Some work, some don't. Try disabling busmastering if your system is unstable. 4.3. Motherboard specific information Please note: Some more specific information can be found with the list of Motherboards rumored to run Linux SMP <http://www.nlug.org/smp/> 4.3.1. Motherboards with known problems ╖ GA686DLX (Andrew Crane) Same BIOS related BogoMIPS problem as other motherboards. Solution from Alan Cox: Congratulations, send the bill for your hair damage to the supplier. You have yet another SMP box with faulty bios. There is a patch for 2.0.x on www.uk.linux.org and there are people working on generic MTRR handling for 2.1.x From Claus-Justus Heine: I'm able to boot above MB with 2.1.90 + Ingo's apic + Richard's mtrr patches. From David MentrΘ: the kernel > 2.1.99 have the MTRR patch included and Florian Hinzmann told me that with 2.1.101, both CPUs are detected fine. David Maslen tells: Fortunately I suspect these problems have been fixed, and would like to report my success so that others won't be put off what seems to be a nice motherboard. [...] My system has been running for about 3 days without a lockup (72hrs) Kernel 2.1.103 compiled easily. /proc/cpuinfo says the cpus are both there. This was also true of 2.0.33. Other possible solution: BIOS upgrade (see http://www.giga- byte.com). (14 July 1998)Volker Reichelt confirms that with a BIOS upgrade from v1.14 to v1.20 everything seems to work fine. ╖ EPoX KP6-LS (Christopher Allen Wing, 16 march 1998) It appears to have the same BIOS related BogoMIPS problem as other motherboards. (one CPU only gives about 3 BogoMIPS, the other gives the full amount) All 2.0.x kernels lock up soon after booting, late 2.1.x kernels run slowly but don't seem to lock up. There is no BIOS upgrade available (yet). I wrote the manufacturer but have not received a reply. (update, 10 november 1998) First of all, there are 2 revisions on the KP6-LS: (1) Revision 2 (production release, has a power-leak on the second processor connector - To fix this problem, contact Epox support and they will guide you for further assistence. - For some boards it is needed to replace the entire board. Cause the leak is corrected in the middle of this revision). (2) Revision 3 (production release). For both the revisions is a patch avaliable from their site (http://www.epox.com/support/bios.html). Please make sure that Epox has been contacted about the bios upgrade before applying. After applying the patch, kernel 2.0.35 and all other expirimental kernels from 2.1.110 up to 2.1.125 report a correct BogoMIPS index. (Niels Ammerlaan) ╖ Tyan Tyan motherboards are known to be picky about RAM speed (Jakob Oestergaard). From Doug Ledford about the onboard aic-7895 SCSI controller (for which he wrote the driver): "BTW, make sure you have at least BIOS version 1.16 on that Tyan motherboard. The 1.15 and below BIOS versions have a bug relating to IRQ allocation for the 7895 SCSI controller" (submitted by Szakacsits Szabolcs). (27 august 1998) From hASCII: If you have a tyan with award bios, the newest version you can get is 1.02. But, as motherboard problems are more close to grayshade than black and white ;), Richard Jelinek (from S.u.S.E.) tells that they sell/sold several Tyan Titan Pro (Dual-PPro) Motherboards. They work perfectly with SMP. (3 june 1998) maybe an explanation found by Leonard Zubkoff and told tp me by Sean Reifschneider: The Tyan Titan Pro motherboards with 3.03 BIOS version (dated 10/31/96) has an option labled "DRAM Fast Leadoff", which MUST be disabled. ╖ MS-6114 More details for this motherboard at http://www.msi.com.tw/product/6114/6114.htm <http://www.msi.com.tw/product/6114/6114.htm> Solution: BIOS upgrade Somebody experienced solid hangs (nothing in the log files) under constant load of about 5 running processes within less than 12 hours with AMI BIOS v1.1. v1.4b3 runs without problems. 4.3.2. Motherboards with NO known problems ╖ AIR P6NDP and P6NDI (Leonard N. Zubkoff) My primary production machine is based on an AIR P6NDP and one of my test machines uses a P6NDI. Both seem to be fine motherboards in my experience. The P6NDI BIOS is a little conservative in its programming of the Natoma chipset for 50ns EDO, but a minor tweek to one register in rc.local took care of that. ╖ AIR 54CDP (Chris Mauritz) You can also list the following motherboard as working with no problems: AIR 54CDP motherboard / EISA/PCI / onboard aic7870 / dual P120 / Redhat 5.0 (2.0.32 and 2.0.33 kernels) However, Jon Lewis adds the following comments: Has Chris tried putting multiple PCI cards in these motherboards...say a SCSI card and a network card? Due to BIOS brain damage, the 54CDP insists on having all PCI devices share a single IRQ. AIR lead me on for 2 weeks and then said "there have been / will be no updates for your board". This means I can't have Tulip network cards and PCI SCSI without hacking the drives (which seems to work, though I was cautioned not to do it). ╖ HP XU 6/200 (Jean-Francois Micouleau) Works with 2.0 and 2.1 kernels. Some problems under high network load with 2.0.x kernel. Works under 2.1.78 with Ingo Molnar IO-APIC patch. ╖ Elitegroup P6FX2-A (Benedikt Heinen) Had this mainboard running with ONE PPro on it for several months, and since about a year, it's running without problems with TWO PPro 200MHz. The only crashes this machine ever experienced were before Leonard Zubkoff's deadlock-patches for Linux 2.0.30... ;) Elitegroup P6FX2-A / ISA/PCI / Dual PPro200 / Debian "hamm" ╖ Elitegroup P6LX2-A (Ulf Nielsen) This motherboard works with a post 2.1.99 kernel (100 and 101 tested). All kernels pre 2.1.100 does not work on P6LX2-A. ╖ QDI P6I440LX/DP "Legend IV" (Tony Kocurko) The QDI P6I440LX/DP "Legend IV" dual Pentium II motherboard now boots properly with Linux-SMP. As of 0326 hours on 15 April, there is a necessary BIOS upgrade to version 1.6 at the Canadian site (http://www.qdi.ca). Previously, the Legend IV would bring up the first processor normally, but the /proc/cpuinfo file would show the second processor barely alive. The BIOS update seems to have corrected the problem. ╖ Asus P2L97-DS ASUS P2L97-DS works great under Linux-SMP (from Richard Jelinek). ╖ Asus P/I-P65UP5 (Jukka Tainio) I have run it with both 2x233 MMX Pentium and 2x200 PPRO. Both boards are in heavy www-proxy use and work without problems. Kernel version is 2.0.33. ╖ Asus P2B-DS (Ulf Rompe) Asus P2B-DS is working without problems. It runs with two PII-266, the onboard Adaptec 7890 (Patch from ftp://ftp.dialnet.net/pub/linux/aic7xxx/ required) in U2W mode, an Elsa Victory Erazor AGP an some obscure other hardware. The only thing missing is support for the new SMP-aware APM, but that is a general Linux problem. ╖ Micronics W6-LI From Mark Garlanger: 2.0.33 is very stable on the board, and the 2.1.xx kernels are improving. People have also been able to run other OSes including Windows NT(yuck...), FreeBSD-SMP, Rhapsody. ╖ SuperMicro Thomas Schenk is running about 80 SMP machines with SuperMicro motherboards without any problem. Kernel used: 2.0.33. No "bad bogomips" problem for him (was I [david] wrong about this ?). ╖ N440BX Server Board Moni Hollmann using a N440BX Server Board unter Linux (SMP) and no Problems occured so far. RAM: 256 MB, 2x Pentium/II- 350 (Deschutes). 4.4. How to obtain maximum performance ? (not strictly SMP related) 4.4.1. From disks Some useful information pointed out by Robert G. Brown, one of the bigest contributor to this FAQ. Date: Wed, 8 Jul 1998 09:12:41 +1000 (EST) From: "Andrew Mc.Ghee" <mcghee@mech.uq.edu.au> I believe there was work done on this at - (I've got the report sitting in front of me, but there is no http reference, must have used UNIX netscape to print it, but it was done by Eric Hendriks at cesdis.gsfc.nasa.gov) Distilling it down, the paper presents Interface Disk Arrangement Cpu Usage Throughput ============================================================= IDE-PIO 4 Disks / 2 Channels 98% read 7.53 MB/s 58% write 4.84 MB/s IDE-DMA 4 Disks / 2 Channels 34% read 11.78 MB/s 57% write 9.21 MB/s IDE-DMA 2 Disks / 2 Channels 31% ----- 9.87 MB/s SCSI 4 Disks / 1 Channel 53% read 13.41 MB/s 92% write 11.93 MB/s SCSI 4 Disks / 2 Channels 77% read 21.99 MB/s 97% write 13.44 MB/s The test where done using RAID0 via the md system, but of course the test involved using a large, single contiguous data transfer. Andrew Mc.Ghee Mechanical Engineering University of Queensland 5. Useful pointers 5.1. Various ╖ Parallel Processing using Linux <http://yara.ecn.purdue.edu/~pplinux/> ╖ Linux Parallel Processing HOWTO <http://yara.ecn.purdue.edu/~pplinux/PPHOWTO/pphowto.html> ╖ (outdated) Linux SMP home page <http://www.uk.linux.org/SMP/title.html> ╖ linux-smp mailing list To subscribe, send subscribe linux-smp in the message body at majordomo@vger.rutgers.edu To unsubscribe, send unsubscribe linux-smp in the message body at majordomo@vger.rutgers.edu Linux SMP archives <http://www.linuxhq.com/lnxlists/linux-smp/> Linux SMP archives at progressive-comp.com <http://www.progressive- comp.com/Lists/?l=linux-smp&r=1&w=2#linux-smp> ╖ pthread library made by Xavier Leroy <http://pauillac.inria.fr/~xleroy/linuxthreads/> ╖ Motherboards rumored to run Linux SMP <http://www.nlug.org/smp/> ╖ procps <http://www.cs.inf.ethz.ch/~rauch/procps.html> ╖ xosview <http://lore.ece.utexas.edu/~bgrayson/xosview.html> ╖ SMP Performance of Linux [OUTDATED] <http://www.interlog.com/~mackin/linux-smp.html> ╖ CESDIS Linux Ethernet device drivers site <http://cesdis.gsfc.nasa.gov/linux/drivers/> 5.2. Multithreaded programs and library ╖ Linux Threads FAQ <http://linas.org/linux/threads-faq.html> ╖ Multithreaded programs on linux <http://www.e.kth.se/~e94_bek/mthread.html> ╖ Pentium Pro Optimized BLAS and FFTs for Intel Linux <http://www.cs.utk.edu/~ghenry/distrib/> (not available right now, but a dual proc library is planned for 5/27/98, see Blas News <http://www.cs.utk.edu/~ghenry/distrib/blasnews> for details) ╖ Mesa library <http://www.ssec.wisc.edu/~brianp/Mesa.html> (with experimental multi-threading) ╖ Parallel plugins for The GIMP <http://nemo.physics.ncsu.edu/~briggs/gimp/index.html> 5.3. SMP specific patches ╖ Forissier kernel patches <http://www- isia.cma.fr/~forissie/smp_kernel_patch/> ╖ Patch for a bug in the 440FX chipset <http://nemo.physics.ncsu.edu/~briggs/vfix.html> ╖ MTRR patch (latest version: 1.9) <http://www.atnf.csiro.au/~rgooch/kernel-patches.html> ╖ PSET - Processor Sets for the Linux kernel <http://isunix.it.ilstu.edu/~thockin/pset/> 5.4. ( Sumit Roy ) Parallelizing/Optimizing Compilers for 586/686 machines ╖ Pentium Compiler Group <http://www.goof.com/pcg/> creators of pgcc ╖ Absoft <http://www.absoft.com/> , Fortran 90 and Fortran 77 compilers ╖ The Portland Group, Inc. <http://www.pgroup.com/>, supports the OpenMP <http://www.openmp.org> standard for Fortran parallelization on Linux ╖ Pacific-Sierra Research Corporation <http://www.psrv.com/>, has a free F90 compiler for Linux, as well as parallelizing compilers for SMP Linux ╖ Applied Parallel Research <http://s006.infomall.org/index.html>, currently have parallelizing compilers for WinNT 6. Glossary ╖ SMP Symetric Multi-Processors ╖ APIC Advanced Programmable Interrupt Controler ╖ thread A thread is a processor activity in a process. The same process can have multiple threads. Those threads share the process address space and can therefore share data. ╖ pthread Posix thread, threads defined by the Posix standard. 7. List of contributors Many thanks to those who help me to maintain this FAQ : ╖ Tigran A. Aivazian ╖ Niels Ammerlaan ╖ Emil Briggs ╖ Robert G. Brown ╖ Samuel S. Chessman ╖ Alan Cox ╖ Andrew Crane ╖ Mark Duguid ╖ Jocelyne Erhel ╖ Byron Faber ╖ Mark Garlanger ╖ hASCII ╖ Claus-Justus Heine ╖ Benedikt Heinen ╖ Florian Hinzmann ╖ Moni Hollmann ╖ Robert Hyatt ╖ Richard Jelinek ╖ Tony Kocurko ╖ Doug Ledford ╖ Hank Leininger ╖ Cameron MacKinnon ╖ Joel Marchand ╖ David Maslen ╖ Chris Mauritz ╖ Jean-Francois Micouleau ╖ Ulf Nielsen ╖ Jakob Oestergaard ╖ Ulf Rompe ╖ Jean-Michel Rouet ╖ Ralf BΣchle ╖ Volker Reichelt ╖ Sean Reifschneider ╖ Sumit Roy ╖ Thomas Schenk ╖ Szakacsits Szabolcs ╖ Jukka Tainio ╖ Simen Timian Thoresen ╖ El Warren ╖ Christopher Allen Wing ╖ Leonard N. Zubkoff