/ Source Code 1994 March / Source_Code_CD-ROM_Walnut_Creek_March_1994.iso / unix_c / utils / backprop.tar

< prev

next >

MacOS 8.1

Win98

DOS

view JSON data

view as text

+--------+-------------------------+-------------------------+--------+--------+
|00000000| 62 61 63 6b 70 72 6f 70 | 2f 00 00 00 00 00 00 00 |backprop|/.......|
|00000010| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000020| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000030| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000040| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000050| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000060| 00 00 00 00 20 20 20 37 | 35 35 20 00 20 35 32 37 |....   7|55 . 527|
|00000070| 35 30 20 00 20 20 20 20 | 31 33 20 00 20 20 20 20 |50 .    |13 .    |
|00000080| 20 20 20 20 20 20 30 20 | 20 34 36 37 35 31 37 30 |      0 | 4675170|
|00000090| 33 37 37 20 20 20 35 37 | 31 30 00 20 00 00 00 00 |377   57|10. ....|
|000000a0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000000b0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000000c0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000000d0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000000e0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000000f0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000100| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000110| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000120| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000130| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000140| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000150| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000160| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000170| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000180| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000190| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000001a0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000001b0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000001c0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000001d0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000001e0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000001f0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000200| 62 61 63 6b 70 72 6f 70 | 2f 53 68 61 72 46 69 6c |backprop|/SharFil|
|00000210| 65 2e 48 65 61 64 65 72 | 73 00 00 00 00 00 00 00 |e.Header|s.......|
|00000220| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000230| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000240| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000250| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000260| 00 00 00 00 20 20 20 36 | 34 34 20 00 20 35 32 37 |....   6|44 . 527|
|00000270| 35 30 20 00 20 20 20 20 | 31 33 20 00 20 20 20 20 |50 .    |13 .    |
|00000280| 20 20 20 31 37 37 34 20 | 20 34 36 37 35 31 37 30 |   1774 | 4675170|
|00000290| 33 37 37 20 20 31 31 30 | 30 30 00 20 00 00 00 00 |377  110|00. ....|
|000002a0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000002b0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000002c0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000002d0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000002e0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000002f0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000300| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000310| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000320| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000330| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000340| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000350| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000360| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000370| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000380| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000390| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000003a0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000003b0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000003c0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000003d0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000003e0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000003f0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000400| 3d 3d 3d 20 62 61 63 6b | 70 72 6f 70 2e 73 68 61 |=== back|prop.sha|
|00000410| 72 31 20 3d 3d 3d 3d 3d | 3d 3d 3d 3d 3d 3d 3d 3d |r1 =====|========|
|00000420| 3d 3d 3d 3d 3d 3d 3d 3d | 3d 3d 3d 3d 0a 46 72 6f |========|====.Fro|
|00000430| 6d 20 73 70 61 72 6b 79 | 66 73 21 68 65 72 63 75 |m sparky|fs!hercu|
|00000440| 6c 65 73 21 61 70 70 6c | 65 21 75 73 63 21 63 73 |les!appl|e!usc!cs|
|00000450| 2e 75 74 65 78 61 73 2e | 65 64 75 21 75 75 6e 65 |.utexas.|edu!uune|
|00000460| 74 21 61 6c 6c 62 65 72 | 79 20 4d 6f 6e 20 53 65 |t!allber|y Mon Se|
|00000470| 70 20 31 37 20 30 38 3a | 35 30 3a 34 34 20 50 44 |p 17 08:|50:44 PD|
|00000480| 54 20 31 39 39 30 0a 0a | 50 6f 73 74 69 6e 67 2d |T 1990..|Posting-|
|00000490| 6e 75 6d 62 65 72 3a 20 | 56 6f 6c 75 6d 65 20 31 |number: |Volume 1|
|000004a0| 34 2c 20 49 73 73 75 65 | 20 38 34 0a 53 75 62 6d |4, Issue| 84.Subm|
|000004b0| 69 74 74 65 64 2d 62 79 | 3a 20 44 6f 6e 61 6c 64 |itted-by|: Donald|
|000004c0| 20 54 76 65 74 65 72 20 | 3c 64 72 74 40 63 68 69 | Tveter |<drt@chi|
|000004d0| 6e 65 74 2e 63 68 69 2e | 69 6c 2e 75 73 3e 0a 41 |net.chi.|il.us>.A|
|000004e0| 72 63 68 69 76 65 2d 6e | 61 6d 65 3a 20 62 61 63 |rchive-n|ame: bac|
|000004f0| 6b 2d 70 72 6f 70 2f 70 | 61 72 74 30 31 0a 0a 3d |k-prop/p|art01..=|
|00000500| 3d 3d 20 62 61 63 6b 70 | 72 6f 70 2e 73 68 61 72 |== backp|rop.shar|
|00000510| 32 20 3d 3d 3d 3d 3d 3d | 3d 3d 3d 3d 3d 3d 3d 3d |2 ======|========|
|00000520| 3d 3d 3d 3d 3d 3d 3d 3d | 3d 3d 3d 0a 46 72 6f 6d |========|===.From|
|00000530| 20 73 70 61 72 6b 79 66 | 73 21 68 65 72 63 75 6c | sparkyf|s!hercul|
|00000540| 65 73 21 61 70 70 6c 65 | 21 75 73 63 21 63 73 2e |es!apple|!usc!cs.|
|00000550| 75 74 65 78 61 73 2e 65 | 64 75 21 75 75 6e 65 74 |utexas.e|du!uunet|
|00000560| 21 61 6c 6c 62 65 72 79 | 20 4d 6f 6e 20 53 65 70 |!allbery| Mon Sep|
|00000570| 20 31 37 20 30 38 3a 35 | 30 3a 35 31 20 50 44 54 | 17 08:5|0:51 PDT|
|00000580| 20 31 39 39 30 0a 0a 50 | 6f 73 74 69 6e 67 2d 6e | 1990..P|osting-n|
|00000590| 75 6d 62 65 72 3a 20 56 | 6f 6c 75 6d 65 20 31 34 |umber: V|olume 14|
|000005a0| 2c 20 49 73 73 75 65 20 | 38 35 0a 53 75 62 6d 69 |, Issue |85.Submi|
|000005b0| 74 74 65 64 2d 62 79 3a | 20 44 6f 6e 61 6c 64 20 |tted-by:| Donald |
|000005c0| 54 76 65 74 65 72 20 3c | 64 72 74 40 63 68 69 6e |Tveter <|drt@chin|
|000005d0| 65 74 2e 63 68 69 2e 69 | 6c 2e 75 73 3e 0a 41 72 |et.chi.i|l.us>.Ar|
|000005e0| 63 68 69 76 65 2d 6e 61 | 6d 65 3a 20 62 61 63 6b |chive-na|me: back|
|000005f0| 2d 70 72 6f 70 2f 70 61 | 72 74 30 32 0a 0a 3d 3d |-prop/pa|rt02..==|
|00000600| 3d 20 62 61 63 6b 70 72 | 6f 70 2e 73 68 61 72 33 |= backpr|op.shar3|
|00000610| 20 3d 3d 3d 3d 3d 3d 3d | 3d 3d 3d 3d 3d 3d 3d 3d | =======|========|
|00000620| 3d 3d 3d 3d 3d 3d 3d 3d | 3d 3d 0a 46 72 6f 6d 20 |========|==.From |
|00000630| 73 70 61 72 6b 79 66 73 | 21 68 65 72 63 75 6c 65 |sparkyfs|!hercule|
|00000640| 73 21 61 70 70 6c 65 21 | 75 73 63 21 63 73 2e 75 |s!apple!|usc!cs.u|
|00000650| 74 65 78 61 73 2e 65 64 | 75 21 75 75 6e 65 74 21 |texas.ed|u!uunet!|
|00000660| 61 6c 6c 62 65 72 79 20 | 4d 6f 6e 20 53 65 70 20 |allbery |Mon Sep |
|00000670| 31 37 20 30 38 3a 35 30 | 3a 35 38 20 50 44 54 20 |17 08:50|:58 PDT |
|00000680| 31 39 39 30 0a 0a 50 6f | 73 74 69 6e 67 2d 6e 75 |1990..Po|sting-nu|
|00000690| 6d 62 65 72 3a 20 56 6f | 6c 75 6d 65 20 31 34 2c |mber: Vo|lume 14,|
|000006a0| 20 49 73 73 75 65 20 38 | 36 0a 53 75 62 6d 69 74 | Issue 8|6.Submit|
|000006b0| 74 65 64 2d 62 79 3a 20 | 44 6f 6e 61 6c 64 20 54 |ted-by: |Donald T|
|000006c0| 76 65 74 65 72 20 3c 64 | 72 74 40 63 68 69 6e 65 |veter <d|rt@chine|
|000006d0| 74 2e 63 68 69 2e 69 6c | 2e 75 73 3e 0a 41 72 63 |t.chi.il|.us>.Arc|
|000006e0| 68 69 76 65 2d 6e 61 6d | 65 3a 20 62 61 63 6b 2d |hive-nam|e: back-|
|000006f0| 70 72 6f 70 2f 70 61 72 | 74 30 33 0a 0a 3d 3d 3d |prop/par|t03..===|
|00000700| 20 62 61 63 6b 70 72 6f | 70 2e 73 68 61 72 34 20 | backpro|p.shar4 |
|00000710| 3d 3d 3d 3d 3d 3d 3d 3d | 3d 3d 3d 3d 3d 3d 3d 3d |========|========|
|00000720| 3d 3d 3d 3d 3d 3d 3d 3d | 3d 0a 46 72 6f 6d 20 73 |========|=.From s|
|00000730| 70 61 72 6b 79 66 73 21 | 68 65 72 63 75 6c 65 73 |parkyfs!|hercules|
|00000740| 21 61 70 70 6c 65 21 75 | 73 63 21 63 73 2e 75 74 |!apple!u|sc!cs.ut|
|00000750| 65 78 61 73 2e 65 64 75 | 21 75 75 6e 65 74 21 61 |exas.edu|!uunet!a|
|00000760| 6c 6c 62 65 72 79 20 4d | 6f 6e 20 53 65 70 20 31 |llbery M|on Sep 1|
|00000770| 37 20 30 38 3a 35 31 3a | 30 36 20 50 44 54 20 31 |7 08:51:|06 PDT 1|
|00000780| 39 39 30 0a 0a 50 6f 73 | 74 69 6e 67 2d 6e 75 6d |990..Pos|ting-num|
|00000790| 62 65 72 3a 20 56 6f 6c | 75 6d 65 20 31 34 2c 20 |ber: Vol|ume 14, |
|000007a0| 49 73 73 75 65 20 38 37 | 0a 53 75 62 6d 69 74 74 |Issue 87|.Submitt|
|000007b0| 65 64 2d 62 79 3a 20 44 | 6f 6e 61 6c 64 20 54 76 |ed-by: D|onald Tv|
|000007c0| 65 74 65 72 20 3c 64 72 | 74 40 63 68 69 6e 65 74 |eter <dr|t@chinet|
|000007d0| 2e 63 68 69 2e 69 6c 2e | 75 73 3e 0a 41 72 63 68 |.chi.il.|us>.Arch|
|000007e0| 69 76 65 2d 6e 61 6d 65 | 3a 20 62 61 63 6b 2d 70 |ive-name|: back-p|
|000007f0| 72 6f 70 2f 70 61 72 74 | 30 34 0a 0a 00 00 00 00 |rop/part|04......|
|00000800| 62 61 63 6b 70 72 6f 70 | 2f 52 45 41 44 4d 45 00 |backprop|/README.|
|00000810| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000820| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000830| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000840| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000850| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000860| 00 00 00 00 20 20 20 36 | 34 34 20 00 20 35 32 37 |....   6|44 . 527|
|00000870| 35 30 20 00 20 20 20 20 | 31 33 20 00 20 20 20 20 |50 .    |13 .    |
|00000880| 20 31 30 34 32 31 35 20 | 20 34 36 37 35 31 37 30 | 104215 | 4675170|
|00000890| 33 36 36 20 20 20 36 37 | 31 36 00 20 00 00 00 00 |366   67|16. ....|
|000008a0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000008b0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000008c0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000008d0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000008e0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000008f0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000900| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000910| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000920| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000930| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000940| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000950| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000960| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000970| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000980| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000990| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000009a0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000009b0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000009c0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000009d0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000009e0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|000009f0| 00 00 00 00 00 00 00 00 | 00 00 00 00 00 00 00 00 |........|........|
|00000a00| 2e 63 65 0a 46 61 73 74 | 20 42 61 63 6b 2d 50 72 |.ce.Fast| Back-Pr|
|00000a10| 6f 70 61 67 61 74 69 6f | 6e 0a 2e 63 65 0a 43 6f |opagatio|n..ce.Co|
|00000a20| 70 79 72 69 67 68 74 20 | 28 63 29 20 31 39 39 30 |pyright |(c) 1990|
|00000a30| 20 62 79 20 44 6f 6e 61 | 6c 64 20 52 2e 20 54 76 | by Dona|ld R. Tv|
|00000a40| 65 74 65 72 0a 0a 0a 2e | 75 6c 0a 49 6e 74 72 6f |eter....|ul.Intro|
|00000a50| 64 75 63 74 69 6f 6e 0a | 0a 20 20 20 54 68 65 20 |duction.|.   The |
|00000a60| 70 72 6f 67 72 61 6d 73 | 20 64 65 73 63 72 69 62 |programs| describ|
|00000a70| 65 64 20 62 65 6c 6f 77 | 20 77 65 72 65 20 70 72 |ed below| were pr|
|00000a80| 6f 64 75 63 65 64 20 66 | 6f 72 20 6d 79 20 6f 77 |oduced f|or my ow|
|00000a90| 6e 20 75 73 65 20 69 6e | 20 73 74 75 64 79 69 6e |n use in| studyin|
|00000aa0| 67 0a 62 61 63 6b 2d 70 | 72 6f 70 61 67 61 74 69 |g.back-p|ropagati|
|00000ab0| 6f 6e 20 61 6e 64 20 66 | 6f 72 20 64 6f 69 6e 67 |on and f|or doing|
|00000ac0| 20 65 78 70 65 72 69 6d | 65 6e 74 73 20 74 68 61 | experim|ents tha|
|00000ad0| 74 20 61 72 65 20 66 6f | 75 6e 64 20 69 6e 20 6d |t are fo|und in m|
|00000ae0| 79 0a 69 6e 74 72 6f 64 | 75 63 74 69 6f 6e 20 74 |y.introd|uction t|
|00000af0| 6f 20 41 72 74 69 66 69 | 63 69 61 6c 20 49 6e 74 |o Artifi|cial Int|
|00000b00| 65 6c 6c 69 67 65 6e 63 | 65 20 74 65 78 74 62 6f |elligenc|e textbo|
|00000b10| 6f 6b 2c 20 5c 66 49 54 | 68 65 20 42 61 73 69 73 |ok, \fIT|he Basis|
|00000b20| 20 6f 66 0a 41 72 74 69 | 66 69 63 69 61 6c 20 49 | of.Arti|ficial I|
|00000b30| 6e 74 65 6c 6c 69 67 65 | 6e 63 65 5c 66 52 2c 20 |ntellige|nce\fR, |
|00000b40| 74 6f 20 62 65 20 70 75 | 62 6c 69 73 68 65 64 20 |to be pu|blished |
|00000b50| 62 79 20 43 6f 6d 70 75 | 74 65 72 20 53 63 69 65 |by Compu|ter Scie|
|00000b60| 6e 63 65 20 50 72 65 73 | 73 2e 0a 49 20 68 61 76 |nce Pres|s..I hav|
|00000b70| 65 20 63 6f 70 79 72 69 | 67 68 74 65 64 20 74 68 |e copyri|ghted th|
|00000b80| 65 73 65 20 66 69 6c 65 | 73 20 62 75 74 20 49 20 |ese file|s but I |
|00000b90| 68 65 72 65 62 79 20 67 | 69 76 65 20 70 65 72 6d |hereby g|ive perm|
|00000ba0| 69 73 73 69 6f 6e 20 74 | 6f 20 61 6e 79 6f 6e 65 |ission t|o anyone|
|00000bb0| 20 74 6f 0a 75 73 65 20 | 74 68 65 6d 20 66 6f 72 | to.use |them for|
|00000bc0| 20 65 78 70 65 72 69 6d | 65 6e 74 61 74 69 6f 6e | experim|entation|
|00000bd0| 2c 20 65 64 75 63 61 74 | 69 6f 6e 61 6c 20 70 75 |, educat|ional pu|
|00000be0| 72 70 6f 73 65 73 20 6f | 72 20 74 6f 20 72 65 64 |rposes o|r to red|
|00000bf0| 69 73 74 72 69 62 75 74 | 65 0a 74 68 65 6d 20 6f |istribut|e.them o|
|00000c00| 6e 20 61 20 6e 6f 74 20 | 66 6f 72 20 70 72 6f 66 |n a not |for prof|
|00000c10| 69 74 20 62 61 73 69 73 | 2e 20 20 41 6c 6c 20 6f |it basis|.  All o|
|00000c20| 74 68 65 72 73 20 74 68 | 61 74 20 6d 61 79 20 77 |thers th|at may w|
|00000c30| 61 6e 74 20 74 6f 20 75 | 73 65 2c 20 63 68 61 6e |ant to u|se, chan|
|00000c40| 67 65 0a 6f 72 20 72 65 | 64 69 73 74 72 69 62 75 |ge.or re|distribu|
|00000c50| 74 65 20 74 68 65 73 65 | 20 70 72 6f 67 72 61 6d |te these| program|
|00000c60| 73 20 66 6f 72 20 63 6f | 6d 6d 65 72 63 69 61 6c |s for co|mmercial|
|00000c70| 20 70 75 72 70 6f 73 65 | 73 2c 20 73 68 6f 75 6c | purpose|s, shoul|
|00000c80| 64 20 63 6f 6e 74 61 63 | 74 0a 6d 65 20 62 79 20 |d contac|t.me by |
|00000c90| 6d 61 69 6c 20 61 74 3a | 0a 0a 2e 6e 61 0a 2e 6e |mail at:|...na..n|
|00000ca0| 66 0a 20 20 20 20 20 20 | 20 20 20 20 20 20 20 20 |f.      |        |
|00000cb0| 20 20 20 20 44 72 2e 20 | 44 6f 6e 61 6c 64 20 52 |    Dr. |Donald R|
|00000cc0| 2e 20 54 76 65 74 65 72 | 0a 20 20 20 20 20 20 20 |. Tveter|.       |
|00000cd0| 20 20 20 20 20 20 20 20 | 20 20 20 35 32 32 38 20 |        |   5228 |
|00000ce0| 4e 2e 20 4e 61 73 68 76 | 69 6c 6c 65 20 41 76 65 |N. Nashv|ille Ave|
|00000cf0| 2e 0a 20 20 20 20 20 20 | 20 20 20 20 20 20 20 20 |..      |        |
|00000d00| 20 20 20 20 43 68 69 63 | 61 67 6f 2c 20 49 6c 6c |    Chic|ago, Ill|
|00000d10| 69 6e 6f 69 73 20 20 20 | 36 30 36 35 36 0a 20 20 |inois   |60656.  |
|00000d20| 20 20 20 20 20 20 20 20 | 20 20 20 20 20 20 20 20 |        |        |
|00000d30| 55 53 45 4e 45 54 3a 20 | 20 64 72 74 5c 40 63 68 |USENET: | drt\@ch|
|00000d40| 69 6e 65 74 2e 63 68 69 | 2e 69 6c 2e 75 73 0a 2e |inet.chi|.il.us..|
|00000d50| 61 64 0a 2e 66 69 0a 0a | 41 6c 73 6f 2c 20 49 20 |ad..fi..|Also, I |
|00000d60| 77 6f 75 6c 64 20 62 65 | 20 69 6e 74 65 72 65 73 |would be| interes|
|00000d70| 74 65 64 20 69 6e 20 68 | 65 61 72 69 6e 67 20 79 |ted in h|earing y|
|00000d80| 6f 75 72 20 73 75 67 67 | 65 73 74 69 6f 6e 73 2c |our sugg|estions,|
|00000d90| 20 62 75 67 20 72 65 70 | 6f 72 74 73 0a 61 6e 64 | bug rep|orts.and|
|00000da0| 20 6d 61 6a 6f 72 20 73 | 75 63 63 65 73 73 65 73 | major s|uccesses|
|00000db0| 20 6f 72 20 66 61 69 6c | 75 72 65 73 2e 0a 0a 20 | or fail|ures... |
|00000dc0| 20 20 54 68 65 72 65 20 | 61 72 65 20 66 6f 75 72 |  There |are four|
|00000dd0| 20 73 69 6d 75 6c 61 74 | 6f 72 73 20 74 68 61 74 | simulat|ors that|
|00000de0| 20 63 61 6e 20 62 65 20 | 63 6f 6e 73 74 72 75 63 | can be |construc|
|00000df0| 74 65 64 20 66 72 6f 6d | 20 74 68 65 0a 69 6e 63 |ted from| the.inc|
|00000e00| 6c 75 64 65 64 20 66 69 | 6c 65 73 2e 20 20 54 68 |luded fi|les.  Th|
|00000e10| 65 20 70 72 6f 67 72 61 | 6d 2c 20 72 62 70 2c 20 |e progra|m, rbp, |
|00000e20| 64 6f 65 73 20 62 61 63 | 6b 2d 70 72 6f 70 61 67 |does bac|k-propag|
|00000e30| 61 74 69 6f 6e 20 75 73 | 69 6e 67 20 64 6f 75 62 |ation us|ing doub|
|00000e40| 6c 65 0a 70 72 65 63 69 | 73 69 6f 6e 20 66 6c 6f |le.preci|sion flo|
|00000e50| 61 74 69 6e 67 20 70 6f | 69 6e 74 20 77 65 69 67 |ating po|int weig|
|00000e60| 68 74 73 20 61 6e 64 20 | 61 72 69 74 68 6d 65 74 |hts and |arithmet|
|00000e70| 69 63 2e 20 20 54 68 65 | 20 70 72 6f 67 72 61 6d |ic.  The| program|
|00000e80| 2c 20 62 70 2c 20 64 6f | 65 73 0a 62 61 63 6b 2d |, bp, do|es.back-|
|00000e90| 70 72 6f 70 61 67 61 74 | 69 6f 6e 20 75 73 69 6e |propagat|ion usin|
|00000ea0| 67 20 31 36 2d 62 69 74 | 20 69 6e 74 65 67 65 72 |g 16-bit| integer|
|00000eb0| 20 77 65 69 67 68 74 73 | 2c 20 31 36 20 61 6e 64 | weights|, 16 and|
|00000ec0| 20 33 32 2d 62 69 74 20 | 69 6e 74 65 67 65 72 0a | 32-bit |integer.|
|00000ed0| 61 72 69 74 68 6d 65 74 | 69 63 20 61 6e 64 20 73 |arithmet|ic and s|
|00000ee0| 6f 6d 65 20 64 6f 75 62 | 6c 65 20 70 72 65 63 69 |ome doub|le preci|
|00000ef0| 73 69 6f 6e 20 66 6c 6f | 61 74 69 6e 67 20 70 6f |sion flo|ating po|
|00000f00| 69 6e 74 20 61 72 69 74 | 68 6d 65 74 69 63 2e 20 |int arit|hmetic. |
|00000f10| 20 54 68 65 0a 70 72 6f | 67 72 61 6d 2c 20 73 62 | The.pro|gram, sb|
|00000f20| 70 2c 20 75 73 65 73 20 | 31 36 2d 62 69 74 20 69 |p, uses |16-bit i|
|00000f30| 6e 74 65 67 65 72 20 73 | 79 6d 6d 65 74 72 69 63 |nteger s|ymmetric|
|00000f40| 20 77 65 69 67 68 74 73 | 20 62 75 74 20 6f 6e 6c | weights| but onl|
|00000f50| 79 20 61 6c 6c 6f 77 73 | 0a 74 77 6f 2d 6c 61 79 |y allows|.two-lay|
|00000f60| 65 72 20 6e 65 74 77 6f | 72 6b 73 2e 20 20 54 68 |er netwo|rks.  Th|
|00000f70| 65 20 70 72 6f 67 72 61 | 6d 20 73 72 62 70 20 64 |e progra|m srbp d|
|00000f80| 6f 65 73 20 74 68 65 20 | 73 61 6d 65 20 75 73 69 |oes the |same usi|
|00000f90| 6e 67 20 36 34 2d 62 69 | 74 0a 66 6c 6f 61 74 69 |ng 64-bi|t.floati|
|00000fa0| 6e 67 20 70 6f 69 6e 74 | 20 77 65 69 67 68 74 73 |ng point| weights|
|00000fb0| 2e 20 20 54 68 65 20 70 | 75 72 70 6f 73 65 20 6f |.  The p|urpose o|
|00000fc0| 66 20 73 62 70 20 61 6e | 64 20 73 72 62 70 20 69 |f sbp an|d srbp i|
|00000fd0| 73 20 74 6f 20 70 72 6f | 64 75 63 65 0a 6e 65 74 |s to pro|duce.net|
|00000fe0| 77 6f 72 6b 73 20 74 68 | 61 74 20 63 61 6e 20 62 |works th|at can b|
|00000ff0| 65 20 75 73 65 64 20 77 | 69 74 68 20 74 68 65 20 |e used w|ith the |
|00001000| 42 6f 6c 74 7a 6d 61 6e | 20 6d 61 63 68 69 6e 65 |Boltzman| machine|
|00001010| 20 72 65 6c 61 78 61 74 | 69 6f 6e 0a 61 6c 67 6f | relaxat|ion.algo|
|00001020| 72 69 74 68 6d 20 28 6e | 6f 74 20 69 6e 63 6c 75 |rithm (n|ot inclu|
|00001030| 64 65 64 29 2e 0a 0a 20 | 20 20 49 6e 20 6d 6f 73 |ded)... |  In mos|
|00001040| 74 20 63 61 73 65 73 2c | 20 74 68 65 20 31 36 2d |t cases,| the 16-|
|00001050| 62 69 74 20 69 6e 74 65 | 67 65 72 20 70 72 6f 67 |bit inte|ger prog|
|00001060| 72 61 6d 73 20 61 72 65 | 20 74 68 65 20 6d 6f 73 |rams are| the mos|
|00001070| 74 20 75 73 65 66 75 6c | 2c 0a 62 65 63 61 75 73 |t useful|,.becaus|
|00001080| 65 20 74 68 65 79 20 61 | 72 65 20 74 68 65 20 66 |e they a|re the f|
|00001090| 61 73 74 65 73 74 2e 20 | 20 57 69 74 68 20 61 20 |astest. | With a |
|000010a0| 31 30 20 4d 48 7a 20 36 | 38 30 31 30 2c 20 63 6f |10 MHz 6|8010, co|
|000010b0| 6e 6e 65 63 74 69 6f 6e | 73 20 63 61 6e 20 62 65 |nnection|s can be|
|000010c0| 0a 70 72 6f 63 65 73 73 | 65 64 20 61 74 20 75 70 |.process|ed at up|
|000010d0| 20 74 6f 20 61 62 6f 75 | 74 20 34 35 2c 30 30 30 | to abou|t 45,000|
|000010e0| 20 70 65 72 20 73 65 63 | 6f 6e 64 20 61 6e 64 20 | per sec|ond and |
|000010f0| 77 65 69 67 68 74 20 63 | 68 61 6e 67 65 73 20 63 |weight c|hanges c|
|00001100| 61 6e 20 62 65 0a 64 6f | 6e 65 20 61 74 20 75 70 |an be.do|ne at up|
|00001110| 20 74 6f 20 61 62 6f 75 | 74 20 32 35 2c 30 30 30 | to abou|t 25,000|
|00001120| 20 70 65 72 20 73 65 63 | 6f 6e 64 2e 20 20 54 68 | per sec|ond.  Th|
|00001130| 65 73 65 20 76 61 6c 75 | 65 73 20 64 65 70 65 6e |ese valu|es depen|
|00001140| 64 20 6f 6e 20 74 68 65 | 20 65 78 61 63 74 0a 70 |d on the| exact.p|
|00001150| 72 6f 62 6c 65 6d 2e 20 | 20 54 68 65 20 69 6e 74 |roblem. | The int|
|00001160| 65 67 65 72 20 76 65 72 | 73 69 6f 6e 73 20 77 69 |eger ver|sions wi|
|00001170| 6c 6c 20 70 72 6f 62 61 | 62 6c 79 20 62 65 20 66 |ll proba|bly be f|
|00001180| 61 73 74 65 72 20 6f 6e | 20 6d 6f 73 74 20 6d 61 |aster on| most ma|
|00001190| 63 68 69 6e 65 73 0a 74 | 68 61 6e 20 74 68 65 20 |chines.t|han the |
|000011a0| 76 65 72 73 69 6f 6e 73 | 20 74 68 61 74 20 75 73 |versions| that us|
|000011b0| 65 20 72 65 61 6c 20 61 | 72 69 74 68 6d 65 74 69 |e real a|rithmeti|
|000011c0| 63 2e 20 20 55 6e 66 6f | 72 74 75 6e 61 74 65 6c |c.  Unfo|rtunatel|
|000011d0| 79 2c 20 73 6f 6d 65 74 | 69 6d 65 73 0a 31 36 2d |y, somet|imes.16-|
|000011e0| 62 69 74 20 69 6e 74 65 | 67 65 72 20 77 65 69 67 |bit inte|ger weig|
|000011f0| 68 74 73 20 64 6f 6e 27 | 74 20 68 61 76 65 20 65 |hts don'|t have e|
|00001200| 6e 6f 75 67 68 20 72 61 | 6e 67 65 20 6f 72 20 70 |nough ra|nge or p|
|00001210| 72 65 63 69 73 69 6f 6e | 20 61 6e 64 20 74 68 65 |recision| and the|
|00001220| 6e 0a 75 73 69 6e 67 20 | 74 68 65 20 66 6c 6f 61 |n.using |the floa|
|00001230| 74 69 6e 67 20 70 6f 69 | 6e 74 20 76 65 72 73 69 |ting poi|nt versi|
|00001240| 6f 6e 73 20 6d 61 79 20 | 62 65 20 6e 65 63 65 73 |ons may |be neces|
|00001250| 73 61 72 79 2e 20 20 4d | 61 6e 79 20 6f 74 68 65 |sary.  M|any othe|
|00001260| 72 20 73 70 65 65 64 2d | 75 70 0a 74 65 63 68 6e |r speed-|up.techn|
|00001270| 69 71 75 65 73 20 61 72 | 65 20 69 6e 63 6c 75 64 |iques ar|e includ|
|00001280| 65 64 20 69 6e 20 74 68 | 65 73 65 20 70 72 6f 67 |ed in th|ese prog|
|00001290| 72 61 6d 73 2e 0a 0a 2e | 75 6c 0a 4d 61 6b 69 6e |rams....|ul.Makin|
|000012a0| 67 20 74 68 65 20 53 69 | 6d 75 6c 61 74 6f 72 73 |g the Si|mulators|
|000012b0| 0a 0a 20 20 20 54 6f 20 | 6d 61 6b 65 20 61 20 70 |..   To |make a p|
|000012c0| 61 72 74 69 63 75 6c 61 | 72 20 65 78 65 63 75 74 |articula|r execut|
|000012d0| 61 62 6c 65 20 66 69 6c | 65 2c 20 75 73 65 20 74 |able fil|e, use t|
|000012e0| 68 65 20 6d 61 6b 65 66 | 69 6c 65 20 67 69 76 65 |he makef|ile give|
|000012f0| 6e 0a 77 69 74 68 20 74 | 68 65 20 64 61 74 61 20 |n.with t|he data |
|00001300| 66 69 6c 65 73 20 61 6e | 64 20 6d 61 6b 65 20 61 |files an|d make a|
|00001310| 6e 79 20 6f 72 20 61 6c | 6c 20 6f 66 20 74 68 65 |ny or al|l of the|
|00001320| 6d 20 6c 69 6b 65 20 73 | 6f 3a 0a 0a 2e 63 65 0a |m like s|o:...ce.|
|00001330| 6d 61 6b 65 20 62 70 0a | 2e 63 65 0a 20 6d 61 6b |make bp.|.ce. mak|
|00001340| 65 20 73 62 70 0a 2e 63 | 65 0a 20 6d 61 6b 65 20 |e sbp..c|e. make |
|00001350| 72 62 70 0a 2e 63 65 0a | 20 6d 61 6b 65 20 73 72 |rbp..ce.| make sr|
|00001360| 62 70 0a 0a 4f 6e 65 20 | 6f 70 74 69 6f 6e 20 65 |bp..One |option e|
|00001370| 78 69 73 74 73 20 66 6f | 72 20 62 70 20 61 6e 64 |xists fo|r bp and|
|00001380| 20 73 62 70 2e 20 20 49 | 66 20 79 6f 75 72 20 63 | sbp.  I|f your c|
|00001390| 6f 6d 70 69 6c 65 72 20 | 69 73 20 73 6d 61 72 74 |ompiler |is smart|
|000013a0| 20 65 6e 6f 75 67 68 0a | 74 6f 20 64 69 76 69 64 | enough.|to divid|
|000013b0| 65 20 62 79 20 31 30 32 | 34 20 62 79 20 73 68 69 |e by 102|4 by shi|
|000013c0| 66 74 69 6e 67 2c 20 75 | 73 65 20 22 2d 44 53 4d |fting, u|se "-DSM|
|000013d0| 41 52 54 22 2e 0a 0a 20 | 20 20 54 6f 20 6d 61 6b |ART"... |  To mak|
|000013e0| 65 20 61 20 72 65 63 6f | 72 64 20 6f 66 20 61 6c |e a reco|rd of al|
|000013f0| 6c 20 74 68 65 20 69 6e | 70 75 74 20 61 6e 64 20 |l the in|put and |
|00001400| 6f 75 74 70 75 74 20 66 | 72 6f 6d 20 74 68 65 20 |output f|rom the |
|00001410| 70 72 6f 67 72 61 6d 73 | 2c 0a 74 68 65 20 66 6f |programs|,.the fo|
|00001420| 6c 6c 6f 77 69 6e 67 20 | 73 6d 61 6c 6c 20 55 4e |llowing |small UN|
|00001430| 49 58 20 63 6f 6d 6d 61 | 6e 64 20 66 69 6c 65 20 |IX comma|nd file |
|00001440| 49 20 63 61 6c 6c 20 72 | 65 63 6f 72 64 20 63 61 |I call r|ecord ca|
|00001450| 6e 20 62 65 20 75 73 65 | 64 3a 0a 0a 2e 6e 61 0a |n be use|d:...na.|
|00001460| 2e 6e 66 0a 74 72 61 70 | 20 22 22 20 32 0a 6f 75 |.nf.trap| "" 2.ou|
|00001470| 74 66 69 6c 65 3d 22 24 | 7b 31 7d 2e 72 65 63 6f |tfile="$|{1}.reco|
|00001480| 72 64 22 0a 69 66 20 74 | 65 73 74 20 2d 66 20 24 |rd".if t|est -f $|
|00001490| 6f 75 74 66 69 6c 65 20 | 0a 20 20 20 74 68 65 6e |outfile |.   then|
|000014a0| 0a 20 20 20 20 20 20 72 | 6d 20 24 6f 75 74 66 69 |.      r|m $outfi|
|000014b0| 6c 65 0a 20 20 20 66 69 | 0a 65 63 68 6f 20 24 6f |le.   fi|.echo $o|
|000014c0| 75 74 66 69 6c 65 0a 28 | 74 65 65 20 2d 61 20 24 |utfile.(|tee -a $|
|000014d0| 6f 75 74 66 69 6c 65 20 | 7c 20 24 2a 29 20 7c 20 |outfile || $*) | |
|000014e0| 74 65 65 20 2d 61 20 24 | 6f 75 74 66 69 6c 65 0a |tee -a $|outfile.|
|000014f0| 70 72 6f 63 65 73 73 3d | 60 70 73 20 7c 20 67 72 |process=|`ps | gr|
|00001500| 65 70 20 74 65 65 20 7c | 20 63 75 74 20 2d 63 31 |ep tee || cut -c1|
|00001510| 2d 36 60 0a 6b 69 6c 6c | 20 24 70 72 6f 63 65 73 |-6`.kill| $proces|
|00001520| 73 0a 2e 61 64 0a 2e 66 | 69 0a 0a 46 6f 72 20 65 |s..ad..f|i..For e|
|00001530| 78 61 6d 70 6c 65 20 74 | 6f 20 6d 61 6b 65 20 61 |xample t|o make a|
|00001540| 20 72 65 63 6f 72 64 20 | 6f 66 20 61 6c 6c 20 74 | record |of all t|
|00001550| 68 65 20 69 6e 70 75 74 | 20 61 6e 64 20 6f 75 74 |he input| and out|
|00001560| 70 75 74 20 66 72 6f 6d | 20 74 68 65 0a 70 72 6f |put from| the.pro|
|00001570| 67 72 61 6d 20 62 70 20 | 75 73 69 6e 67 20 64 61 |gram bp |using da|
|00001580| 74 61 20 66 69 6c 65 2c | 20 78 6f 72 2c 20 75 73 |ta file,| xor, us|
|00001590| 65 3a 0a 0a 2e 63 65 0a | 72 65 63 6f 72 64 20 62 |e:...ce.|record b|
|000015a0| 70 20 78 6f 72 0a 0a 0a | 2e 75 6c 0a 41 20 53 69 |p xor...|.ul.A Si|
|000015b0| 6d 70 6c 65 20 45 78 61 | 6d 70 6c 65 0a 0a 20 20 |mple Exa|mple..  |
|000015c0| 45 61 63 68 20 76 65 72 | 73 69 6f 6e 20 77 6f 75 |Each ver|sion wou|
|000015d0| 6c 64 20 6e 6f 72 6d 61 | 6c 6c 79 20 62 65 20 63 |ld norma|lly be c|
|000015e0| 61 6c 6c 65 64 20 77 69 | 74 68 20 74 68 65 20 6e |alled wi|th the n|
|000015f0| 61 6d 65 20 6f 66 20 61 | 20 66 69 6c 65 20 74 6f |ame of a| file to|
|00001600| 20 72 65 61 64 0a 63 6f | 6d 6d 61 6e 64 73 20 66 | read.co|mmands f|
|00001610| 72 6f 6d 2c 20 61 73 20 | 69 6e 3a 0a 0a 2e 63 65 |rom, as |in:...ce|
|00001620| 0a 62 70 20 78 6f 72 0a | 0a 57 68 65 6e 20 6e 6f |.bp xor.|.When no|
|00001630| 20 66 69 6c 65 20 6e 61 | 6d 65 20 69 73 20 73 70 | file na|me is sp|
|00001640| 65 63 69 66 69 65 64 2c | 20 62 70 20 65 78 70 65 |ecified,| bp expe|
|00001650| 63 74 73 20 74 6f 20 74 | 61 6b 65 20 63 6f 6d 6d |cts to t|ake comm|
|00001660| 61 6e 64 73 20 66 72 6f | 6d 20 74 68 65 0a 6b 65 |ands fro|m the.ke|
|00001670| 79 62 6f 61 72 64 20 28 | 55 4e 49 58 20 73 74 64 |yboard (|UNIX std|
|00001680| 69 6e 20 66 69 6c 65 29 | 2e 20 20 41 66 74 65 72 |in file)|.  After|
|00001690| 20 74 68 65 20 66 69 6c | 65 20 6e 61 6d 65 20 66 | the fil|e name f|
|000016a0| 72 6f 6d 20 74 68 65 20 | 63 6f 6d 6d 61 6e 64 20 |rom the |command |
|000016b0| 6c 69 6e 65 0a 69 73 20 | 72 65 61 64 20 61 6e 64 |line.is |read and|
|000016c0| 20 74 68 65 20 63 6f 6d | 6d 61 6e 64 73 20 69 6e | the com|mands in|
|000016d0| 20 74 68 65 20 66 69 6c | 65 20 61 72 65 20 65 78 | the fil|e are ex|
|000016e0| 65 63 75 74 65 64 2c 20 | 63 6f 6d 6d 61 6e 64 73 |ecuted, |commands|
|000016f0| 20 61 72 65 20 74 68 65 | 6e 0a 74 61 6b 65 6e 20 | are the|n.taken |
|00001700| 66 72 6f 6d 20 74 68 65 | 20 6b 65 79 62 6f 61 72 |from the| keyboar|
|00001710| 64 2e 0a 0a 20 20 20 54 | 68 65 20 63 6f 6d 6d 61 |d...   T|he comma|
|00001720| 6e 64 73 20 61 72 65 20 | 6f 6e 65 20 6c 65 74 74 |nds are |one lett|
|00001730| 65 72 20 63 6f 6d 6d 61 | 6e 64 73 2e 20 20 4d 6f |er comma|nds.  Mo|
|00001740| 73 74 20 63 6f 6d 6d 61 | 6e 64 73 20 68 61 76 65 |st comma|nds have|
|00001750| 0a 6f 70 74 69 6f 6e 61 | 6c 20 70 61 72 61 6d 65 |.optiona|l parame|
|00001760| 74 65 72 73 2e 20 20 54 | 68 65 20 60 2a 27 20 63 |ters.  T|he `*' c|
|00001770| 68 61 72 61 63 74 65 72 | 20 69 73 20 61 20 63 6f |haracter| is a co|
|00001780| 6d 6d 65 6e 74 2e 20 20 | 49 74 20 63 61 6e 20 62 |mment.  |It can b|
|00001790| 65 20 75 73 65 64 0a 74 | 6f 20 6d 61 6b 65 20 74 |e used.t|o make t|
|000017a0| 68 65 20 72 65 6d 61 69 | 6e 64 65 72 20 6f 66 20 |he remai|nder of |
|000017b0| 74 68 65 20 6c 69 6e 65 | 20 61 20 63 6f 6d 6d 65 |the line| a comme|
|000017c0| 6e 74 2e 20 20 48 65 72 | 65 20 69 73 20 61 6e 20 |nt.  Her|e is an |
|000017d0| 65 78 61 6d 70 6c 65 20 | 6f 66 0a 61 6e 20 69 6e |example |of.an in|
|000017e0| 70 75 74 20 66 69 6c 65 | 20 74 6f 20 64 6f 20 74 |put file| to do t|
|000017f0| 68 65 20 78 6f 72 20 70 | 72 6f 62 6c 65 6d 3a 0a |he xor p|roblem:.|
|00001800| 20 20 20 20 20 20 20 20 | 20 20 20 0a 2e 6e 61 0a |        |   ..na.|
|00001810| 2e 6e 66 0a 2a 20 69 6e | 70 75 74 20 66 69 6c 65 |.nf.* in|put file|
|00001820| 20 66 6f 72 20 74 68 65 | 20 78 6f 72 20 70 72 6f | for the| xor pro|
|00001830| 62 6c 65 6d 0a 20 20 20 | 20 20 20 20 20 20 20 20 |blem.   |        |
|00001840| 0a 6d 20 32 20 31 20 31 | 20 20 20 20 20 20 20 20 |.m 2 1 1|        |
|00001850| 20 20 20 2a 20 6d 61 6b | 65 20 61 20 32 2d 31 2d |   * mak|e a 2-1-|
|00001860| 31 20 6e 65 74 77 6f 72 | 6b 0a 63 20 31 20 31 20 |1 networ|k.c 1 1 |
|00001870| 33 20 31 20 20 20 20 20 | 20 20 20 20 2a 20 61 64 |3 1     |    * ad|
|00001880| 64 20 74 68 69 73 20 65 | 78 74 72 61 20 63 6f 6e |d this e|xtra con|
|00001890| 6e 65 63 74 69 6f 6e 0a | 63 20 31 20 32 20 33 20 |nection.|c 1 2 3 |
|000018a0| 31 20 20 20 20 20 20 20 | 20 20 2a 20 61 64 64 20 |1       |  * add |
|000018b0| 74 68 69 73 20 65 78 74 | 72 61 20 63 6f 6e 6e 65 |this ext|ra conne|
|000018c0| 63 74 69 6f 6e 0a 73 20 | 37 20 20 20 20 20 20 20 |ction.s |7       |
|000018d0| 20 20 20 20 20 20 20 20 | 2a 20 73 65 65 64 20 74 |        |* seed t|
|000018e0| 68 65 20 72 61 6e 64 6f | 6d 20 6e 75 6d 62 65 72 |he rando|m number|
|000018f0| 20 66 75 6e 63 74 69 6f | 6e 0a 6b 20 30 20 31 20 | functio|n.k 0 1 |
|00001900| 20 20 20 20 20 20 20 20 | 20 20 20 20 2a 20 67 69 |        |    * gi|
|00001910| 76 65 20 74 68 65 20 6e | 65 74 77 6f 72 6b 20 72 |ve the n|etwork r|
|00001920| 61 6e 64 6f 6d 20 77 65 | 69 67 68 74 73 0a 0a 6e |andom we|ights..n|
|00001930| 20 34 20 20 20 20 20 20 | 20 20 20 20 20 20 20 20 | 4      |        |
|00001940| 20 2a 20 72 65 61 64 20 | 66 6f 75 72 20 6e 65 77 | * read |four new|
|00001950| 20 70 61 74 74 65 72 6e | 73 20 69 6e 74 6f 20 6d | pattern|s into m|
|00001960| 65 6d 6f 72 79 0a 31 20 | 30 20 31 0a 30 20 30 20 |emory.1 |0 1.0 0 |
|00001970| 30 0a 30 20 31 20 31 0a | 31 20 31 20 30 0a 0a 65 |0.0 1 1.|1 1 0..e|
|00001980| 20 30 2e 35 20 20 20 20 | 20 20 20 20 20 20 20 20 | 0.5    |        |
|00001990| 20 2a 20 73 65 74 20 65 | 74 61 20 74 6f 20 30 2e | * set e|ta to 0.|
|000019a0| 35 20 28 61 6e 64 20 65 | 74 61 32 20 74 6f 20 30 |5 (and e|ta2 to 0|
|000019b0| 2e 30 35 29 0a 61 20 30 | 2e 39 20 20 20 20 20 20 |.05).a 0|.9      |
|000019c0| 20 20 20 20 20 20 20 2a | 20 73 65 74 20 61 6c 70 |       *| set alp|
|000019d0| 68 61 20 74 6f 20 30 2e | 39 0a 2e 61 64 0a 2e 66 |ha to 0.|9..ad..f|
|000019e0| 69 0a 0a 49 6e 20 74 68 | 69 73 20 65 78 61 6d 70 |i..In th|is examp|
|000019f0| 6c 65 2c 20 74 68 65 20 | 6d 20 63 6f 6d 6d 61 6e |le, the |m comman|
|00001a00| 64 20 69 73 20 61 20 63 | 6f 6d 6d 61 6e 64 20 74 |d is a c|ommand t|
|00001a10| 6f 20 6d 61 6b 65 20 61 | 20 6e 65 74 77 6f 72 6b |o make a| network|
|00001a20| 2e 20 20 54 68 65 0a 6e | 75 6d 62 65 72 73 20 66 |.  The.n|umbers f|
|00001a30| 6f 6c 6c 6f 77 69 6e 67 | 20 69 74 20 61 72 65 20 |ollowing| it are |
|00001a40| 74 68 65 20 6e 75 6d 62 | 65 72 20 6f 66 20 75 6e |the numb|er of un|
|00001a50| 69 74 73 20 66 6f 72 20 | 65 61 63 68 20 6c 61 79 |its for |each lay|
|00001a60| 65 72 2e 20 20 54 68 65 | 20 6d 0a 63 6f 6d 6d 61 |er.  The| m.comma|
|00001a70| 6e 64 20 63 6f 6e 6e 65 | 63 74 73 20 61 64 6a 61 |nd conne|cts adja|
|00001a80| 63 65 6e 74 20 6c 61 79 | 65 72 73 20 77 69 74 68 |cent lay|ers with|
|00001a90| 20 77 65 69 67 68 74 73 | 2e 20 20 54 68 65 20 66 | weights|.  The f|
|00001aa0| 6f 6c 6c 6f 77 69 6e 67 | 20 63 0a 63 6f 6d 6d 61 |ollowing| c.comma|
|00001ab0| 6e 64 73 20 63 72 65 61 | 74 65 20 65 78 74 72 61 |nds crea|te extra|
|00001ac0| 20 63 6f 6e 6e 65 63 74 | 69 6f 6e 73 20 66 72 6f | connect|ions fro|
|00001ad0| 6d 20 6c 61 79 65 72 20 | 31 2c 20 75 6e 69 74 20 |m layer |1, unit |
|00001ae0| 31 20 74 6f 20 6c 61 79 | 65 72 20 33 2c 0a 75 6e |1 to lay|er 3,.un|
|00001af0| 69 74 20 31 20 61 6e 64 | 20 66 72 6f 6d 20 6c 61 |it 1 and| from la|
|00001b00| 79 65 72 20 31 2c 20 75 | 6e 69 74 20 32 20 74 6f |yer 1, u|nit 2 to|
|00001b10| 20 6c 61 79 65 72 20 33 | 2c 20 75 6e 69 74 20 31 | layer 3|, unit 1|
|00001b20| 2e 20 20 54 68 65 20 60 | 73 27 20 63 6f 6d 6d 61 |.  The `|s' comma|
|00001b30| 6e 64 0a 73 65 74 73 20 | 74 68 65 20 73 65 65 64 |nd.sets |the seed|
|00001b40| 20 66 6f 72 20 74 68 65 | 20 72 61 6e 64 6f 6d 20 | for the| random |
|00001b50| 6e 75 6d 62 65 72 20 66 | 75 6e 63 74 69 6f 6e 2e |number f|unction.|
|00001b60| 20 20 54 68 65 20 60 6b | 27 20 63 6f 6d 6d 61 6e |  The `k|' comman|
|00001b70| 64 20 74 68 65 6e 0a 67 | 69 76 65 73 20 74 68 65 |d then.g|ives the|
|00001b80| 20 6e 65 74 77 6f 72 6b | 20 72 61 6e 64 6f 6d 20 | network| random |
|00001b90| 77 65 69 67 68 74 73 2e | 20 20 54 68 65 20 60 6b |weights.|  The `k|
|00001ba0| 27 20 63 6f 6d 6d 61 6e | 64 20 68 61 73 20 61 6e |' comman|d has an|
|00001bb0| 6f 74 68 65 72 20 75 73 | 65 20 61 73 0a 77 65 6c |other us|e as.wel|
|00001bc0| 6c 2e 20 20 49 74 20 63 | 61 6e 20 62 65 20 75 73 |l.  It c|an be us|
|00001bd0| 65 64 20 74 6f 20 74 72 | 79 20 74 6f 20 6b 69 63 |ed to tr|y to kic|
|00001be0| 6b 20 61 20 6e 65 74 77 | 6f 72 6b 20 6f 75 74 20 |k a netw|ork out |
|00001bf0| 6f 66 20 61 20 6c 6f 63 | 61 6c 20 6d 69 6e 69 6d |of a loc|al minim|
|00001c00| 75 6d 2e 0a 48 65 72 65 | 2c 20 74 68 65 20 6d 65 |um..Here|, the me|
|00001c10| 61 6e 69 6e 67 20 6f 66 | 20 22 6b 20 30 20 31 22 |aning of| "k 0 1"|
|00001c20| 20 69 73 20 74 6f 20 65 | 78 61 6d 69 6e 65 20 61 | is to e|xamine a|
|00001c30| 6c 6c 20 74 68 65 20 77 | 65 69 67 68 74 73 20 69 |ll the w|eights i|
|00001c40| 6e 20 74 68 65 0a 6e 65 | 74 77 6f 72 6b 20 61 6e |n the.ne|twork an|
|00001c50| 64 20 66 6f 72 20 65 76 | 65 72 79 20 77 65 69 67 |d for ev|ery weig|
|00001c60| 68 74 20 65 71 75 61 6c | 20 74 6f 20 30 20 28 61 |ht equal| to 0 (a|
|00001c70| 6e 64 20 74 68 65 79 20 | 61 6c 6c 20 73 74 61 72 |nd they |all star|
|00001c80| 74 20 6f 75 74 20 61 74 | 20 30 29 2c 0a 61 64 64 |t out at| 0),.add|
|00001c90| 20 69 6e 20 61 20 72 61 | 6e 64 6f 6d 20 6e 75 6d | in a ra|ndom num|
|00001ca0| 62 65 72 20 62 65 74 77 | 65 65 6e 20 2d 31 20 61 |ber betw|een -1 a|
|00001cb0| 6e 64 20 2b 31 2e 20 20 | 54 68 65 20 60 6e 27 20 |nd +1.  |The `n' |
|00001cc0| 63 6f 6d 6d 61 6e 64 0a | 73 70 65 63 69 66 69 65 |command.|specifie|
|00001cd0| 73 20 66 6f 75 72 20 6e | 65 77 20 70 61 74 74 65 |s four n|ew patte|
|00001ce0| 72 6e 73 20 74 6f 20 62 | 65 20 72 65 61 64 20 69 |rns to b|e read i|
|00001cf0| 6e 74 6f 20 6d 65 6d 6f | 72 79 2e 20 20 57 69 74 |nto memo|ry.  Wit|
|00001d00| 68 20 74 68 65 20 60 6e | 27 0a 63 6f 6d 6d 61 6e |h the `n|'.comman|
|00001d10| 64 2c 20 61 6e 79 20 6f | 6c 64 20 70 61 74 74 65 |d, any o|ld patte|
|00001d20| 72 6e 73 20 74 68 61 74 | 20 6d 61 79 20 68 61 76 |rns that| may hav|
|00001d30| 65 20 62 65 65 6e 20 70 | 72 65 73 65 6e 74 20 61 |e been p|resent a|
|00001d40| 72 65 20 72 65 6d 6f 76 | 65 64 2e 0a 54 68 65 72 |re remov|ed..Ther|
|00001d50| 65 20 69 73 20 61 6c 73 | 6f 20 61 6e 20 60 78 27 |e is als|o an `x'|
|00001d60| 20 63 6f 6d 6d 61 6e 64 | 20 74 68 61 74 20 62 65 | command| that be|
|00001d70| 68 61 76 65 73 20 6c 69 | 6b 65 20 74 68 65 20 60 |haves li|ke the `|
|00001d80| 6e 27 20 63 6f 6d 6d 61 | 6e 64 2c 20 65 78 63 65 |n' comma|nd, exce|
|00001d90| 70 74 0a 74 68 65 20 60 | 78 27 20 63 6f 6d 6d 61 |pt.the `|x' comma|
|00001da0| 6e 64 73 20 5c 66 49 61 | 64 64 73 5c 66 52 20 74 |nds \fIa|dds\fR t|
|00001db0| 68 65 20 65 78 74 72 61 | 20 70 61 74 74 65 72 6e |he extra| pattern|
|00001dc0| 73 20 74 6f 20 74 68 65 | 20 63 75 72 72 65 6e 74 |s to the| current|
|00001dd0| 20 74 72 61 69 6e 69 6e | 67 0a 73 65 74 2e 20 20 | trainin|g.set.  |
|00001de0| 54 68 65 20 69 6e 70 75 | 74 20 70 61 74 74 65 72 |The inpu|t patter|
|00001df0| 6e 20 63 6f 6d 65 73 20 | 66 69 72 73 74 2c 20 66 |n comes |first, f|
|00001e00| 6f 6c 6c 6f 77 65 64 20 | 62 79 20 74 68 65 20 6f |ollowed |by the o|
|00001e10| 75 74 70 75 74 20 70 61 | 74 74 65 72 6e 2e 0a 54 |utput pa|ttern..T|
|00001e20| 68 65 20 73 74 61 74 65 | 6d 65 6e 74 2c 20 65 20 |he state|ment, e |
|00001e30| 30 2e 35 2c 20 73 65 74 | 73 20 65 74 61 2c 20 74 |0.5, set|s eta, t|
|00001e40| 68 65 20 6c 65 61 72 6e | 69 6e 67 20 72 61 74 65 |he learn|ing rate|
|00001e50| 2c 20 74 6f 20 30 2e 35 | 20 61 6e 64 20 65 74 61 |, to 0.5| and eta|
|00001e60| 32 20 66 72 6f 6d 0a 74 | 68 65 20 64 69 66 66 65 |2 from.t|he diffe|
|00001e70| 72 65 6e 74 69 61 6c 20 | 73 74 65 70 20 73 69 7a |rential |step siz|
|00001e80| 65 20 61 6c 67 6f 72 69 | 74 68 6d 20 74 6f 20 6f |e algori|thm to o|
|00001e90| 6e 65 20 74 65 6e 74 68 | 20 74 68 69 73 2c 20 6f |ne tenth| this, o|
|00001ea0| 72 20 30 2e 30 35 2e 20 | 20 54 68 65 0a 6c 61 73 |r 0.05. | The.las|
|00001eb0| 74 20 6c 69 6e 65 20 73 | 65 74 73 20 61 6c 70 68 |t line s|ets alph|
|00001ec0| 61 2c 20 74 68 65 20 6d | 6f 6d 65 6e 74 75 6d 20 |a, the m|omentum |
|00001ed0| 70 61 72 61 6d 65 74 65 | 72 2c 20 74 6f 20 30 2e |paramete|r, to 0.|
|00001ee0| 39 2e 0a 0a 20 20 20 54 | 68 65 20 61 62 6f 76 65 |9...   T|he above|
|00001ef0| 20 73 74 61 74 65 6d 65 | 6e 74 73 20 73 65 74 20 | stateme|nts set |
|00001f00| 75 70 20 74 68 65 20 6e | 65 74 77 6f 72 6b 20 61 |up the n|etwork a|
|00001f10| 6e 64 20 77 68 65 6e 20 | 74 68 65 20 6c 69 73 74 |nd when |the list|
|00001f20| 20 6f 66 0a 63 6f 6d 6d | 61 6e 64 73 20 72 75 6e | of.comm|ands run|
|00001f30| 73 20 6f 75 74 2c 20 63 | 6f 6d 6d 61 6e 64 73 20 |s out, c|ommands |
|00001f40| 61 72 65 20 74 61 6b 65 | 6e 20 66 72 6f 6d 20 74 |are take|n from t|
|00001f50| 68 65 20 6b 65 79 62 6f | 61 72 64 2e 20 20 54 68 |he keybo|ard.  Th|
|00001f60| 65 0a 66 6f 6c 6c 6f 77 | 69 6e 67 20 6d 65 73 73 |e.follow|ing mess|
|00001f70| 61 67 65 73 20 61 6e 64 | 20 70 72 6f 6d 70 74 20 |ages and| prompt |
|00001f80| 61 70 70 65 61 72 73 3a | 0a 0a 2e 6e 61 0a 2e 6e |appears:|...na..n|
|00001f90| 66 0a 2e 6e 65 32 0a 46 | 61 73 74 20 42 61 63 6b |f..ne2.F|ast Back|
|00001fa0| 70 72 6f 70 61 67 61 74 | 69 6f 6e 20 43 6f 70 79 |propagat|ion Copy|
|00001fb0| 72 69 67 68 74 20 28 63 | 29 20 31 39 39 30 20 62 |right (c|) 1990 b|
|00001fc0| 79 20 44 6f 6e 61 6c 64 | 20 52 2e 20 54 76 65 74 |y Donald| R. Tvet|
|00001fd0| 65 72 0a 74 61 6b 69 6e | 67 20 63 6f 6d 6d 61 6e |er.takin|g comman|
|00001fe0| 64 73 20 66 72 6f 6d 20 | 73 74 64 69 6e 20 6e 6f |ds from |stdin no|
|00001ff0| 77 0a 5b 3f 21 2a 41 61 | 62 43 63 45 65 66 48 68 |w.[?!*Aa|bCcEefHh|
|00002000| 69 6a 6b 6c 6d 6e 6f 50 | 70 51 71 52 72 53 73 74 |ijklmnoP|pQqRrSst|
|00002010| 57 77 78 5d 3f 0a 2e 61 | 64 0a 2e 66 69 0a 0a 54 |Wwx]?..a|d..fi..T|
|00002020| 68 65 20 73 71 75 61 72 | 65 20 62 72 61 63 6b 65 |he squar|e bracke|
|00002030| 74 73 20 65 6e 63 6c 6f | 73 65 20 61 20 6c 69 73 |ts enclo|se a lis|
|00002040| 74 20 6f 66 20 74 68 65 | 20 70 6f 73 73 69 62 6c |t of the| possibl|
|00002050| 65 20 63 6f 6d 6d 61 6e | 64 73 2e 0a 54 68 65 20 |e comman|ds..The |
|00002060| 60 72 27 20 63 6f 6d 6d | 61 6e 64 20 69 73 20 75 |`r' comm|and is u|
|00002070| 73 65 64 20 74 6f 20 72 | 75 6e 20 74 68 65 20 74 |sed to r|un the t|
|00002080| 72 61 69 6e 69 6e 67 20 | 61 6c 67 6f 72 69 74 68 |raining |algorith|
|00002090| 6d 2e 20 20 54 79 70 69 | 6e 67 20 69 6e 20 22 72 |m.  Typi|ng in "r|
|000020a0| 20 32 30 30 0a 31 30 30 | 22 20 61 73 20 73 68 6f | 200.100|" as sho|
|000020b0| 77 6e 20 62 65 6c 6f 77 | 2c 20 6d 65 61 6e 73 20 |wn below|, means |
|000020c0| 72 75 6e 20 32 30 30 20 | 69 74 65 72 61 74 69 6f |run 200 |iteratio|
|000020d0| 6e 73 20 74 68 72 6f 75 | 67 68 20 74 68 65 20 70 |ns throu|gh the p|
|000020e0| 61 74 74 65 72 6e 73 0a | 61 6e 64 20 70 72 69 6e |atterns.|and prin|
|000020f0| 74 20 74 68 65 20 6f 75 | 74 70 75 74 20 70 61 74 |t the ou|tput pat|
|00002100| 74 65 72 6e 73 20 65 76 | 65 72 79 20 31 30 30 20 |terns ev|ery 100 |
|00002110| 69 74 65 72 61 74 69 6f | 6e 73 3a 0a 0a 2e 6e 61 |iteratio|ns:...na|
|00002120| 0a 2e 6e 66 0a 2e 6e 65 | 33 0a 5b 3f 21 2a 41 61 |..nf..ne|3.[?!*Aa|
|00002130| 62 43 63 45 65 66 48 68 | 69 6a 6b 6c 6d 6e 6f 50 |bCcEefHh|ijklmnoP|
|00002140| 70 51 71 52 72 53 73 74 | 57 77 78 5d 3f 20 72 20 |pQqRrSst|Wwx]? r |
|00002150| 32 30 30 20 31 30 30 0a | 72 75 6e 6e 69 6e 67 20 |200 100.|running |
|00002160| 2e 20 2e 20 2e 0a 2e 6e | 65 35 0a 31 30 30 20 69 |. . ...n|e5.100 i|
|00002170| 74 65 72 61 74 69 6f 6e | 73 2c 20 73 20 37 2c 20 |teration|s, s 7, |
|00002180| 6b 20 30 20 31 2e 30 30 | 2c 20 66 69 6c 65 20 3d |k 0 1.00|, file =|
|00002190| 20 78 6f 72 0a 20 20 31 | 20 20 30 2e 38 31 20 20 | xor.  1|  0.81  |
|000021a0| 28 30 2e 30 33 37 33 39 | 29 0a 20 20 32 20 20 30 |(0.03739|).  2  0|
|000021b0| 2e 31 33 20 20 28 30 2e | 30 31 36 33 37 29 0a 20 |.13  (0.|01637). |
|000021c0| 20 33 20 20 30 2e 38 35 | 20 20 28 30 2e 30 32 32 | 3  0.85|  (0.022|
|000021d0| 36 32 29 0a 20 20 34 20 | 20 30 2e 31 37 20 20 28 |62).  4 | 0.17  (|
|000021e0| 30 2e 30 32 39 38 38 29 | 0a 2e 6e 65 35 0a 31 35 |0.02988)|..ne5.15|
|000021f0| 39 20 69 74 65 72 61 74 | 69 6f 6e 73 2c 20 73 20 |9 iterat|ions, s |
|00002200| 37 2c 20 6b 20 30 20 31 | 2e 30 30 2c 20 66 69 6c |7, k 0 1|.00, fil|
|00002210| 65 20 3d 20 78 6f 72 0a | 20 20 31 20 20 30 2e 39 |e = xor.|  1  0.9|
|00002220| 30 20 20 28 30 2e 30 30 | 39 37 33 29 0a 20 20 32 |0  (0.00|973).  2|
|00002230| 20 20 30 2e 30 37 20 20 | 28 30 2e 30 30 34 36 37 |  0.07  |(0.00467|
|00002240| 29 0a 20 20 33 20 20 30 | 2e 39 32 20 20 28 30 2e |).  3  0|.92  (0.|
|00002250| 30 30 35 36 35 29 0a 20 | 20 34 20 20 30 2e 30 39 |00565). | 4  0.09|
|00002260| 20 20 28 30 2e 30 30 37 | 33 39 29 0a 70 61 74 74 |  (0.007|39).patt|
|00002270| 65 72 6e 73 20 6c 65 61 | 72 6e 65 64 20 74 6f 20 |erns lea|rned to |
|00002280| 77 69 74 68 69 6e 20 30 | 2e 31 30 20 61 74 20 69 |within 0|.10 at i|
|00002290| 74 65 72 61 74 69 6f 6e | 20 31 35 39 0a 2e 61 64 |teration| 159..ad|
|000022a0| 0a 2e 66 69 0a 0a 54 68 | 65 20 70 72 6f 67 72 61 |..fi..Th|e progra|
|000022b0| 6d 20 69 6d 6d 65 64 69 | 61 74 65 6c 79 20 70 72 |m immedi|ately pr|
|000022c0| 69 6e 74 73 20 6f 75 74 | 20 74 68 65 20 22 72 75 |ints out| the "ru|
|000022d0| 6e 6e 69 6e 67 20 2e 20 | 2e 20 2e 22 20 6d 65 73 |nning . |. ." mes|
|000022e0| 73 61 67 65 2e 20 20 41 | 66 74 65 72 0a 65 61 63 |sage.  A|fter.eac|
|000022f0| 68 20 31 30 30 20 69 74 | 65 72 61 74 69 6f 6e 73 |h 100 it|erations|
|00002300| 2c 20 61 20 68 65 61 64 | 65 72 20 6c 69 6e 65 20 |, a head|er line |
|00002310| 67 69 76 69 6e 67 20 73 | 6f 6d 65 20 70 72 6f 67 |giving s|ome prog|
|00002320| 72 61 6d 20 70 61 72 61 | 6d 65 74 65 72 73 0a 69 |ram para|meters.i|
|00002330| 73 20 70 72 69 6e 74 65 | 64 20 6f 75 74 2c 20 66 |s printe|d out, f|
|00002340| 6f 6c 6c 6f 77 65 64 20 | 62 79 20 74 68 65 20 72 |ollowed |by the r|
|00002350| 65 73 75 6c 74 73 20 74 | 68 61 74 20 6f 63 63 75 |esults t|hat occu|
|00002360| 72 20 77 68 65 6e 20 65 | 61 63 68 20 6f 66 20 74 |r when e|ach of t|
|00002370| 68 65 20 66 6f 75 72 0a | 70 61 74 74 65 72 6e 73 |he four.|patterns|
|00002380| 20 69 73 20 73 75 62 6d | 69 74 74 65 64 20 74 6f | is subm|itted to|
|00002390| 20 74 68 65 20 6e 65 74 | 77 6f 72 6b 2e 20 20 49 | the net|work.  I|
|000023a0| 66 20 74 68 65 20 73 65 | 63 6f 6e 64 20 6e 75 6d |f the se|cond num|
|000023b0| 62 65 72 20 64 65 66 69 | 6e 69 6e 67 0a 68 6f 77 |ber defi|ning.how|
|000023c0| 20 6f 66 74 65 6e 20 74 | 6f 20 70 72 69 6e 74 20 | often t|o print |
|000023d0| 6f 75 74 20 76 61 6c 75 | 65 73 20 69 73 20 6f 6d |out valu|es is om|
|000023e0| 69 74 74 65 64 2c 20 74 | 68 65 20 76 61 6c 75 65 |itted, t|he value|
|000023f0| 73 20 77 69 6c 6c 20 6e | 6f 74 20 70 72 69 6e 74 |s will n|ot print|
|00002400| 0a 65 76 65 6e 20 77 68 | 65 6e 20 74 68 65 20 6c |.even wh|en the l|
|00002410| 65 61 72 6e 69 6e 67 20 | 69 73 20 66 69 6e 69 73 |earning |is finis|
|00002420| 68 65 64 2e 20 20 54 68 | 65 20 76 61 6c 75 65 73 |hed.  Th|e values|
|00002430| 20 69 6e 20 70 61 72 65 | 6e 74 68 65 73 65 73 20 | in pare|ntheses |
|00002440| 61 74 20 74 68 65 0a 65 | 6e 64 20 6f 66 20 65 61 |at the.e|nd of ea|
|00002450| 63 68 20 6c 69 6e 65 20 | 67 69 76 65 20 74 68 65 |ch line |give the|
|00002460| 20 73 75 6d 20 6f 66 20 | 74 68 65 20 73 71 75 61 | sum of |the squa|
|00002470| 72 65 64 20 65 72 72 6f | 72 20 6f 6e 20 74 68 65 |red erro|r on the|
|00002480| 20 6f 75 74 70 75 74 20 | 75 6e 69 74 73 0a 66 6f | output |units.fo|
|00002490| 72 20 65 61 63 68 20 6f | 75 74 70 75 74 20 70 61 |r each o|utput pa|
|000024a0| 74 74 65 72 6e 2e 20 20 | 54 68 65 73 65 20 65 72 |ttern.  |These er|
|000024b0| 72 6f 72 20 6e 75 6d 62 | 65 72 73 20 61 72 65 20 |ror numb|ers are |
|000024c0| 75 73 65 66 75 6c 20 74 | 6f 20 73 65 65 20 62 65 |useful t|o see be|
|000024d0| 63 61 75 73 65 0a 74 68 | 65 79 20 67 69 76 65 20 |cause.th|ey give |
|000024e0| 79 6f 75 20 73 6f 6d 65 | 20 69 64 65 61 20 6f 66 |you some| idea of|
|000024f0| 20 68 6f 77 20 66 61 73 | 74 20 65 61 63 68 20 70 | how fas|t each p|
|00002500| 61 74 74 65 72 6e 20 69 | 73 20 62 65 69 6e 67 20 |attern i|s being |
|00002510| 6c 65 61 72 6e 65 64 2e | 0a 54 68 65 20 70 72 6f |learned.|.The pro|
|00002520| 67 72 61 6d 20 61 6c 73 | 6f 20 72 65 70 6f 72 74 |gram als|o report|
|00002530| 73 20 74 68 61 74 20 74 | 68 65 20 70 61 74 74 65 |s that t|he patte|
|00002540| 72 6e 73 20 68 61 76 65 | 20 62 65 65 6e 20 6c 65 |rns have| been le|
|00002550| 61 72 6e 65 64 20 74 6f | 20 77 69 74 68 69 6e 0a |arned to| within.|
|00002560| 74 68 65 20 64 65 66 61 | 75 6c 74 20 74 6f 6c 65 |the defa|ult tole|
|00002570| 72 61 6e 63 65 20 6f 66 | 20 30 2e 31 2e 20 20 54 |rance of| 0.1.  T|
|00002580| 68 69 73 20 63 68 65 63 | 6b 20 66 6f 72 20 74 68 |his chec|k for th|
|00002590| 65 20 74 6f 6c 65 72 61 | 6e 63 65 20 62 65 69 6e |e tolera|nce bein|
|000025a0| 67 20 6d 65 74 0a 69 73 | 20 64 6f 6e 65 20 66 6f |g met.is| done fo|
|000025b0| 72 20 65 76 65 72 79 20 | 6c 65 61 72 6e 69 6e 67 |r every |learning|
|000025c0| 20 69 74 65 72 61 74 69 | 6f 6e 2e 20 20 53 6f 6d | iterati|on.  Som|
|000025d0| 65 74 69 6d 65 73 20 69 | 6e 20 74 68 65 20 69 6e |etimes i|n the in|
|000025e0| 74 65 67 65 72 20 76 65 | 72 73 69 6f 6e 0a 74 68 |teger ve|rsion.th|
|000025f0| 65 20 70 72 6f 67 72 61 | 6d 20 77 69 6c 6c 20 64 |e progra|m will d|
|00002600| 6f 20 61 20 66 65 77 20 | 65 78 74 72 61 20 69 74 |o a few |extra it|
|00002610| 65 72 61 74 69 6f 6e 73 | 20 62 65 66 6f 72 65 20 |erations| before |
|00002620| 64 65 63 6c 61 72 69 6e | 67 0a 74 68 65 20 70 72 |declarin|g.the pr|
|00002630| 6f 62 6c 65 6d 20 64 6f | 6e 65 2e 20 20 54 68 69 |oblem do|ne.  Thi|
|00002640| 73 20 69 73 20 62 65 63 | 61 75 73 65 20 6f 66 20 |s is bec|ause of |
|00002650| 74 72 75 6e 63 61 74 69 | 6f 6e 20 65 72 72 6f 72 |truncati|on error|
|00002660| 73 20 69 6e 20 74 68 65 | 0a 61 72 69 74 68 6d 65 |s in the|.arithme|
|00002670| 74 69 63 20 64 6f 6e 65 | 20 74 6f 20 63 68 65 63 |tic done| to chec|
|00002680| 6b 20 66 6f 72 20 63 6f | 6e 76 65 72 67 65 6e 63 |k for co|nvergenc|
|00002690| 65 2e 0a 0a 20 20 20 41 | 20 70 61 72 74 69 63 75 |e...   A| particu|
|000026a0| 6c 61 72 20 74 65 73 74 | 20 70 61 74 74 65 72 6e |lar test| pattern|
|000026b0| 20 63 61 6e 20 62 65 20 | 69 6e 70 75 74 20 74 6f | can be |input to|
|000026c0| 20 74 68 65 20 6e 65 74 | 77 6f 72 6b 20 77 69 74 | the net|work wit|
|000026d0| 68 20 74 68 65 20 60 70 | 27 0a 63 6f 6d 6d 61 6e |h the `p|'.comman|
|000026e0| 64 2c 20 61 73 20 69 6e | 3a 0a 0a 2e 6e 61 0a 2e |d, as in|:...na..|
|000026f0| 6e 66 0a 2e 6e 65 32 0a | 5b 3f 21 2a 41 61 62 43 |nf..ne2.|[?!*AabC|
|00002700| 63 45 65 66 68 69 6a 6b | 6c 6d 6e 6f 50 70 51 71 |cEefhijk|lmnoPpQq|
|00002710| 52 72 53 73 74 77 78 5d | 3f 20 70 20 31 20 30 0a |RrSstwx]|? p 1 0.|
|00002720| 20 20 20 20 20 30 2e 39 | 31 20 0a 2e 61 64 0a 2e |     0.9|1 ..ad..|
|00002730| 66 69 0a 0a 54 6f 20 68 | 61 76 65 20 74 68 65 20 |fi..To h|ave the |
|00002740| 73 79 73 74 65 6d 20 65 | 76 61 6c 75 61 74 65 20 |system e|valuate |
|00002750| 61 20 70 61 72 74 69 63 | 75 6c 61 72 20 73 74 6f |a partic|ular sto|
|00002760| 72 65 64 20 70 61 74 74 | 65 72 6e 2c 20 73 61 79 |red patt|ern, say|
|00002770| 20 70 61 74 74 65 72 6e | 0a 6e 75 6d 62 65 72 20 | pattern|.number |
|00002780| 34 2c 20 75 73 65 20 74 | 68 65 20 60 50 27 20 63 |4, use t|he `P' c|
|00002790| 6f 6d 6d 61 6e 64 20 61 | 73 20 69 6e 3a 0a 0a 2e |ommand a|s in:...|
|000027a0| 6e 61 0a 2e 6e 66 0a 2e | 6e 65 32 0a 5b 3f 21 2a |na..nf..|ne2.[?!*|
|000027b0| 41 61 62 43 63 45 65 66 | 48 68 69 6a 6b 6c 6d 6e |AabCcEef|Hhijklmn|
|000027c0| 6f 50 70 51 71 52 72 53 | 73 74 57 77 78 5d 3f 20 |oPpQqRrS|stWwx]? |
|000027d0| 50 34 0a 20 20 34 20 20 | 30 2e 30 39 20 20 28 30 |P4.  4  |0.09  (0|
|000027e0| 2e 30 30 37 33 39 29 0a | 2e 61 64 0a 2e 66 69 0a |.00739).|.ad..fi.|
|000027f0| 0a 54 6f 20 70 72 69 6e | 74 20 61 6c 6c 20 74 68 |.To prin|t all th|
|00002800| 65 20 76 61 6c 75 65 73 | 20 66 6f 72 20 61 6c 6c |e values| for all|
|00002810| 20 74 68 65 20 74 72 61 | 69 6e 69 6e 67 20 70 61 | the tra|ining pa|
|00002820| 74 74 65 72 6e 73 20 77 | 69 74 68 6f 75 74 20 64 |tterns w|ithout d|
|00002830| 6f 69 6e 67 0a 61 6e 79 | 20 6c 65 61 72 6e 69 6e |oing.any| learnin|
|00002840| 67 2c 20 74 79 70 65 20 | 60 50 27 3a 0a 0a 2e 6e |g, type |`P':...n|
|00002850| 61 0a 2e 6e 66 0a 2e 6e | 65 35 0a 5b 3f 21 2a 41 |a..nf..n|e5.[?!*A|
|00002860| 61 62 43 63 45 65 66 48 | 68 69 6a 6b 6c 6d 6e 6f |abCcEefH|hijklmno|
|00002870| 50 70 51 71 52 72 53 73 | 74 57 77 78 5d 3f 20 50 |PpQqRrSs|tWwx]? P|
|00002880| 0a 20 20 31 20 20 30 2e | 39 30 20 20 28 30 2e 30 |.  1  0.|90  (0.0|
|00002890| 30 39 37 33 29 0a 20 20 | 32 20 20 30 2e 30 37 20 |0973).  |2  0.07 |
|000028a0| 20 28 30 2e 30 30 34 36 | 37 29 0a 20 20 33 20 20 | (0.0046|7).  3  |
|000028b0| 30 2e 39 32 20 20 28 30 | 2e 30 30 35 36 35 29 0a |0.92  (0|.00565).|
|000028c0| 20 20 34 20 20 30 2e 30 | 39 20 20 28 30 2e 30 30 |  4  0.0|9  (0.00|
|000028d0| 37 33 39 29 0a 2e 61 64 | 0a 2e 66 69 0a 0a 20 20 |739)..ad|..fi..  |
|000028e0| 20 4f 6e 65 20 74 68 69 | 6e 67 20 79 6f 75 20 6d | One thi|ng you m|
|000028f0| 69 67 68 74 20 77 61 6e | 74 20 74 6f 20 6b 6e 6f |ight wan|t to kno|
|00002900| 77 20 61 72 65 20 74 68 | 65 20 76 61 6c 75 65 73 |w are th|e values|
|00002910| 20 6f 66 20 74 68 65 20 | 77 65 69 67 68 74 73 0a | of the |weights.|
|00002920| 74 68 61 74 20 68 61 76 | 65 20 62 65 65 6e 20 70 |that hav|e been p|
|00002930| 72 6f 64 75 63 65 64 2e | 20 20 54 6f 20 73 65 65 |roduced.|  To see|
|00002940| 20 74 68 69 73 2c 20 74 | 68 65 72 65 20 69 73 20 | this, t|here is |
|00002950| 74 68 65 20 60 77 27 20 | 63 6f 6d 6d 61 6e 64 2e |the `w' |command.|
|00002960| 0a 54 68 65 20 60 77 27 | 20 63 6f 6d 6d 61 6e 64 |.The `w'| command|
|00002970| 20 67 69 76 65 73 20 74 | 68 65 20 76 61 6c 75 65 | gives t|he value|
|00002980| 20 6f 66 20 74 68 65 20 | 77 65 69 67 68 74 73 20 | of the |weights |
|00002990| 6c 65 61 64 69 6e 67 20 | 69 6e 74 6f 0a 61 20 70 |leading |into.a p|
|000029a0| 61 72 74 69 63 75 6c 61 | 72 20 75 6e 69 74 20 61 |articula|r unit a|
|000029b0| 6e 64 20 61 6c 73 6f 20 | 64 61 74 61 20 61 62 6f |nd also |data abo|
|000029c0| 75 74 20 68 6f 77 20 74 | 68 65 20 61 63 74 69 76 |ut how t|he activ|
|000029d0| 61 74 69 6f 6e 20 76 61 | 6c 75 65 20 6f 66 20 74 |ation va|lue of t|
|000029e0| 68 65 0a 75 6e 69 74 20 | 69 73 20 63 6f 6d 70 75 |he.unit |is compu|
|000029f0| 74 65 64 2e 20 20 54 77 | 6f 20 69 6e 74 65 67 65 |ted.  Tw|o intege|
|00002a00| 72 73 20 61 66 74 65 72 | 20 74 68 65 20 77 20 73 |rs after| the w s|
|00002a10| 70 65 63 69 66 79 20 74 | 68 65 20 6c 61 79 65 72 |pecify t|he layer|
|00002a20| 20 61 6e 64 0a 75 6e 69 | 74 20 6e 75 6d 62 65 72 | and.uni|t number|
|00002a30| 20 77 69 74 68 69 6e 20 | 74 68 65 20 6c 61 79 65 | within |the laye|
|00002a40| 72 20 77 68 6f 73 65 20 | 77 65 69 67 68 74 73 20 |r whose |weights |
|00002a50| 73 68 6f 75 6c 64 20 62 | 65 20 70 72 69 6e 74 65 |should b|e printe|
|00002a60| 64 2e 20 20 46 6f 72 0a | 65 78 61 6d 70 6c 65 2c |d.  For.|example,|
|00002a70| 20 69 66 20 79 6f 75 20 | 77 61 6e 74 20 74 68 65 | if you |want the|
|00002a80| 20 77 65 69 67 68 74 73 | 20 6c 65 61 64 69 6e 67 | weights| leading|
|00002a90| 20 69 6e 74 6f 20 74 68 | 65 20 75 6e 69 74 20 61 | into th|e unit a|
|00002aa0| 74 20 6c 61 79 65 72 20 | 32 2c 0a 70 6f 73 69 74 |t layer |2,.posit|
|00002ab0| 69 6f 6e 20 6e 75 6d 62 | 65 72 20 31 2c 20 74 79 |ion numb|er 1, ty|
|00002ac0| 70 65 3a 0a 0a 2e 6e 61 | 0a 2e 6e 66 0a 2e 6e 65 |pe:...na|..nf..ne|
|00002ad0| 36 0a 5b 3f 21 2a 41 61 | 62 43 63 45 65 66 48 68 |6.[?!*Aa|bCcEefHh|
|00002ae0| 69 6a 6b 6c 6d 6e 6f 50 | 70 51 71 52 72 53 73 74 |ijklmnoP|pQqRrSst|
|00002af0| 57 77 78 5d 3f 20 77 20 | 32 20 31 0a 6c 61 79 65 |Wwx]? w |2 1.laye|
|00002b00| 72 20 75 6e 69 74 20 20 | 75 6e 69 74 20 76 61 6c |r unit  |unit val|
|00002b10| 75 65 20 20 20 20 20 77 | 65 69 67 68 74 20 20 20 |ue     w|eight   |
|00002b20| 20 20 20 20 20 20 69 6e | 70 75 74 20 66 72 6f 6d |      in|put from|
|00002b30| 20 75 6e 69 74 0a 20 20 | 31 20 20 20 20 20 20 31 | unit.  |1      1|
|00002b40| 20 20 20 20 31 2e 30 30 | 30 30 30 20 20 20 20 20 |    1.00|000     |
|00002b50| 37 2e 32 37 39 33 30 20 | 20 20 20 20 20 20 20 20 |7.27930 |        |
|00002b60| 20 20 20 20 37 2e 32 37 | 39 33 30 0a 20 20 31 20 |    7.27|930.  1 |
|00002b70| 20 20 20 20 20 32 20 20 | 20 20 31 2e 30 30 30 30 |     2  |  1.0000|
|00002b80| 30 20 20 20 20 2d 35 2e | 36 36 37 39 37 20 20 20 |0    -5.|66797   |
|00002b90| 20 20 20 20 20 20 20 20 | 20 2d 35 2e 36 36 37 39 |        | -5.6679|
|00002ba0| 37 0a 20 20 32 20 20 20 | 20 20 20 74 20 20 20 20 |7.  2   |   t    |
|00002bb0| 31 2e 30 30 30 30 30 20 | 20 20 20 20 32 2e 37 34 |1.00000 |    2.74|
|00002bc0| 39 30 32 20 20 20 20 20 | 20 20 20 20 20 20 20 20 |902     |        |
|00002bd0| 32 2e 37 34 39 30 32 0a | 20 20 20 20 20 20 20 20 |2.74902.|        |
|00002be0| 20 20 20 20 20 20 20 20 | 20 20 20 20 20 20 20 20 |        |        |
|00002bf0| 20 20 20 20 20 20 20 20 | 20 20 20 20 20 20 73 75 |        |      su|
|00002c00| 6d 20 3d 20 20 20 34 2e | 33 36 30 33 35 0a 2e 61 |m =   4.|36035..a|
|00002c10| 64 0a 2e 66 69 0a 0a 49 | 6e 20 74 68 69 73 20 65 |d..fi..I|n this e|
|00002c20| 78 61 6d 70 6c 65 2c 20 | 74 68 65 20 75 6e 69 74 |xample, |the unit|
|00002c30| 20 61 74 20 6c 61 79 65 | 72 20 32 2c 20 6e 75 6d | at laye|r 2, num|
|00002c40| 62 65 72 20 31 20 69 73 | 20 72 65 63 65 69 76 69 |ber 1 is| receivi|
|00002c50| 6e 67 20 69 6e 70 75 74 | 20 66 72 6f 6d 0a 75 6e |ng input| from.un|
|00002c60| 69 74 73 20 31 20 61 6e | 64 20 32 20 69 6e 20 74 |its 1 an|d 2 in t|
|00002c70| 68 65 20 70 72 65 76 69 | 6f 75 73 20 28 74 68 65 |he previ|ous (the|
|00002c80| 20 69 6e 70 75 74 29 20 | 6c 61 79 65 72 20 61 6e | input) |layer an|
|00002c90| 64 20 66 72 6f 6d 20 61 | 20 75 6e 69 74 2c 20 74 |d from a| unit, t|
|00002ca0| 2e 0a 55 6e 69 74 20 74 | 20 69 73 20 74 68 65 20 |..Unit t| is the |
|00002cb0| 74 68 72 65 73 68 6f 6c | 64 20 75 6e 69 74 2e 20 |threshol|d unit. |
|00002cc0| 20 54 68 65 20 22 75 6e | 69 74 20 76 61 6c 75 65 | The "un|it value|
|00002cd0| 22 20 63 6f 6c 75 6d 6e | 20 67 69 76 65 73 20 74 |" column| gives t|
|00002ce0| 68 65 20 76 61 6c 75 65 | 0a 6f 66 20 74 68 65 20 |he value|.of the |
|00002cf0| 69 6e 70 75 74 20 75 6e | 69 74 73 20 66 6f 72 20 |input un|its for |
|00002d00| 74 68 65 20 6c 61 73 74 | 20 74 69 6d 65 20 73 6f |the last| time so|
|00002d10| 6d 65 20 70 61 74 74 65 | 72 6e 20 77 61 73 20 70 |me patte|rn was p|
|00002d20| 6c 61 63 65 64 20 6f 6e | 20 74 68 65 0a 69 6e 70 |laced on| the.inp|
|00002d30| 75 74 20 75 6e 69 74 73 | 2e 20 20 49 6e 20 74 68 |ut units|.  In th|
|00002d40| 69 73 20 63 61 73 65 2c | 20 74 68 65 20 66 6f 75 |is case,| the fou|
|00002d50| 72 74 68 20 70 61 74 74 | 65 72 6e 20 77 61 73 20 |rth patt|ern was |
|00002d60| 74 68 65 20 6c 61 73 74 | 20 6f 6e 65 20 74 68 61 |the last| one tha|
|00002d70| 74 20 74 68 65 0a 6e 65 | 74 77 6f 72 6b 20 68 61 |t the.ne|twork ha|
|00002d80| 73 20 73 65 65 6e 2e 20 | 20 54 68 65 20 6e 65 78 |s seen. | The nex|
|00002d90| 74 20 63 6f 6c 75 6d 6e | 20 6c 69 73 74 73 20 74 |t column| lists t|
|00002da0| 68 65 20 77 65 69 67 68 | 74 73 20 6f 6e 20 74 68 |he weigh|ts on th|
|00002db0| 65 20 63 6f 6e 6e 65 63 | 74 69 6f 6e 73 0a 69 6e |e connec|tions.in|
|00002dc0| 74 6f 20 74 68 65 20 75 | 6e 69 74 20 61 74 20 28 |to the u|nit at (|
|00002dd0| 32 2c 31 29 2e 20 20 54 | 68 65 20 66 69 6e 61 6c |2,1).  T|he final|
|00002de0| 20 63 6f 6c 75 6d 6e 20 | 69 73 20 74 68 65 20 72 | column |is the r|
|00002df0| 65 73 75 6c 74 20 66 72 | 6f 6d 20 6d 75 6c 74 69 |esult fr|om multi|
|00002e00| 70 6c 79 69 6e 67 0a 74 | 6f 67 65 74 68 65 72 20 |plying.t|ogether |
|00002e10| 74 68 65 20 75 6e 69 74 | 20 76 61 6c 75 65 20 61 |the unit| value a|
|00002e20| 6e 64 20 74 68 65 20 77 | 65 69 67 68 74 2e 20 20 |nd the w|eight.  |
|00002e30| 42 65 6e 65 61 74 68 20 | 74 68 69 73 20 63 6f 6c |Beneath |this col|
|00002e40| 75 6d 6e 2c 20 74 68 65 | 20 73 75 6d 20 6f 66 0a |umn, the| sum of.|
|00002e50| 74 68 65 20 69 6e 70 75 | 74 73 20 69 73 20 67 69 |the inpu|ts is gi|
|00002e60| 76 65 6e 2e 0a 0a 20 20 | 20 41 6e 6f 74 68 65 72 |ven...  | Another|
|00002e70| 20 69 6d 70 6f 72 74 61 | 6e 74 20 63 6f 6d 6d 61 | importa|nt comma|
|00002e80| 6e 64 20 69 73 20 74 68 | 65 20 68 65 6c 70 20 63 |nd is th|e help c|
|00002e90| 6f 6d 6d 61 6e 64 2e 20 | 20 49 74 20 69 73 20 74 |ommand. | It is t|
|00002ea0| 68 65 20 6c 65 74 74 65 | 72 0a 60 68 27 20 28 6e |he lette|r.`h' (n|
|00002eb0| 6f 74 20 60 3f 27 29 20 | 66 6f 6c 6c 6f 77 65 64 |ot `?') |followed|
|00002ec0| 20 62 79 20 74 68 65 20 | 6c 65 74 74 65 72 20 6f | by the |letter o|
|00002ed0| 66 20 74 68 65 20 63 6f | 6d 6d 61 6e 64 2e 20 20 |f the co|mmand.  |
|00002ee0| 54 68 65 20 68 65 6c 70 | 20 63 6f 6d 6d 61 6e 64 |The help| command|
|00002ef0| 0a 77 69 6c 6c 20 67 69 | 76 65 20 61 20 62 72 69 |.will gi|ve a bri|
|00002f00| 65 66 20 73 75 6d 6d 61 | 72 79 20 6f 66 20 68 6f |ef summa|ry of ho|
|00002f10| 77 20 74 6f 20 75 73 65 | 20 74 68 65 20 63 6f 6d |w to use| the com|
|00002f20| 6d 61 6e 64 2e 20 20 48 | 65 72 65 2c 20 77 65 20 |mand.  H|ere, we |
|00002f30| 74 79 70 65 0a 68 20 68 | 20 66 6f 72 20 68 65 6c |type.h h| for hel|
|00002f40| 70 20 77 69 74 68 20 68 | 65 6c 70 3a 0a 0a 2e 6e |p with h|elp:...n|
|00002f50| 61 0a 2e 6e 66 0a 2e 6e | 65 33 0a 5b 3f 21 2a 41 |a..nf..n|e3.[?!*A|
|00002f60| 61 62 43 63 45 65 66 48 | 68 69 6a 6b 6c 6d 6e 6f |abCcEefH|hijklmno|
|00002f70| 50 70 51 71 52 72 53 73 | 74 57 77 78 5d 3f 20 68 |PpQqRrSs|tWwx]? h|
|00002f80| 20 68 0a 0a 68 20 3c 6c | 65 74 74 65 72 3e 20 67 | h..h <l|etter> g|
|00002f90| 69 76 65 73 20 68 65 6c | 70 20 66 6f 72 20 63 6f |ives hel|p for co|
|00002fa0| 6d 6d 61 6e 64 20 3c 6c | 65 74 74 65 72 3e 2e 0a |mmand <l|etter>..|
|00002fb0| 2e 61 64 0a 2e 66 69 0a | 0a 20 20 20 46 69 6e 61 |.ad..fi.|.   Fina|
|00002fc0| 6c 6c 79 2c 20 74 6f 20 | 65 6e 64 20 74 68 65 20 |lly, to |end the |
|00002fd0| 70 72 6f 67 72 61 6d 2c | 20 74 68 65 20 60 71 27 |program,| the `q'|
|00002fe0| 20 28 66 6f 72 20 71 75 | 69 74 29 20 63 6f 6d 6d | (for qu|it) comm|
|00002ff0| 61 6e 64 20 69 73 20 65 | 6e 74 65 72 65 64 3a 0a |and is e|ntered:.|
|00003000| 0a 5b 3f 21 2a 41 61 62 | 43 63 45 65 66 48 68 69 |.[?!*Aab|CcEefHhi|
|00003010| 6a 6b 6c 6d 6e 6f 50 70 | 51 71 52 72 53 73 74 57 |jklmnoPp|QqRrSstW|
|00003020| 77 78 5d 3f 20 71 0a 0a | 2e 75 6c 0a 49 6e 70 75 |wx]? q..|.ul.Inpu|
|00003030| 74 20 61 6e 64 20 4f 75 | 74 70 75 74 20 46 6f 72 |t and Ou|tput For|
|00003040| 6d 61 74 73 0a 0a 20 20 | 20 54 68 65 20 70 72 6f |mats..  | The pro|
|00003050| 67 72 61 6d 73 20 61 72 | 65 20 61 62 6c 65 20 74 |grams ar|e able t|
|00003060| 6f 20 72 65 61 64 20 70 | 61 74 74 65 72 6e 73 20 |o read p|atterns |
|00003070| 69 6e 20 74 77 6f 20 64 | 69 66 66 65 72 65 6e 74 |in two d|ifferent|
|00003080| 20 66 6f 72 6d 61 74 73 | 2e 20 20 54 68 65 0a 64 | formats|.  The.d|
|00003090| 65 66 61 75 6c 74 20 69 | 6e 70 75 74 20 66 6f 72 |efault i|nput for|
|000030a0| 6d 61 74 20 69 73 20 74 | 68 65 20 63 6f 6d 70 72 |mat is t|he compr|
|000030b0| 65 73 73 65 64 20 28 63 | 6f 6e 64 65 6e 73 65 64 |essed (c|ondensed|
|000030c0| 29 20 66 6f 72 6d 61 74 | 2e 20 20 49 6e 20 69 74 |) format|.  In it|
|000030d0| 2c 20 65 61 63 68 0a 76 | 61 6c 75 65 20 69 73 20 |, each.v|alue is |
|000030e0| 6f 6e 65 20 63 68 61 72 | 61 63 74 65 72 20 61 6e |one char|acter an|
|000030f0| 64 20 69 74 20 69 73 20 | 6e 6f 74 20 6e 65 63 65 |d it is |not nece|
|00003100| 73 73 61 72 79 20 74 6f | 20 68 61 76 65 20 62 6c |ssary to| have bl|
|00003110| 61 6e 6b 73 20 62 65 74 | 77 65 65 6e 0a 74 68 65 |anks bet|ween.the|
|00003120| 20 63 68 61 72 61 63 74 | 65 72 73 2e 20 20 46 6f | charact|ers.  Fo|
|00003130| 72 20 65 78 61 6d 70 6c | 65 2c 20 69 6e 20 63 6f |r exampl|e, in co|
|00003140| 6d 70 72 65 73 73 65 64 | 20 66 6f 72 6d 61 74 2c |mpressed| format,|
|00003150| 20 74 68 65 20 70 61 74 | 74 65 72 6e 73 20 66 6f | the pat|terns fo|
|00003160| 72 20 78 6f 72 0a 63 6f | 75 6c 64 20 62 65 20 77 |r xor.co|uld be w|
|00003170| 72 69 74 74 65 6e 20 6f | 75 74 20 69 6e 20 65 69 |ritten o|ut in ei|
|00003180| 74 68 65 72 20 6f 66 20 | 74 68 65 20 66 6f 6c 6c |ther of |the foll|
|00003190| 6f 77 69 6e 67 20 77 61 | 79 73 3a 0a 0a 2e 63 65 |owing wa|ys:...ce|
|000031a0| 0a 31 30 31 20 20 20 20 | 20 20 20 20 20 20 20 20 |.101    |        |
|000031b0| 20 20 20 31 30 20 31 0a | 2e 63 65 0a 30 30 30 20 |   10 1.|.ce.000 |
|000031c0| 20 20 20 20 20 20 20 20 | 20 20 20 20 20 20 30 30 |        |      00|
|000031d0| 20 30 0a 2e 63 65 0a 30 | 31 31 20 20 20 20 20 20 | 0..ce.0|11      |
|000031e0| 20 20 20 20 20 20 20 20 | 20 30 31 20 31 0a 2e 63 |        | 01 1..c|
|000031f0| 65 0a 31 31 30 20 20 20 | 20 20 20 20 20 20 20 20 |e.110   |        |
|00003200| 20 20 20 20 31 31 20 30 | 0a 0a 54 68 65 20 73 65 |    11 0|..The se|
|00003210| 63 6f 6e 64 20 65 78 61 | 6d 70 6c 65 20 69 73 20 |cond exa|mple is |
|00003220| 70 72 65 66 65 72 61 62 | 6c 65 20 62 65 63 61 75 |preferab|le becau|
|00003230| 73 65 20 69 74 20 6d 61 | 6b 65 73 20 69 74 0a 65 |se it ma|kes it.e|
|00003240| 61 73 69 65 72 20 74 6f | 20 73 65 65 20 74 68 65 |asier to| see the|
|00003250| 20 69 6e 70 75 74 20 61 | 6e 64 20 74 68 65 20 6f | input a|nd the o|
|00003260| 75 74 70 75 74 20 70 61 | 74 74 65 72 6e 73 2e 20 |utput pa|tterns. |
|00003270| 20 43 6f 6d 70 72 65 73 | 73 65 64 20 66 6f 72 6d | Compres|sed form|
|00003280| 61 74 20 63 61 6e 0a 61 | 6c 73 6f 20 62 65 20 75 |at can.a|lso be u|
|00003290| 73 65 64 20 74 6f 20 69 | 6e 70 75 74 20 70 61 74 |sed to i|nput pat|
|000032a0| 74 65 72 6e 73 20 77 69 | 74 68 20 74 68 65 20 60 |terns wi|th the `|
|000032b0| 70 27 20 63 6f 6d 6d 61 | 6e 64 2e 0a 49 6e 20 61 |p' comma|nd..In a|
|000032c0| 64 64 69 74 69 6f 6e 20 | 74 6f 20 75 73 69 6e 67 |ddition |to using|
|000032d0| 20 31 20 61 6e 64 20 30 | 20 61 73 20 69 6e 70 75 | 1 and 0| as inpu|
|000032e0| 74 2c 20 74 68 65 20 63 | 68 61 72 61 63 74 65 72 |t, the c|haracter|
|000032f0| 2c 20 60 3f 27 20 63 61 | 6e 20 62 65 20 75 73 65 |, `?' ca|n be use|
|00003300| 64 2e 0a 54 68 69 73 20 | 63 68 61 72 61 63 74 65 |d..This |characte|
|00003310| 72 20 69 73 20 69 6e 69 | 74 69 61 6c 6c 79 20 64 |r is ini|tially d|
|00003320| 65 66 69 6e 65 64 20 74 | 6f 20 62 65 20 30 2e 35 |efined t|o be 0.5|
|00003330| 2c 20 62 75 74 20 69 74 | 20 63 61 6e 20 62 65 20 |, but it| can be |
|00003340| 72 65 64 65 66 69 6e 65 | 64 0a 75 73 69 6e 67 20 |redefine|d.using |
|00003350| 74 68 65 20 51 20 63 6f | 6d 6d 61 6e 64 20 6c 69 |the Q co|mmand li|
|00003360| 6b 65 20 73 6f 3a 0a 0a | 2e 63 65 0a 51 20 30 2e |ke so:..|.ce.Q 0.|
|00003370| 37 0a 0a 54 68 69 73 20 | 73 65 74 73 20 74 68 65 |7..This |sets the|
|00003380| 20 76 61 6c 75 65 20 6f | 66 20 3f 20 74 6f 20 30 | value o|f ? to 0|
|00003390| 2e 37 2e 20 20 4f 74 68 | 65 72 20 76 61 6c 69 64 |.7.  Oth|er valid|
|000033a0| 20 69 6e 70 75 74 20 63 | 68 61 72 61 63 74 65 72 | input c|haracter|
|000033b0| 73 20 61 72 65 20 74 68 | 65 0a 6c 65 74 74 65 72 |s are th|e.letter|
|000033c0| 73 2c 20 60 68 27 2c 20 | 60 69 27 2c 20 60 6a 27 |s, `h', |`i', `j'|
|000033d0| 20 61 6e 64 20 60 6b 27 | 2e 20 20 54 68 65 20 60 | and `k'|.  The `|
|000033e0| 68 27 20 73 74 61 6e 64 | 73 20 66 6f 72 20 60 68 |h' stand|s for `h|
|000033f0| 69 64 64 65 6e 27 2e 20 | 20 49 74 73 0a 6d 65 61 |idden'. | Its.mea|
|00003400| 6e 69 6e 67 20 69 6e 20 | 61 6e 20 69 6e 70 75 74 |ning in |an input|
|00003410| 20 73 74 72 69 6e 67 20 | 69 73 20 74 68 61 74 20 | string |is that |
|00003420| 74 68 65 20 76 61 6c 75 | 65 20 61 74 20 74 68 69 |the valu|e at thi|
|00003430| 73 20 70 6f 69 6e 74 20 | 69 6e 20 74 68 65 20 73 |s point |in the s|
|00003440| 74 72 69 6e 67 0a 73 68 | 6f 75 6c 64 20 62 65 20 |tring.sh|ould be |
|00003450| 74 61 6b 65 6e 20 66 72 | 6f 6d 20 74 68 65 20 6e |taken fr|om the n|
|00003460| 65 78 74 20 75 6e 69 74 | 20 69 6e 20 74 68 65 20 |ext unit| in the |
|00003470| 73 65 63 6f 6e 64 20 6c | 61 79 65 72 20 6f 66 20 |second l|ayer of |
|00003480| 74 68 65 20 6e 65 74 77 | 6f 72 6b 2e 0a 4e 6f 72 |the netw|ork..Nor|
|00003490| 6d 61 6c 6c 79 20 74 68 | 69 73 20 77 69 6c 6c 20 |mally th|is will |
|000034a0| 62 65 20 74 68 65 20 73 | 65 63 6f 6e 64 20 6c 61 |be the s|econd la|
|000034b0| 79 65 72 20 6f 66 20 61 | 20 74 68 72 65 65 2d 6c |yer of a| three-l|
|000034c0| 61 79 65 72 20 6e 65 74 | 77 6f 72 6b 2e 20 20 54 |ayer net|work.  T|
|000034d0| 68 69 73 0a 6e 6f 74 61 | 74 69 6f 6e 20 69 73 20 |his.nota|tion is |
|000034e0| 75 73 65 66 75 6c 20 66 | 6f 72 20 73 70 65 63 69 |useful f|or speci|
|000034f0| 66 79 69 6e 67 20 73 69 | 6d 70 6c 65 20 72 65 63 |fying si|mple rec|
|00003500| 75 72 72 65 6e 74 0a 6e | 65 74 77 6f 72 6b 73 2e |urrent.n|etworks.|
|00003510| 20 20 4e 61 74 75 72 61 | 6c 6c 79 2c 20 60 69 27 |  Natura|lly, `i'|
|00003520| 2c 20 60 6a 27 20 61 6e | 64 20 60 6b 27 20 73 74 |, `j' an|d `k' st|
|00003530| 61 6e 64 20 66 6f 72 20 | 74 61 6b 69 6e 67 20 69 |and for |taking i|
|00003540| 6e 70 75 74 0a 76 61 6c | 75 65 73 20 66 72 6f 6d |nput.val|ues from|
|00003550| 20 74 68 65 20 74 68 69 | 72 64 2c 20 66 6f 75 72 | the thi|rd, four|
|00003560| 74 68 20 61 6e 64 20 66 | 69 66 74 68 20 6c 61 79 |th and f|ifth lay|
|00003570| 65 72 73 20 28 69 66 20 | 74 68 65 79 20 65 78 69 |ers (if |they exi|
|00003580| 73 74 29 2e 20 20 41 0a | 73 69 6d 70 6c 65 20 65 |st).  A.|simple e|
|00003590| 78 61 6d 70 6c 65 20 6f | 66 20 61 20 72 65 63 75 |xample o|f a recu|
|000035a0| 72 72 65 6e 74 20 6e 65 | 74 77 6f 72 6b 20 69 73 |rrent ne|twork is|
|000035b0| 20 67 69 76 65 6e 20 6c | 61 74 65 72 2e 0a 0a 20 | given l|ater... |
|000035c0| 20 20 54 68 65 20 6f 74 | 68 65 72 20 69 6e 70 75 |  The ot|her inpu|
|000035d0| 74 20 66 6f 72 6d 61 74 | 20 66 6f 72 20 6e 75 6d |t format| for num|
|000035e0| 62 65 72 73 20 69 73 20 | 72 65 61 6c 2e 20 20 54 |bers is |real.  T|
|000035f0| 68 65 20 6e 75 6d 62 65 | 72 20 70 6f 72 74 69 6f |he numbe|r portio|
|00003600| 6e 20 6d 75 73 74 0a 73 | 74 61 72 74 20 77 69 74 |n must.s|tart wit|
|00003610| 68 20 61 20 64 69 67 69 | 74 20 28 2e 33 35 20 69 |h a digi|t (.35 i|
|00003620| 73 20 6e 6f 74 20 61 6c | 6c 6f 77 65 64 2c 20 62 |s not al|lowed, b|
|00003630| 75 74 20 30 2e 33 35 20 | 69 73 29 2e 20 20 45 78 |ut 0.35 |is).  Ex|
|00003640| 70 6f 6e 65 6e 74 69 61 | 6c 0a 6e 6f 74 61 74 69 |ponentia|l.notati|
|00003650| 6f 6e 20 69 73 20 6e 6f | 74 20 61 6c 6c 6f 77 65 |on is no|t allowe|
|00003660| 64 2e 20 20 52 65 61 6c | 20 6e 75 6d 62 65 72 73 |d.  Real| numbers|
|00003670| 20 68 61 76 65 20 74 6f | 20 62 65 20 73 65 70 61 | have to| be sepa|
|00003680| 72 61 74 65 64 20 62 79 | 20 61 20 73 70 61 63 65 |rated by| a space|
|00003690| 2e 0a 54 68 65 20 60 68 | 27 2c 20 60 69 27 2c 20 |..The `h|', `i', |
|000036a0| 60 6a 27 2c 20 60 6b 27 | 20 61 6e 64 20 60 3f 27 |`j', `k'| and `?'|
|000036b0| 20 63 68 61 72 61 63 74 | 65 72 73 20 61 72 65 20 | charact|ers are |
|000036c0| 61 6c 73 6f 20 61 6c 6c | 6f 77 65 64 20 77 69 74 |also all|owed wit|
|000036d0| 68 20 72 65 61 6c 0a 69 | 6e 70 75 74 20 70 61 74 |h real.i|nput pat|
|000036e0| 74 65 72 6e 73 2e 20 20 | 54 6f 20 74 61 6b 65 20 |terns.  |To take |
|000036f0| 69 6e 70 75 74 20 69 6e | 20 74 68 69 73 20 66 6f |input in| this fo|
|00003700| 72 6d 61 74 2c 20 69 74 | 20 69 73 20 6e 65 63 65 |rmat, it| is nece|
|00003710| 73 73 61 72 79 0a 74 6f | 20 73 65 74 20 74 68 65 |ssary.to| set the|
|00003720| 20 69 6e 70 75 74 20 66 | 6f 72 6d 61 74 20 74 6f | input f|ormat to|
|00003730| 20 62 65 20 72 65 61 6c | 20 75 73 69 6e 67 20 74 | be real| using t|
|00003740| 68 65 20 60 66 27 20 28 | 66 6f 72 6d 61 74 29 20 |he `f' (|format) |
|00003750| 63 6f 6d 6d 61 6e 64 20 | 61 73 20 69 6e 3a 0a 0a |command |as in:..|
|00003760| 2e 63 65 0a 66 20 69 72 | 0a 0a 54 6f 20 63 68 61 |.ce.f ir|..To cha|
|00003770| 6e 67 65 20 62 61 63 6b | 20 74 6f 20 74 68 65 20 |nge back| to the |
|00003780| 63 6f 6d 70 72 65 73 73 | 65 64 20 66 6f 72 6d 61 |compress|ed forma|
|00003790| 74 2c 20 75 73 65 3a 0a | 0a 2e 63 65 0a 66 20 69 |t, use:.|..ce.f i|
|000037a0| 63 0a 0a 4f 75 74 70 75 | 74 20 66 6f 72 6d 61 74 |c..Outpu|t format|
|000037b0| 20 69 73 20 63 6f 6e 74 | 72 6f 6c 6c 65 64 20 77 | is cont|rolled w|
|000037c0| 69 74 68 20 74 68 65 20 | 60 66 27 20 63 6f 6d 6d |ith the |`f' comm|
|000037d0| 61 6e 64 20 61 73 20 69 | 6e 3a 0a 0a 2e 63 65 0a |and as i|n:...ce.|
|000037e0| 66 20 6f 72 0a 2e 63 65 | 0a 66 20 6f 63 0a 2e 63 |f or..ce|.f oc..c|
|000037f0| 65 0a 66 20 6f 61 0a 0a | 54 68 65 20 66 69 72 73 |e.f oa..|The firs|
|00003800| 74 20 73 65 74 73 20 74 | 68 65 20 6f 75 74 70 75 |t sets t|he outpu|
|00003810| 74 20 74 6f 20 72 65 61 | 6c 20 6e 75 6d 62 65 72 |t to rea|l number|
|00003820| 73 2e 20 20 54 68 65 20 | 73 65 63 6f 6e 64 20 73 |s.  The |second s|
|00003830| 65 74 73 20 74 68 65 0a | 6f 75 74 70 75 74 20 74 |ets the.|output t|
|00003840| 6f 20 62 65 20 63 6f 6e | 64 65 6e 73 65 64 20 6d |o be con|densed m|
|00003850| 6f 64 65 20 77 68 65 72 | 65 20 74 68 65 20 76 61 |ode wher|e the va|
|00003860| 6c 75 65 20 70 72 69 6e | 74 65 64 20 77 69 6c 6c |lue prin|ted will|
|00003870| 20 62 65 20 61 20 60 31 | 27 20 77 68 65 6e 0a 74 | be a `1|' when.t|
|00003880| 68 65 20 75 6e 69 74 20 | 76 61 6c 75 65 20 69 73 |he unit |value is|
|00003890| 20 67 72 65 61 74 65 72 | 20 74 68 61 6e 20 31 2e | greater| than 1.|
|000038a0| 30 20 2d 20 74 6f 6c 65 | 72 61 6e 63 65 2c 20 61 |0 - tole|rance, a|
|000038b0| 20 60 5e 27 20 77 68 65 | 6e 20 74 68 65 20 76 61 | `^' whe|n the va|
|000038c0| 6c 75 65 0a 69 73 20 61 | 62 6f 76 65 20 30 2e 35 |lue.is a|bove 0.5|
|000038d0| 20 62 75 74 20 6c 65 73 | 73 20 74 68 61 6e 20 31 | but les|s than 1|
|000038e0| 2e 30 20 2d 20 74 6f 6c | 65 72 61 6e 63 65 2c 20 |.0 - tol|erance, |
|000038f0| 61 20 60 76 27 20 77 68 | 65 6e 20 74 68 65 20 76 |a `v' wh|en the v|
|00003900| 61 6c 75 65 20 69 73 0a | 6c 65 73 73 20 74 68 61 |alue is.|less tha|
|00003910| 6e 20 30 2e 35 20 62 75 | 74 20 67 72 65 61 74 65 |n 0.5 bu|t greate|
|00003920| 72 20 74 68 61 6e 20 74 | 68 65 20 74 6f 6c 65 72 |r than t|he toler|
|00003930| 61 6e 63 65 2e 20 20 42 | 65 6c 6f 77 20 74 68 65 |ance.  B|elow the|
|00003940| 20 74 6f 6c 65 72 61 6e | 63 65 0a 76 61 6c 75 65 | toleran|ce.value|
|00003950| 2c 20 61 20 60 30 27 20 | 69 73 20 70 72 69 6e 74 |, a `0' |is print|
|00003960| 65 64 2e 20 20 54 68 65 | 20 74 6f 6c 65 72 61 6e |ed.  The| toleran|
|00003970| 63 65 20 63 61 6e 20 62 | 65 20 63 68 61 6e 67 65 |ce can b|e change|
|00003980| 64 20 75 73 69 6e 67 20 | 74 68 65 20 60 74 27 0a |d using |the `t'.|
|00003990| 63 6f 6d 6d 61 6e 64 2e | 20 20 46 6f 72 20 65 78 |command.|  For ex|
|000039a0| 61 6d 70 6c 65 2c 20 74 | 6f 20 6d 61 6b 65 20 61 |ample, t|o make a|
|000039b0| 6c 6c 20 76 61 6c 75 65 | 73 20 67 72 65 61 74 65 |ll value|s greate|
|000039c0| 72 20 74 68 61 6e 20 30 | 2e 38 20 70 72 69 6e 74 |r than 0|.8 print|
|000039d0| 0a 61 73 20 60 31 27 20 | 61 6e 64 20 61 6c 6c 20 |.as `1' |and all |
|000039e0| 76 61 6c 75 65 73 20 6c | 65 73 73 20 74 68 61 6e |values l|ess than|
|000039f0| 20 30 2e 32 20 70 72 69 | 6e 74 20 61 73 20 60 30 | 0.2 pri|nt as `0|
|00003a00| 27 2c 20 75 73 65 3a 0a | 0a 2e 63 65 0a 74 20 30 |', use:.|..ce.t 0|
|00003a10| 2e 32 0a 0a 4f 66 20 63 | 6f 75 72 73 65 2c 20 74 |.2..Of c|ourse, t|
|00003a20| 68 69 73 20 73 61 6d 65 | 20 74 6f 6c 65 72 61 6e |his same| toleran|
|00003a30| 63 65 20 76 61 6c 75 65 | 20 69 73 20 61 6c 73 6f |ce value| is also|
|00003a40| 20 75 73 65 64 20 74 6f | 20 63 68 65 63 6b 20 74 | used to| check t|
|00003a50| 6f 20 73 65 65 20 69 66 | 20 61 6c 6c 0a 74 68 65 |o see if| all.the|
|00003a60| 20 70 61 74 74 65 72 6e | 73 20 68 61 76 65 20 63 | pattern|s have c|
|00003a70| 6f 6e 76 65 72 67 65 64 | 2e 20 20 54 68 65 20 74 |onverged|.  The t|
|00003a80| 68 69 72 64 20 6f 75 74 | 70 75 74 20 66 6f 72 6d |hird out|put form|
|00003a90| 61 74 20 69 73 20 6d 65 | 61 6e 74 20 74 6f 0a 67 |at is me|ant to.g|
|00003aa0| 69 76 65 20 22 61 6e 61 | 6c 6f 67 20 63 6f 6e 64 |ive "ana|log cond|
|00003ab0| 65 6e 73 65 64 22 20 6f | 75 74 70 75 74 2e 20 20 |ensed" o|utput.  |
|00003ac0| 49 6e 20 74 68 69 73 20 | 66 6f 72 6d 61 74 2c 20 |In this |format, |
|00003ad0| 61 20 60 63 27 20 69 73 | 20 70 72 69 6e 74 65 64 |a `c' is| printed|
|00003ae0| 20 77 68 65 6e 0a 61 20 | 76 61 6c 75 65 20 69 73 | when.a |value is|
|00003af0| 20 63 6c 6f 73 65 20 65 | 6e 6f 75 67 68 20 74 6f | close e|nough to|
|00003b00| 20 69 74 73 20 74 61 72 | 67 65 74 20 76 61 6c 75 | its tar|get valu|
|00003b10| 65 2e 20 20 4f 74 68 65 | 72 77 69 73 65 2c 20 69 |e.  Othe|rwise, i|
|00003b20| 66 20 74 68 65 20 61 6e | 73 77 65 72 0a 69 73 20 |f the an|swer.is |
|00003b30| 63 6c 6f 73 65 20 74 6f | 0a 31 2c 20 61 20 60 31 |close to|.1, a `1|
|00003b40| 27 20 69 73 20 70 72 69 | 6e 74 65 64 2c 20 69 66 |' is pri|nted, if|
|00003b50| 20 74 68 65 20 61 6e 73 | 77 65 72 20 69 73 20 63 | the ans|wer is c|
|00003b60| 6c 6f 73 65 20 74 6f 20 | 30 2c 20 61 20 60 30 27 |lose to |0, a `0'|
|00003b70| 20 69 73 20 70 72 69 6e | 74 65 64 2c 20 69 66 0a | is prin|ted, if.|
|00003b80| 74 68 65 20 61 6e 73 77 | 65 72 20 69 73 20 61 62 |the answ|er is ab|
|00003b90| 6f 76 65 20 74 68 65 20 | 74 61 72 67 65 74 20 62 |ove the |target b|
|00003ba0| 75 74 20 6e 6f 74 20 63 | 6c 6f 73 65 20 74 6f 20 |ut not c|lose to |
|00003bb0| 31 2c 20 61 20 60 5e 27 | 20 69 73 20 70 72 69 6e |1, a `^'| is prin|
|00003bc0| 74 65 64 20 61 6e 64 0a | 69 66 20 74 68 65 20 61 |ted and.|if the a|
|00003bd0| 6e 73 77 65 72 20 69 73 | 20 62 65 6c 6f 77 20 74 |nswer is| below t|
|00003be0| 68 65 20 74 61 72 67 65 | 74 20 62 75 74 20 6e 6f |he targe|t but no|
|00003bf0| 74 20 63 6c 6f 73 65 20 | 74 6f 20 30 2c 20 61 20 |t close |to 0, a |
|00003c00| 60 76 27 20 69 73 20 70 | 72 69 6e 74 65 64 2e 0a |`v' is p|rinted..|
|00003c10| 54 68 69 73 20 6f 75 74 | 70 75 74 20 66 6f 72 6d |This out|put form|
|00003c20| 61 74 20 69 73 20 64 65 | 73 69 67 6e 65 64 20 66 |at is de|signed f|
|00003c30| 6f 72 0a 70 72 6f 62 6c | 65 6d 73 20 77 68 65 72 |or.probl|ems wher|
|00003c40| 65 20 74 68 65 20 6f 75 | 74 70 75 74 20 69 73 20 |e the ou|tput is |
|00003c50| 61 20 72 65 61 6c 20 6e | 75 6d 62 65 72 2c 20 61 |a real n|umber, a|
|00003c60| 73 20 66 6f 72 20 69 6e | 73 74 61 6e 63 65 2c 20 |s for in|stance, |
|00003c70| 77 68 65 6e 20 74 68 65 | 0a 70 72 6f 62 6c 65 6d |when the|.problem|
|00003c80| 20 69 73 20 74 6f 20 6d | 61 6b 65 20 61 20 6e 65 | is to m|ake a ne|
|00003c90| 74 77 6f 72 6b 20 6c 65 | 61 72 6e 20 73 69 6e 28 |twork le|arn sin(|
|00003ca0| 78 29 2e 0a 0a 20 20 20 | 57 69 74 68 20 74 68 65 |x)...   |With the|
|00003cb0| 20 66 20 63 6f 6d 6d 61 | 6e 64 2c 20 61 20 6e 75 | f comma|nd, a nu|
|00003cc0| 6d 62 65 72 20 6f 66 20 | 73 75 62 2d 63 6f 6d 6d |mber of |sub-comm|
|00003cd0| 61 6e 64 73 20 63 61 6e | 20 62 65 20 70 75 74 20 |ands can| be put |
|00003ce0| 6f 6e 20 6f 6e 65 20 6c | 69 6e 65 0a 61 73 20 69 |on one l|ine.as i|
|00003cf0| 6e 20 74 68 65 20 66 6f | 6c 6c 6f 77 69 6e 67 2c |n the fo|llowing,|
|00003d00| 20 77 68 65 72 65 20 74 | 68 65 20 69 6e 70 75 74 | where t|he input|
|00003d10| 20 69 73 20 73 65 74 20 | 74 6f 20 72 65 61 6c 20 | is set |to real |
|00003d20| 61 6e 64 20 74 68 65 20 | 6f 75 74 70 75 74 0a 69 |and the |output.i|
|00003d30| 73 20 73 65 74 20 74 6f | 20 61 6e 61 6c 6f 67 20 |s set to| analog |
|00003d40| 63 6f 6e 64 65 6e 73 65 | 64 3a 0a 0a 2e 63 65 0a |condense|d:...ce.|
|00003d50| 66 20 69 72 20 6f 61 0a | 0a 41 6c 73 6f 2c 20 66 |f ir oa.|.Also, f|
|00003d60| 6f 72 20 74 68 65 20 73 | 61 6b 65 20 6f 66 20 63 |or the s|ake of c|
|00003d70| 6f 6e 76 65 6e 69 65 6e | 63 65 2c 20 74 68 65 20 |onvenien|ce, the |
|00003d80| 6f 75 74 70 75 74 20 66 | 6f 72 6d 61 74 20 28 61 |output f|ormat (a|
|00003d90| 6e 64 20 6f 6e 6c 79 20 | 74 68 65 0a 6f 75 74 70 |nd only |the.outp|
|00003da0| 75 74 20 66 6f 72 6d 61 | 74 29 20 63 61 6e 20 62 |ut forma|t) can b|
|00003db0| 65 20 73 65 74 20 77 69 | 74 68 6f 75 74 20 75 73 |e set wi|thout us|
|00003dc0| 69 6e 67 20 74 68 65 20 | 60 66 27 2c 20 73 6f 20 |ing the |`f', so |
|00003dd0| 74 68 61 74 3a 0a 0a 2e | 63 65 0a 6f 72 0a 0a 77 |that:...|ce.or..w|
|00003de0| 69 6c 6c 20 61 6c 73 6f | 20 6d 61 6b 65 20 74 68 |ill also| make th|
|00003df0| 65 20 6f 75 74 70 75 74 | 20 66 6f 72 6d 61 74 20 |e output| format |
|00003e00| 72 65 61 6c 2e 0a 0a 20 | 20 20 49 6e 20 74 68 65 |real... |  In the|
|00003e10| 20 63 6f 6e 64 65 6e 73 | 65 64 20 66 6f 72 6d 61 | condens|ed forma|
|00003e20| 74 73 2c 20 74 68 65 20 | 64 65 66 61 75 6c 74 20 |ts, the |default |
|00003e30| 69 73 20 74 6f 20 70 72 | 69 6e 74 20 61 20 62 6c |is to pr|int a bl|
|00003e40| 61 6e 6b 20 61 66 74 65 | 72 20 65 76 65 72 79 0a |ank afte|r every.|
|00003e50| 31 30 20 76 61 6c 75 65 | 73 2e 20 20 54 68 69 73 |10 value|s.  This|
|00003e60| 20 63 61 6e 20 62 65 20 | 61 6c 74 65 72 65 64 20 | can be |altered |
|00003e70| 75 73 69 6e 67 20 74 68 | 65 20 60 62 27 20 28 66 |using th|e `b' (f|
|00003e80| 6f 72 20 69 6e 73 65 72 | 74 69 6e 67 20 62 72 65 |or inser|ting bre|
|00003e90| 61 6b 73 29 0a 63 6f 6d | 6d 61 6e 64 2e 20 20 54 |aks).com|mand.  T|
|00003ea0| 68 65 20 75 73 65 20 66 | 6f 72 20 74 68 69 73 20 |he use f|or this |
|00003eb0| 63 6f 6d 6d 61 6e 64 20 | 69 73 20 74 6f 20 73 65 |command |is to se|
|00003ec0| 70 61 72 61 74 65 20 6f | 75 74 70 75 74 20 76 61 |parate o|utput va|
|00003ed0| 6c 75 65 73 20 69 6e 74 | 6f 0a 6c 6f 67 69 63 61 |lues int|o.logica|
|00003ee0| 6c 20 67 72 6f 75 70 73 | 20 74 6f 20 6d 61 6b 65 |l groups| to make|
|00003ef0| 20 74 68 65 20 6f 75 74 | 70 75 74 20 6d 6f 72 65 | the out|put more|
|00003f00| 20 72 65 61 64 61 62 6c | 65 2e 20 20 46 6f 72 20 | readabl|e.  For |
|00003f10| 69 6e 73 74 61 6e 63 65 | 2c 20 79 6f 75 20 6d 61 |instance|, you ma|
|00003f20| 79 0a 68 61 76 65 20 32 | 34 20 6f 75 74 70 75 74 |y.have 2|4 output|
|00003f30| 20 75 6e 69 74 73 20 77 | 68 65 72 65 20 69 74 20 | units w|here it |
|00003f40| 6d 61 6b 65 73 20 73 65 | 6e 73 65 20 74 6f 20 69 |makes se|nse to i|
|00003f50| 6e 73 65 72 74 20 62 6c | 61 6e 6b 73 20 61 66 74 |nsert bl|anks aft|
|00003f60| 65 72 20 74 68 65 0a 34 | 74 68 2c 20 37 74 68 20 |er the.4|th, 7th |
|00003f70| 61 6e 64 20 31 39 74 68 | 20 70 6f 73 69 74 69 6f |and 19th| positio|
|00003f80| 6e 73 2e 20 20 54 6f 20 | 64 6f 20 74 68 69 73 2c |ns.  To |do this,|
|00003f90| 20 73 70 65 63 69 66 79 | 3a 0a 0a 2e 63 65 0a 62 | specify|:...ce.b|
|00003fa0| 20 34 20 37 20 31 39 0a | 0a 54 68 65 6e 20 66 6f | 4 7 19.|.Then fo|
|00003fb0| 72 20 65 78 61 6d 70 6c | 65 2c 20 74 68 65 20 6f |r exampl|e, the o|
|00003fc0| 75 74 70 75 74 20 77 69 | 6c 6c 20 6c 6f 6f 6b 20 |utput wi|ll look |
|00003fd0| 6c 69 6b 65 3a 0a 0a 2e | 6e 61 0a 2e 6e 66 0a 20 |like:...|na..nf. |
|00003fe0| 20 31 20 31 30 5e 30 20 | 31 30 5e 20 5e 30 30 30 | 1 10^0 |10^ ^000|
|00003ff0| 76 30 30 30 30 30 76 30 | 20 30 31 30 30 30 20 28 |v00000v0| 01000 (|
|00004000| 30 2e 31 37 35 37 37 29 | 0a 20 20 32 20 31 30 31 |0.17577)|.  2 101|
|00004010| 30 20 30 31 76 20 30 5e | 30 30 30 30 76 30 30 30 |0 01v 0^|0000v000|
|00004020| 30 30 20 5e 31 30 30 30 | 20 28 30 2e 31 36 33 34 |00 ^1000| (0.1634|
|00004030| 31 29 0a 20 20 33 20 30 | 31 30 31 20 31 30 5e 20 |1).  3 0|101 10^ |
|00004040| 30 30 5e 30 30 76 30 30 | 30 30 30 76 20 30 30 30 |00^00v00|000v 000|
|00004050| 30 31 20 28 30 2e 31 36 | 38 38 37 29 0a 20 20 34 |01 (0.16|887).  4|
|00004060| 20 30 31 30 30 20 30 5e | 30 20 30 30 30 5e 30 30 | 0100 0^|0 000^00|
|00004070| 30 30 30 76 30 30 20 30 | 30 5e 30 30 20 28 30 2e |000v00 0|0^00 (0.|
|00004080| 31 39 38 38 30 29 0a 2e | 61 64 0a 2e 66 69 0a 0a |19880)..|ad..fi..|
|00004090| 54 68 65 20 60 62 27 20 | 63 6f 6d 6d 61 6e 64 20 |The `b' |command |
|000040a0| 61 6c 6c 6f 77 73 20 75 | 70 20 74 6f 20 32 30 20 |allows u|p to 20 |
|000040b0| 62 72 65 61 6b 20 70 6f | 73 69 74 69 6f 6e 73 20 |break po|sitions |
|000040c0| 74 6f 20 62 65 20 73 70 | 65 63 69 66 69 65 64 2e |to be sp|ecified.|
|000040d0| 0a 54 68 65 20 64 65 66 | 61 75 6c 74 20 6f 75 74 |.The def|ault out|
|000040e0| 70 75 74 20 66 6f 72 6d | 61 74 20 69 73 20 74 68 |put form|at is th|
|000040f0| 65 20 72 65 61 6c 20 66 | 6f 72 6d 61 74 20 77 69 |e real f|ormat wi|
|00004100| 74 68 20 31 30 20 6e 75 | 6d 62 65 72 73 20 70 65 |th 10 nu|mbers pe|
|00004110| 72 0a 6c 69 6e 65 2e 20 | 20 46 6f 72 20 74 68 65 |r.line. | For the|
|00004120| 20 6f 75 74 70 75 74 20 | 6f 66 20 72 65 61 6c 20 | output |of real |
|00004130| 76 61 6c 75 65 73 2c 20 | 74 68 65 20 60 62 27 20 |values, |the `b' |
|00004140| 63 6f 6d 6d 61 6e 64 20 | 73 70 65 63 69 66 69 65 |command |specifie|
|00004150| 73 20 77 68 65 6e 20 74 | 6f 0a 70 72 69 6e 74 20 |s when t|o.print |
|00004160| 61 20 63 61 72 72 69 61 | 67 65 20 72 65 74 75 72 |a carria|ge retur|
|00004170| 6e 2c 20 72 61 74 68 65 | 72 20 74 68 61 6e 20 77 |n, rathe|r than w|
|00004180| 68 65 6e 20 74 6f 20 70 | 72 69 6e 74 20 61 20 62 |hen to p|rint a b|
|00004190| 6c 61 6e 6b 2e 0a 0a 20 | 20 20 53 6f 6d 65 74 69 |lank... |  Someti|
|000041a0| 6d 65 73 20 74 68 65 20 | 74 72 61 69 6e 69 6e 67 |mes the |training|
|000041b0| 20 73 65 74 20 69 73 20 | 73 6f 20 6c 61 72 67 65 | set is |so large|
|000041c0| 20 74 68 61 74 20 69 74 | 20 69 73 20 61 6e 6e 6f | that it| is anno|
|000041d0| 79 69 6e 67 20 74 6f 0a | 68 61 76 65 20 61 6c 6c |ying to.|have all|
|000041e0| 20 74 68 65 20 70 61 74 | 74 65 72 6e 73 20 70 72 | the pat|terns pr|
|000041f0| 69 6e 74 20 6f 75 74 20 | 65 76 65 72 79 20 6e 20 |int out |every n |
|00004200| 69 74 65 72 61 74 69 6f | 6e 73 2e 20 20 54 6f 20 |iteratio|ns.  To |
|00004210| 67 65 74 20 61 20 73 75 | 6d 6d 61 72 79 20 6f 66 |get a su|mmary of|
|00004220| 0a 68 6f 77 20 6c 65 61 | 72 6e 69 6e 67 20 69 73 |.how lea|rning is|
|00004230| 20 67 6f 69 6e 67 2c 20 | 69 6e 73 74 65 61 64 20 | going, |instead |
|00004240| 6f 66 20 61 6c 6c 20 74 | 68 65 73 65 20 70 61 74 |of all t|hese pat|
|00004250| 74 65 72 6e 73 2c 20 75 | 73 65 20 22 66 20 73 2b |terns, u|se "f s+|
|00004260| 22 2e 0a 4e 6f 77 2c 20 | 69 66 20 74 68 65 20 63 |"..Now, |if the c|
|00004270| 6f 6d 6d 61 6e 64 20 69 | 6e 20 74 68 65 20 78 6f |ommand i|n the xo|
|00004280| 72 20 70 72 6f 62 6c 65 | 6d 20 77 61 73 20 22 72 |r proble|m was "r|
|00004290| 20 32 30 30 20 35 30 22 | 20 74 68 65 20 66 6f 6c | 200 50"| the fol|
|000042a0| 6c 6f 77 69 6e 67 0a 6f | 75 74 70 75 74 20 73 75 |lowing.o|utput su|
|000042b0| 6d 6d 61 72 79 20 77 69 | 6c 6c 20 72 65 73 75 6c |mmary wi|ll resul|
|000042c0| 74 3a 0a 0a 2e 6e 61 0a | 2e 6e 66 0a 20 20 20 20 |t:...na.|.nf.    |
|000042d0| 35 30 20 20 20 20 20 20 | 20 20 30 20 6c 65 61 72 |50      |  0 lear|
|000042e0| 6e 65 64 20 20 20 20 20 | 20 34 20 75 6e 6c 65 61 |ned     | 4 unlea|
|000042f0| 72 6e 65 64 20 20 20 20 | 20 30 2e 34 38 33 36 34 |rned    | 0.48364|
|00004300| 20 65 72 72 6f 72 2f 75 | 6e 69 74 0a 20 20 20 31 | error/u|nit.   1|
|00004310| 30 30 20 20 20 20 20 20 | 20 20 30 20 6c 65 61 72 |00      |  0 lear|
|00004320| 6e 65 64 20 20 20 20 20 | 20 34 20 75 6e 6c 65 61 |ned     | 4 unlea|
|00004330| 72 6e 65 64 20 20 20 20 | 20 30 2e 31 36 35 32 38 |rned    | 0.16528|
|00004340| 20 65 72 72 6f 72 2f 75 | 6e 69 74 0a 20 20 20 31 | error/u|nit.   1|
|00004350| 35 30 20 20 20 20 20 20 | 20 20 33 20 6c 65 61 72 |50      |  3 lear|
|00004360| 6e 65 64 20 20 20 20 20 | 20 31 20 75 6e 6c 65 61 |ned     | 1 unlea|
|00004370| 72 6e 65 64 20 20 20 20 | 20 30 2e 30 38 38 31 33 |rned    | 0.08813|
|00004380| 20 65 72 72 6f 72 2f 75 | 6e 69 74 0a 20 20 20 31 | error/u|nit.   1|
|00004390| 35 39 20 20 20 20 20 20 | 20 20 34 20 6c 65 61 72 |59      |  4 lear|
|000043a0| 6e 65 64 20 20 20 20 20 | 20 30 20 75 6e 6c 65 61 |ned     | 0 unlea|
|000043b0| 72 6e 65 64 20 20 20 20 | 20 30 2e 30 38 32 30 33 |rned    | 0.08203|
|000043c0| 20 65 72 72 6f 72 2f 75 | 6e 69 74 0a 70 61 74 74 | error/u|nit.patt|
|000043d0| 65 72 6e 73 20 6c 65 61 | 72 6e 65 64 20 74 6f 20 |erns lea|rned to |
|000043e0| 77 69 74 68 69 6e 20 30 | 2e 31 30 20 61 74 20 69 |within 0|.10 at i|
|000043f0| 74 65 72 61 74 69 6f 6e | 20 31 35 39 0a 2e 61 64 |teration| 159..ad|
|00004400| 0a 2e 66 69 0a 0a 54 68 | 65 20 70 72 6f 67 72 61 |..fi..Th|e progra|
|00004410| 6d 20 63 6f 75 6e 74 73 | 20 75 70 20 68 6f 77 20 |m counts| up how |
|00004420| 6d 61 6e 79 20 70 61 74 | 74 65 72 6e 73 20 77 65 |many pat|terns we|
|00004430| 72 65 20 6c 65 61 72 6e | 65 64 20 6f 72 20 6e 6f |re learn|ed or no|
|00004440| 74 20 6c 65 61 72 6e 65 | 64 0a 69 6e 20 65 61 63 |t learne|d.in eac|
|00004450| 68 20 74 72 61 69 6e 69 | 6e 67 20 70 61 73 73 20 |h traini|ng pass |
|00004460| 62 65 66 6f 72 65 20 74 | 68 65 20 77 65 69 67 68 |before t|he weigh|
|00004470| 74 73 20 61 72 65 20 75 | 70 64 61 74 65 64 2e 20 |ts are u|pdated. |
|00004480| 20 54 68 65 72 65 66 6f | 72 65 2c 20 74 68 65 0a | Therefo|re, the.|
|00004490| 73 74 61 74 75 73 20 69 | 73 20 6f 6e 65 20 69 74 |status i|s one it|
|000044a0| 65 72 61 74 69 6f 6e 20 | 6f 75 74 20 6f 66 20 64 |eration |out of d|
|000044b0| 61 74 65 2e 20 20 54 68 | 65 20 65 72 72 6f 72 2f |ate.  Th|e error/|
|000044c0| 75 6e 69 74 20 69 73 20 | 74 68 65 20 61 76 65 72 |unit is |the aver|
|000044d0| 61 67 65 0a 61 62 73 6f | 6c 75 74 65 20 76 61 6c |age.abso|lute val|
|000044e0| 75 65 20 6f 66 20 74 68 | 65 20 65 72 72 6f 72 20 |ue of th|e error |
|000044f0| 6f 6e 20 65 61 63 68 20 | 75 6e 69 74 20 66 6f 72 |on each |unit for|
|00004500| 20 65 61 63 68 20 70 61 | 74 74 65 72 6e 2e 20 20 | each pa|ttern.  |
|00004510| 54 6f 20 73 77 69 74 63 | 68 0a 62 61 63 6b 20 74 |To switc|h.back t|
|00004520| 6f 20 74 68 65 20 6c 6f | 6e 67 65 72 20 72 65 70 |o the lo|nger rep|
|00004530| 6f 72 74 2c 20 75 73 65 | 20 22 66 20 73 2d 22 2e |ort, use| "f s-".|
|00004540| 20 20 54 68 65 20 50 20 | 63 6f 6d 6d 61 6e 64 20 |  The P |command |
|00004550| 77 69 6c 6c 20 6c 69 73 | 74 20 61 6c 6c 20 74 68 |will lis|t all th|
|00004560| 65 0a 70 61 74 74 65 72 | 6e 73 20 6e 6f 20 6d 61 |e.patter|ns no ma|
|00004570| 74 74 65 72 20 77 68 61 | 74 20 74 68 65 20 73 65 |tter wha|t the se|
|00004580| 74 74 69 6e 67 20 6f 66 | 20 74 68 65 20 73 75 6d |tting of| the sum|
|00004590| 6d 61 72 79 20 70 61 72 | 61 6d 65 74 65 72 20 69 |mary par|ameter i|
|000045a0| 73 2e 0a 0a 2e 75 6c 0a | 53 61 76 69 6e 67 20 61 |s....ul.|Saving a|
|000045b0| 6e 64 20 52 65 73 74 6f | 72 69 6e 67 20 57 65 69 |nd Resto|ring Wei|
|000045c0| 67 68 74 73 20 61 6e 64 | 20 52 65 6c 61 74 65 64 |ghts and| Related|
|000045d0| 20 56 61 6c 75 65 73 0a | 0a 20 20 20 53 6f 6d 65 | Values.|.   Some|
|000045e0| 74 69 6d 65 73 20 74 68 | 65 20 61 6d 6f 75 6e 74 |times th|e amount|
|000045f0| 20 6f 66 20 74 69 6d 65 | 20 61 6e 64 20 65 66 66 | of time| and eff|
|00004600| 6f 72 74 20 6e 65 65 64 | 65 64 20 74 6f 20 70 72 |ort need|ed to pr|
|00004610| 6f 64 75 63 65 20 61 20 | 73 65 74 20 6f 66 0a 77 |oduce a |set of.w|
|00004620| 65 69 67 68 74 73 20 74 | 6f 20 73 6f 6c 76 65 20 |eights t|o solve |
|00004630| 61 20 70 72 6f 62 6c 65 | 6d 20 69 73 20 73 6f 20 |a proble|m is so |
|00004640| 67 72 65 61 74 20 74 68 | 61 74 20 69 74 20 69 73 |great th|at it is|
|00004650| 20 6d 6f 72 65 20 63 6f | 6e 76 65 6e 69 65 6e 74 | more co|nvenient|
|00004660| 20 74 6f 0a 73 61 76 65 | 20 74 68 65 20 77 65 69 | to.save| the wei|
|00004670| 67 68 74 73 20 72 61 74 | 68 65 72 20 74 68 61 6e |ghts rat|her than|
|00004680| 20 63 6f 6e 73 74 61 6e | 74 6c 79 20 72 65 63 61 | constan|tly reca|
|00004690| 6c 63 75 6c 61 74 65 20 | 74 68 65 6d 2e 20 20 57 |lculate |them.  W|
|000046a0| 65 69 67 68 74 73 20 63 | 61 6e 0a 62 65 20 73 61 |eights c|an.be sa|
|000046b0| 76 65 64 20 61 73 20 72 | 65 61 6c 20 76 61 6c 75 |ved as r|eal valu|
|000046c0| 65 73 20 28 74 68 65 20 | 64 65 66 61 75 6c 74 29 |es (the |default)|
|000046d0| 20 6f 72 20 61 73 20 62 | 69 6e 61 72 79 2c 20 74 | or as b|inary, t|
|000046e0| 6f 20 73 61 76 65 20 73 | 70 61 63 65 2e 20 20 54 |o save s|pace.  T|
|000046f0| 6f 0a 73 61 76 65 20 74 | 68 65 20 77 65 69 67 68 |o.save t|he weigh|
|00004700| 74 73 20 65 6e 74 65 72 | 20 74 68 65 20 63 6f 6d |ts enter| the com|
|00004710| 6d 61 6e 64 2c 20 60 53 | 27 2e 20 20 54 68 65 20 |mand, `S|'.  The |
|00004720| 77 65 69 67 68 74 73 20 | 61 72 65 20 77 72 69 74 |weights |are writ|
|00004730| 74 65 6e 20 6f 6e 20 61 | 0a 66 69 6c 65 20 63 61 |ten on a|.file ca|
|00004740| 6c 6c 65 64 20 22 77 65 | 69 67 68 74 73 22 2e 20 |lled "we|ights". |
|00004750| 20 54 68 65 20 66 6f 6c | 6c 6f 77 69 6e 67 20 66 | The fol|lowing f|
|00004760| 69 6c 65 20 63 6f 6d 65 | 73 20 66 72 6f 6d 20 74 |ile come|s from t|
|00004770| 68 65 0a 78 6f 72 20 70 | 72 6f 62 6c 65 6d 3a 0a |he.xor p|roblem:.|
|00004780| 0a 2e 6e 61 0a 2e 6e 66 | 0a 31 35 39 72 20 20 66 |..na..nf|.159r  f|
|00004790| 69 6c 65 20 3d 20 78 6f | 72 0a 20 20 20 20 37 2e |ile = xo|r.    7.|
|000047a0| 32 37 39 32 39 36 38 37 | 35 30 0a 20 20 20 2d 35 |27929687|50.   -5|
|000047b0| 2e 36 36 37 39 36 38 37 | 35 30 30 0a 20 20 20 20 |.6679687|500.    |
|000047c0| 32 2e 37 34 39 30 32 33 | 34 33 37 35 0a 20 20 20 |2.749023|4375.   |
|000047d0| 20 35 2e 38 34 38 36 33 | 32 38 31 32 35 0a 20 20 | 5.84863|28125.  |
|000047e0| 20 2d 35 2e 30 34 30 30 | 33 39 30 36 32 35 0a 20 | -5.0400|390625. |
|000047f0| 20 2d 31 31 2e 38 35 37 | 34 32 31 38 37 35 30 0a | -11.857|4218750.|
|00004800| 20 20 20 20 38 2e 33 31 | 39 33 33 35 39 33 37 35 |    8.31|93359375|
|00004810| 0a 2e 61 64 0a 2e 66 69 | 0a 0a 54 6f 20 77 72 69 |..ad..fi|..To wri|
|00004820| 74 65 20 74 68 65 20 77 | 65 69 67 68 74 73 2c 20 |te the w|eights, |
|00004830| 74 68 65 20 70 72 6f 67 | 72 61 6d 20 73 74 61 72 |the prog|ram star|
|00004840| 74 73 20 77 69 74 68 20 | 74 68 65 20 73 65 63 6f |ts with |the seco|
|00004850| 6e 64 20 6c 61 79 65 72 | 2c 20 77 72 69 74 65 73 |nd layer|, writes|
|00004860| 0a 6f 75 74 20 74 68 65 | 20 77 65 69 67 68 74 73 |.out the| weights|
|00004870| 20 6c 65 61 64 69 6e 67 | 20 69 6e 74 6f 20 74 68 | leading| into th|
|00004880| 65 73 65 20 75 6e 69 74 | 73 20 69 6e 20 6f 72 64 |ese unit|s in ord|
|00004890| 65 72 20 77 69 74 68 20 | 74 68 65 20 74 68 72 65 |er with |the thre|
|000048a0| 73 68 6f 6c 64 0a 77 65 | 69 67 68 74 20 6c 61 73 |shold.we|ight las|
|000048b0| 74 2e 20 20 54 68 65 6e | 20 69 74 20 6d 6f 76 65 |t.  Then| it move|
|000048c0| 73 20 6f 6e 20 74 6f 0a | 74 68 65 20 74 68 69 72 |s on to.|the thir|
|000048d0| 64 20 6c 61 79 65 72 2c | 20 61 6e 64 20 73 6f 20 |d layer,| and so |
|000048e0| 6f 6e 2e 20 20 54 6f 20 | 72 65 73 74 6f 72 65 20 |on.  To |restore |
|000048f0| 74 68 65 73 65 20 77 65 | 69 67 68 74 73 2c 20 74 |these we|ights, t|
|00004900| 79 70 65 20 61 6e 20 60 | 52 27 20 66 6f 72 0a 72 |ype an `|R' for.r|
|00004910| 65 73 74 6f 72 65 2e 20 | 20 41 74 20 74 68 69 73 |estore. | At this|
|00004920| 20 74 69 6d 65 2c 20 74 | 68 65 20 70 72 6f 67 72 | time, t|he progr|
|00004930| 61 6d 20 72 65 61 64 73 | 20 74 68 65 20 68 65 61 |am reads| the hea|
|00004940| 64 65 72 20 6c 69 6e 65 | 20 61 6e 64 20 73 65 74 |der line| and set|
|00004950| 73 20 74 68 65 0a 74 6f | 74 61 6c 20 6e 75 6d 62 |s the.to|tal numb|
|00004960| 65 72 20 6f 66 20 69 74 | 65 72 61 74 69 6f 6e 73 |er of it|erations|
|00004970| 20 74 68 65 20 70 72 6f | 67 72 61 6d 20 68 61 73 | the pro|gram has|
|00004980| 20 67 6f 6e 65 20 74 68 | 72 6f 75 67 68 20 74 6f | gone th|rough to|
|00004990| 20 62 65 20 74 68 65 20 | 66 69 72 73 74 0a 6e 75 | be the |first.nu|
|000049a0| 6d 62 65 72 20 69 74 20 | 66 69 6e 64 73 20 6f 6e |mber it |finds on|
|000049b0| 20 74 68 65 20 68 65 61 | 64 65 72 20 6c 69 6e 65 | the hea|der line|
|000049c0| 2e 20 20 49 74 20 74 68 | 65 6e 20 72 65 61 64 73 |.  It th|en reads|
|000049d0| 20 74 68 65 20 63 68 61 | 72 61 63 74 65 72 0a 69 | the cha|racter.i|
|000049e0| 6d 6d 65 64 69 61 74 65 | 6c 79 20 61 66 74 65 72 |mmediate|ly after|
|000049f0| 20 74 68 65 20 6e 75 6d | 62 65 72 2e 20 20 54 68 | the num|ber.  Th|
|00004a00| 65 20 60 72 27 20 69 6e | 64 69 63 61 74 65 73 20 |e `r' in|dicates |
|00004a10| 74 68 61 74 20 74 68 65 | 20 77 65 69 67 68 74 73 |that the| weights|
|00004a20| 20 77 69 6c 6c 0a 62 65 | 20 72 65 61 6c 20 6e 75 | will.be| real nu|
|00004a30| 6d 62 65 72 73 20 72 65 | 70 72 65 73 65 6e 74 65 |mbers re|presente|
|00004a40| 64 20 61 73 20 63 68 61 | 72 61 63 74 65 72 20 73 |d as cha|racter s|
|00004a50| 74 72 69 6e 67 73 2e 20 | 20 49 66 20 74 68 65 20 |trings. | If the |
|00004a60| 77 65 69 67 68 74 73 20 | 77 65 72 65 0a 62 69 6e |weights |were.bin|
|00004a70| 61 72 79 2c 20 74 68 65 | 20 63 68 61 72 61 63 74 |ary, the| charact|
|00004a80| 65 72 20 77 6f 75 6c 64 | 20 62 65 20 61 20 60 62 |er would| be a `b|
|00004a90| 27 20 72 61 74 68 65 72 | 20 74 68 61 6e 20 61 6e |' rather| than an|
|00004aa0| 20 60 72 27 2e 20 20 41 | 6c 73 6f 2c 20 69 66 20 | `r'.  A|lso, if |
|00004ab0| 74 68 65 0a 63 68 61 72 | 61 63 74 65 72 20 69 73 |the.char|acter is|
|00004ac0| 20 60 62 27 2c 20 74 68 | 65 20 6e 65 78 74 20 63 | `b', th|e next c|
|00004ad0| 68 61 72 61 63 74 65 72 | 20 69 73 20 72 65 61 64 |haracter| is read|
|00004ae0| 2e 20 20 54 68 69 73 20 | 6e 65 78 74 20 63 68 61 |.  This |next cha|
|00004af0| 72 61 63 74 65 72 0a 69 | 6e 64 69 63 61 74 65 73 |racter.i|ndicates|
|00004b00| 20 68 6f 77 20 6d 61 6e | 79 20 62 79 74 65 73 20 | how man|y bytes |
|00004b10| 61 72 65 20 75 73 65 64 | 20 70 65 72 20 76 61 6c |are used| per val|
|00004b20| 75 65 2e 20 20 54 68 65 | 20 69 6e 74 65 67 65 72 |ue.  The| integer|
|00004b30| 20 76 65 72 73 69 6f 6e | 73 2c 20 62 70 0a 61 6e | version|s, bp.an|
|00004b40| 64 20 73 62 70 20 77 72 | 69 74 65 20 66 69 6c 65 |d sbp wr|ite file|
|00004b50| 73 20 77 69 74 68 20 32 | 20 62 79 74 65 73 20 70 |s with 2| bytes p|
|00004b60| 65 72 20 77 65 69 67 68 | 74 2c 20 77 68 69 6c 65 |er weigh|t, while|
|00004b70| 20 74 68 65 20 72 65 61 | 6c 20 76 65 72 73 69 6f | the rea|l versio|
|00004b80| 6e 73 2c 0a 72 62 70 20 | 61 6e 64 20 73 72 62 70 |ns,.rbp |and srbp|
|00004b90| 20 77 72 69 74 65 20 66 | 69 6c 65 73 20 77 69 74 | write f|iles wit|
|00004ba0| 68 20 38 20 62 79 74 65 | 73 20 70 65 72 20 77 65 |h 8 byte|s per we|
|00004bb0| 69 67 68 74 2e 20 20 57 | 69 74 68 20 74 68 69 73 |ight.  W|ith this|
|00004bc0| 20 6e 6f 74 61 74 69 6f | 6e 2c 0a 77 65 69 67 68 | notatio|n,.weigh|
|00004bd0| 74 20 66 69 6c 65 73 20 | 77 72 69 74 74 65 6e 20 |t files |written |
|00004be0| 62 79 20 6f 6e 65 20 70 | 72 6f 67 72 61 6d 20 63 |by one p|rogram c|
|00004bf0| 61 6e 20 62 65 20 72 65 | 61 64 20 62 79 20 74 68 |an be re|ad by th|
|00004c00| 65 20 6f 74 68 65 72 2e | 20 20 41 20 62 69 6e 61 |e other.|  A bina|
|00004c10| 72 79 0a 77 65 69 67 68 | 74 20 66 6f 72 6d 61 74 |ry.weigh|t format|
|00004c20| 20 69 73 20 73 70 65 63 | 69 66 69 65 64 20 77 69 | is spec|ified wi|
|00004c30| 74 68 69 6e 20 74 68 65 | 20 60 66 27 20 63 6f 6d |thin the| `f' com|
|00004c40| 6d 61 6e 64 20 62 79 20 | 75 73 69 6e 67 20 22 66 |mand by |using "f|
|00004c50| 20 77 62 22 2e 20 20 41 | 0a 72 65 61 6c 20 66 6f | wb".  A|.real fo|
|00004c60| 72 6d 61 74 20 69 73 20 | 73 70 65 63 69 66 69 65 |rmat is |specifie|
|00004c70| 64 20 62 79 20 75 73 69 | 6e 67 20 22 66 20 77 72 |d by usi|ng "f wr|
|00004c80| 22 2e 20 20 49 66 20 79 | 6f 75 72 20 70 72 6f 67 |".  If y|our prog|
|00004c90| 72 61 6d 20 73 70 65 63 | 69 66 69 65 73 0a 74 68 |ram spec|ifies.th|
|00004ca0| 61 74 20 77 65 69 67 68 | 74 73 20 73 68 6f 75 6c |at weigh|ts shoul|
|00004cb0| 64 20 62 65 20 77 72 69 | 74 74 65 6e 20 69 6e 20 |d be wri|tten in |
|00004cc0| 6f 6e 65 20 66 6f 72 6d | 61 74 2c 20 62 75 74 20 |one form|at, but |
|00004cd0| 74 68 65 20 77 65 69 67 | 68 74 20 66 69 6c 65 20 |the weig|ht file |
|00004ce0| 79 6f 75 0a 72 65 61 64 | 20 66 72 6f 6d 20 69 73 |you.read| from is|
|00004cf0| 20 64 69 66 66 65 72 65 | 6e 74 2c 20 61 20 77 61 | differe|nt, a wa|
|00004d00| 72 6e 69 6e 67 20 6d 65 | 73 73 61 67 65 20 77 69 |rning me|ssage wi|
|00004d10| 6c 6c 20 62 65 20 70 72 | 69 6e 74 65 64 2e 20 20 |ll be pr|inted.  |
|00004d20| 54 68 65 72 65 20 69 73 | 20 6e 6f 0a 63 68 65 63 |There is| no.chec|
|00004d30| 6b 20 6d 61 64 65 20 74 | 6f 20 73 65 65 20 69 66 |k made t|o see if|
|00004d40| 20 74 68 65 20 6e 75 6d | 62 65 72 20 6f 66 20 77 | the num|ber of w|
|00004d50| 65 69 67 68 74 73 20 6f | 6e 20 74 68 65 20 66 69 |eights o|n the fi|
|00004d60| 6c 65 20 65 71 75 61 6c | 73 20 74 68 65 20 6e 75 |le equal|s the nu|
|00004d70| 6d 62 65 72 0a 6f 66 20 | 77 65 69 67 68 74 73 20 |mber.of |weights |
|00004d80| 69 6e 20 74 68 65 20 6e | 65 74 77 6f 72 6b 2e 0a |in the n|etwork..|
|00004d90| 0a 20 20 20 54 68 65 20 | 61 62 6f 76 65 20 66 6f |.   The |above fo|
|00004da0| 72 6d 61 74 73 20 73 70 | 65 63 69 66 79 20 74 68 |rmats sp|ecify th|
|00004db0| 61 74 20 6f 6e 6c 79 20 | 77 65 69 67 68 74 73 20 |at only |weights |
|00004dc0| 61 72 65 20 77 72 69 74 | 74 65 6e 20 6f 75 74 20 |are writ|ten out |
|00004dd0| 61 6e 64 0a 74 68 69 73 | 20 69 73 20 61 6c 6c 20 |and.this| is all |
|00004de0| 79 6f 75 20 6e 65 65 64 | 20 6f 6e 63 65 20 74 68 |you need| once th|
|00004df0| 65 20 70 61 74 74 65 72 | 6e 73 20 68 61 76 65 20 |e patter|ns have |
|00004e00| 63 6f 6e 76 65 72 67 65 | 64 2e 20 20 48 6f 77 65 |converge|d.  Howe|
|00004e10| 76 65 72 2c 20 69 66 0a | 79 6f 75 27 72 65 20 73 |ver, if.|you're s|
|00004e20| 74 69 6c 6c 20 74 72 61 | 69 6e 69 6e 67 20 74 68 |till tra|ining th|
|00004e30| 65 20 6e 65 74 77 6f 72 | 6b 20 61 6e 64 20 77 61 |e networ|k and wa|
|00004e40| 6e 74 20 74 6f 20 62 72 | 65 61 6b 20 6f 66 66 20 |nt to br|eak off |
|00004e50| 74 72 61 69 6e 69 6e 67 | 20 61 6e 64 0a 70 69 63 |training| and.pic|
|00004e60| 6b 20 75 70 20 74 68 65 | 20 74 72 61 69 6e 69 6e |k up the| trainin|
|00004e70| 67 20 66 72 6f 6d 20 65 | 78 61 63 74 6c 79 20 74 |g from e|xactly t|
|00004e80| 68 65 20 73 61 6d 65 20 | 70 6f 69 6e 74 20 6c 61 |he same |point la|
|00004e90| 74 65 72 2c 20 79 6f 75 | 20 6e 65 65 64 20 74 6f |ter, you| need to|
|00004ea0| 20 73 61 76 65 0a 74 68 | 65 20 6f 6c 64 20 77 65 | save.th|e old we|
|00004eb0| 69 67 68 74 20 63 68 61 | 6e 67 65 73 20 77 68 65 |ight cha|nges whe|
|00004ec0| 6e 20 75 73 69 6e 67 20 | 6d 6f 6d 65 6e 74 75 6d |n using |momentum|
|00004ed0| 2c 20 61 6e 64 20 74 68 | 65 20 70 61 72 61 6d 65 |, and th|e parame|
|00004ee0| 74 65 72 73 20 66 6f 72 | 20 74 68 65 0a 64 65 6c |ters for| the.del|
|00004ef0| 74 61 2d 62 61 72 2d 64 | 65 6c 74 61 20 6d 65 74 |ta-bar-d|elta met|
|00004f00| 68 6f 64 20 69 66 20 79 | 6f 75 20 61 72 65 20 75 |hod if y|ou are u|
|00004f10| 73 69 6e 67 20 74 68 69 | 73 20 74 65 63 68 6e 69 |sing thi|s techni|
|00004f20| 71 75 65 2e 20 20 54 6f | 20 73 61 76 65 20 74 68 |que.  To| save th|
|00004f30| 65 73 65 0a 65 78 74 72 | 61 20 70 61 72 61 6d 65 |ese.extr|a parame|
|00004f40| 74 65 72 73 20 6f 6e 20 | 74 68 65 20 77 65 69 67 |ters on |the weig|
|00004f50| 68 74 73 20 66 69 6c 65 | 2c 20 75 73 65 20 22 66 |hts file|, use "f|
|00004f60| 20 77 52 22 20 74 6f 20 | 77 72 69 74 65 20 74 68 | wR" to |write th|
|00004f70| 65 20 65 78 74 72 61 0a | 76 61 6c 75 65 73 20 61 |e extra.|values a|
|00004f80| 73 20 72 65 61 6c 20 61 | 6e 64 20 22 66 20 77 42 |s real a|nd "f wB|
|00004f90| 22 20 74 6f 20 77 72 69 | 74 65 20 74 68 65 20 65 |" to wri|te the e|
|00004fa0| 78 74 72 61 20 76 61 6c | 75 65 73 20 61 73 20 62 |xtra val|ues as b|
|00004fb0| 69 6e 61 72 79 2e 0a 0a | 20 20 20 49 6e 20 74 68 |inary...|   In th|
|00004fc0| 65 20 61 62 6f 76 65 20 | 65 78 61 6d 70 6c 65 2c |e above |example,|
|00004fd0| 20 74 68 65 20 63 6f 6d | 6d 61 6e 64 20 53 2c 20 | the com|mand S, |
|00004fe0| 77 61 73 20 75 73 65 64 | 20 74 6f 20 73 61 76 65 |was used| to save|
|00004ff0| 20 74 68 65 20 77 65 69 | 67 68 74 73 0a 69 6d 6d | the wei|ghts.imm|
|00005000| 65 64 69 61 74 65 6c 79 | 2e 20 20 41 6e 6f 74 68 |ediately|.  Anoth|
|00005010| 65 72 20 61 6c 74 65 72 | 6e 61 74 69 76 65 20 69 |er alter|native i|
|00005020| 73 20 74 6f 20 73 61 76 | 65 20 77 65 69 67 68 74 |s to sav|e weight|
|00005030| 73 20 61 74 20 72 65 67 | 75 6c 61 72 0a 69 6e 74 |s at reg|ular.int|
|00005040| 65 72 76 61 6c 73 2e 20 | 20 54 68 65 20 63 6f 6d |ervals. | The com|
|00005050| 6d 61 6e 64 2c 20 53 20 | 31 30 30 2c 20 77 69 6c |mand, S |100, wil|
|00005060| 6c 20 61 75 74 6f 6d 61 | 74 69 63 61 6c 6c 79 20 |l automa|tically |
|00005070| 73 61 76 65 20 77 65 69 | 67 68 74 73 20 65 76 65 |save wei|ghts eve|
|00005080| 72 79 0a 31 30 30 20 69 | 74 65 72 61 74 69 6f 6e |ry.100 i|teration|
|00005090| 73 20 74 68 65 20 70 72 | 6f 67 72 61 6d 20 64 6f |s the pr|ogram do|
|000050a0| 65 73 2c 20 74 68 61 74 | 20 69 73 2c 20 77 68 65 |es, that| is, whe|
|000050b0| 6e 20 74 68 65 20 74 6f | 74 61 6c 20 69 74 65 72 |n the to|tal iter|
|000050c0| 61 74 69 6f 6e 73 20 6d | 6f 64 0a 31 30 30 20 3d |ations m|od.100 =|
|000050d0| 20 30 2e 20 20 54 68 65 | 20 69 6e 69 74 69 61 6c | 0.  The| initial|
|000050e0| 20 72 61 74 65 20 61 74 | 20 77 68 69 63 68 20 74 | rate at| which t|
|000050f0| 6f 20 73 61 76 65 20 77 | 65 69 67 68 74 73 20 69 |o save w|eights i|
|00005100| 73 20 73 65 74 20 61 74 | 20 31 30 30 2c 30 30 30 |s set at| 100,000|
|00005110| 2c 0a 77 68 69 63 68 20 | 67 65 6e 65 72 61 6c 6c |,.which |generall|
|00005120| 79 20 6d 65 61 6e 73 20 | 74 68 61 74 20 6e 6f 20 |y means |that no |
|00005130| 77 65 69 67 68 74 73 20 | 77 69 6c 6c 20 65 76 65 |weights |will eve|
|00005140| 72 20 62 65 20 73 61 76 | 65 64 2e 0a 0a 20 20 20 |r be sav|ed...   |
|00005150| 41 6e 6f 74 68 65 72 20 | 75 73 65 20 66 6f 72 20 |Another |use for |
|00005160| 73 61 76 69 6e 67 20 77 | 65 69 67 68 74 73 20 68 |saving w|eights h|
|00005170| 61 73 20 74 6f 20 64 6f | 20 77 69 74 68 20 74 72 |as to do| with tr|
|00005180| 79 69 6e 67 20 74 6f 20 | 66 69 6e 64 20 74 68 65 |ying to |find the|
|00005190| 0a 70 72 6f 70 65 72 20 | 70 61 72 61 6d 65 74 65 |.proper |paramete|
|000051a0| 72 73 20 74 6f 20 71 75 | 69 63 6b 6c 79 20 73 6f |rs to qu|ickly so|
|000051b0| 6c 76 65 20 74 68 65 20 | 70 72 6f 62 6c 65 6d 2e |lve the |problem.|
|000051c0| 20 20 4f 72 64 69 6e 61 | 72 69 6c 79 2c 20 61 20 |  Ordina|rily, a |
|000051d0| 68 69 67 68 0a 72 61 74 | 65 20 6f 66 20 6c 65 61 |high.rat|e of lea|
|000051e0| 72 6e 69 6e 67 20 69 73 | 20 64 65 73 69 72 61 62 |rning is| desirab|
|000051f0| 6c 65 2c 20 62 75 74 20 | 6f 66 74 65 6e 20 74 6f |le, but |often to|
|00005200| 6f 20 68 69 67 68 20 61 | 20 72 61 74 65 20 6f 66 |o high a| rate of|
|00005210| 20 6c 65 61 72 6e 69 6e | 67 0a 77 69 6c 6c 20 69 | learnin|g.will i|
|00005220| 6e 63 72 65 61 73 65 20 | 74 68 65 20 65 72 72 6f |ncrease |the erro|
|00005230| 72 2c 20 72 61 74 68 65 | 72 20 74 68 61 6e 20 64 |r, rathe|r than d|
|00005240| 65 63 72 65 61 73 65 20 | 69 74 2e 20 20 49 6e 20 |ecrease |it.  In |
|00005250| 74 72 79 69 6e 67 20 74 | 6f 20 66 69 6e 64 0a 74 |trying t|o find.t|
|00005260| 68 65 20 61 6e 73 77 65 | 72 20 61 73 20 71 75 69 |he answe|r as qui|
|00005270| 63 6b 6c 79 20 61 73 20 | 70 6f 73 73 69 62 6c 65 |ckly as |possible|
|00005280| 2c 20 69 66 20 74 68 65 | 20 6e 65 74 77 6f 72 6b |, if the| network|
|00005290| 20 73 65 65 6d 73 20 74 | 6f 20 62 65 0a 63 6f 6e | seems t|o be.con|
|000052a0| 76 65 72 67 69 6e 67 20 | 77 69 74 68 20 74 68 65 |verging |with the|
|000052b0| 20 63 75 72 72 65 6e 74 | 20 70 61 72 61 6d 65 74 | current| paramet|
|000052c0| 65 72 73 20 79 6f 75 20 | 63 61 6e 20 73 61 76 65 |ers you |can save|
|000052d0| 20 74 68 65 20 63 75 72 | 72 65 6e 74 20 77 65 69 | the cur|rent wei|
|000052e0| 67 68 74 73 0a 61 6e 64 | 20 69 6e 63 72 65 61 73 |ghts.and| increas|
|000052f0| 65 20 74 68 65 20 6c 65 | 61 72 6e 69 6e 67 20 72 |e the le|arning r|
|00005300| 61 74 65 2e 20 20 49 66 | 20 74 68 69 73 20 69 6e |ate.  If| this in|
|00005310| 63 72 65 61 73 65 64 20 | 6c 65 61 72 6e 69 6e 67 |creased |learning|
|00005320| 20 72 61 74 65 20 72 75 | 69 6e 73 0a 74 68 65 20 | rate ru|ins.the |
|00005330| 63 6f 6e 76 65 72 67 65 | 6e 63 65 2c 20 74 68 65 |converge|nce, the|
|00005340| 6e 20 79 6f 75 20 63 61 | 6e 20 72 65 73 74 6f 72 |n you ca|n restor|
|00005350| 65 20 74 68 65 20 77 65 | 69 67 68 74 73 20 79 6f |e the we|ights yo|
|00005360| 75 20 68 61 64 20 62 65 | 66 6f 72 65 20 79 6f 75 |u had be|fore you|
|00005370| 0a 6d 61 64 65 20 74 68 | 69 73 20 69 6e 63 72 65 |.made th|is incre|
|00005380| 61 73 65 2e 0a 0a 0a 2e | 75 6c 0a 49 6e 69 74 69 |ase.....|ul.Initi|
|00005390| 61 6c 69 7a 69 6e 67 20 | 57 65 69 67 68 74 73 20 |alizing |Weights |
|000053a0| 61 6e 64 20 47 69 76 69 | 6e 67 20 74 68 65 20 4e |and Givi|ng the N|
|000053b0| 65 74 77 6f 72 6b 20 61 | 20 60 4b 69 63 6b 27 0a |etwork a| `Kick'.|
|000053c0| 0a 20 20 20 41 6c 6c 20 | 74 68 65 20 77 65 69 67 |.   All |the weig|
|000053d0| 68 74 73 20 69 6e 20 74 | 68 65 20 6e 65 74 77 6f |hts in t|he netwo|
|000053e0| 72 6b 20 69 6e 69 74 69 | 61 6c 6c 79 20 73 74 61 |rk initi|ally sta|
|000053f0| 72 74 20 6f 75 74 20 61 | 74 20 30 2e 20 20 49 6e |rt out a|t 0.  In|
|00005400| 0a 73 79 6d 6d 65 74 72 | 69 63 20 6e 65 74 77 6f |.symmetr|ic netwo|
|00005410| 72 6b 73 20 74 68 65 6e | 2c 20 6e 6f 20 6c 65 61 |rks then|, no lea|
|00005420| 72 6e 69 6e 67 20 6d 61 | 79 20 72 65 73 75 6c 74 |rning ma|y result|
|00005430| 20 62 65 63 61 75 73 65 | 20 65 72 72 6f 72 20 73 | because| error s|
|00005440| 69 67 6e 61 6c 73 0a 63 | 61 6e 63 65 6c 20 74 68 |ignals.c|ancel th|
|00005450| 65 6d 73 65 6c 76 65 73 | 20 6f 75 74 2e 20 20 45 |emselves| out.  E|
|00005460| 76 65 6e 20 69 6e 20 6e | 6f 6e 2d 73 79 6d 6d 65 |ven in n|on-symme|
|00005470| 74 72 69 63 0a 6e 65 74 | 77 6f 72 6b 73 2c 20 74 |tric.net|works, t|
|00005480| 68 65 20 74 72 61 69 6e | 69 6e 67 20 70 72 6f 63 |he train|ing proc|
|00005490| 65 73 73 20 77 69 6c 6c | 20 6f 66 74 65 6e 20 63 |ess will| often c|
|000054a0| 6f 6e 76 65 72 67 65 20 | 66 61 73 74 65 72 20 69 |onverge |faster i|
|000054b0| 66 20 74 68 65 20 77 65 | 69 67 68 74 73 0a 73 74 |f the we|ights.st|
|000054c0| 61 72 74 20 6f 75 74 20 | 61 74 20 73 6d 61 6c 6c |art out |at small|
|000054d0| 20 72 61 6e 64 6f 6d 20 | 76 61 6c 75 65 73 2e 20 | random |values. |
|000054e0| 20 54 6f 20 64 6f 20 74 | 68 69 73 2c 20 74 68 65 | To do t|his, the|
|000054f0| 20 60 6b 27 20 63 6f 6d | 6d 61 6e 64 20 77 69 6c | `k' com|mand wil|
|00005500| 6c 0a 74 61 6b 65 20 74 | 68 65 20 6e 65 74 77 6f |l.take t|he netwo|
|00005510| 72 6b 20 61 6e 64 20 61 | 6c 74 65 72 20 74 68 65 |rk and a|lter the|
|00005520| 20 77 65 69 67 68 74 73 | 20 69 6e 20 74 68 65 20 | weights| in the |
|00005530| 66 6f 6c 6c 6f 77 69 6e | 67 20 77 61 79 73 2e 20 |followin|g ways. |
|00005540| 20 53 75 70 70 6f 73 65 | 0a 74 68 65 20 63 6f 6d | Suppose|.the com|
|00005550| 6d 61 6e 64 20 67 69 76 | 65 6e 20 69 73 3a 0a 0a |mand giv|en is:..|
|00005560| 2e 63 65 0a 6b 20 30 20 | 30 2e 35 0a 0a 4e 6f 77 |.ce.k 0 |0.5..Now|
|00005570| 2c 20 69 66 20 61 20 77 | 65 69 67 68 74 20 69 73 |, if a w|eight is|
|00005580| 20 65 78 61 63 74 6c 79 | 20 30 2c 20 74 68 65 6e | exactly| 0, then|
|00005590| 20 74 68 65 20 77 65 69 | 67 68 74 20 77 69 6c 6c | the wei|ght will|
|000055a0| 20 62 65 20 63 68 61 6e | 67 65 64 20 74 6f 20 61 | be chan|ged to a|
|000055b0| 0a 72 61 6e 64 6f 6d 20 | 76 61 6c 75 65 20 62 65 |.random |value be|
|000055c0| 74 77 65 65 6e 20 2b 30 | 2e 35 20 61 6e 64 20 2d |tween +0|.5 and -|
|000055d0| 30 2e 35 2e 20 20 54 68 | 65 20 61 62 6f 76 65 20 |0.5.  Th|e above |
|000055e0| 63 6f 6d 6d 61 6e 64 20 | 63 61 6e 20 74 68 65 72 |command |can ther|
|000055f0| 65 66 6f 72 65 20 62 65 | 0a 75 73 65 64 20 74 6f |efore be|.used to|
|00005600| 20 69 6e 69 74 69 61 6c | 69 7a 65 20 74 68 65 20 | initial|ize the |
|00005610| 77 65 69 67 68 74 73 20 | 69 6e 20 74 68 65 20 6e |weights |in the n|
|00005620| 65 74 77 6f 72 6b 2e 20 | 20 41 20 6d 6f 72 65 20 |etwork. | A more |
|00005630| 63 6f 6d 70 6c 65 78 20 | 75 73 65 20 6f 66 0a 74 |complex |use of.t|
|00005640| 68 65 20 60 6b 27 20 63 | 6f 6d 6d 61 6e 64 20 69 |he `k' c|ommand i|
|00005650| 73 20 74 6f 20 64 65 63 | 72 65 61 73 65 20 74 68 |s to dec|rease th|
|00005660| 65 20 6d 61 67 6e 69 74 | 75 64 65 20 6f 66 20 6c |e magnit|ude of l|
|00005670| 61 72 67 65 20 77 65 69 | 67 68 74 73 20 69 6e 20 |arge wei|ghts in |
|00005680| 74 68 65 0a 6e 65 74 77 | 6f 72 6b 20 62 79 20 61 |the.netw|ork by a|
|00005690| 20 63 65 72 74 61 69 6e | 20 72 61 6e 64 6f 6d 20 | certain| random |
|000056a0| 61 6d 6f 75 6e 74 2e 20 | 20 46 6f 72 20 69 6e 73 |amount. | For ins|
|000056b0| 74 61 6e 63 65 2c 20 69 | 6e 20 74 68 65 20 66 6f |tance, i|n the fo|
|000056c0| 6c 6c 6f 77 69 6e 67 0a | 63 6f 6d 6d 61 6e 64 3a |llowing.|command:|
|000056d0| 0a 0a 2e 63 65 0a 6b 20 | 32 20 38 0a 0a 61 6c 6c |...ce.k |2 8..all|
|000056e0| 20 74 68 65 20 77 65 69 | 67 68 74 73 20 69 6e 20 | the wei|ghts in |
|000056f0| 74 68 65 20 6e 65 74 77 | 6f 72 6b 20 74 68 61 74 |the netw|ork that|
|00005700| 20 61 72 65 20 67 72 65 | 61 74 65 72 20 74 68 61 | are gre|ater tha|
|00005710| 6e 20 6f 72 20 65 71 75 | 61 6c 20 74 6f 20 32 2c |n or equ|al to 2,|
|00005720| 20 77 69 6c 6c 0a 62 65 | 20 64 65 63 72 65 61 73 | will.be| decreas|
|00005730| 65 64 20 62 79 20 61 20 | 72 61 6e 64 6f 6d 20 6e |ed by a |random n|
|00005740| 75 6d 62 65 72 20 62 65 | 74 77 65 65 6e 20 30 20 |umber be|tween 0 |
|00005750| 61 6e 64 20 38 2e 20 20 | 57 65 69 67 68 74 73 0a |and 8.  |Weights.|
|00005760| 6c 65 73 73 20 74 68 61 | 6e 20 6f 72 20 65 71 75 |less tha|n or equ|
|00005770| 61 6c 20 74 6f 20 2d 32 | 20 77 69 6c 6c 20 62 65 |al to -2| will be|
|00005780| 20 69 6e 63 72 65 61 73 | 65 64 20 62 79 20 61 20 | increas|ed by a |
|00005790| 72 61 6e 64 6f 6d 20 6e | 75 6d 62 65 72 0a 62 65 |random n|umber.be|
|000057a0| 74 77 65 65 6e 20 30 20 | 61 6e 64 20 38 2e 20 20 |tween 0 |and 8.  |
|000057b0| 54 68 65 20 73 65 65 64 | 20 74 6f 20 74 68 65 20 |The seed| to the |
|000057c0| 72 61 6e 64 6f 6d 20 6e | 75 6d 62 65 72 20 67 65 |random n|umber ge|
|000057d0| 6e 65 72 61 74 6f 72 20 | 63 61 6e 20 62 65 0a 63 |nerator |can be.c|
|000057e0| 68 61 6e 67 65 64 20 75 | 73 69 6e 67 20 74 68 65 |hanged u|sing the|
|000057f0| 20 60 73 27 20 63 6f 6d | 6d 61 6e 64 20 61 73 20 | `s' com|mand as |
|00005800| 69 6e 20 22 73 20 37 22 | 2e 20 20 54 68 65 20 69 |in "s 7"|.  The i|
|00005810| 6e 74 65 67 65 72 20 70 | 61 72 61 6d 65 74 65 72 |nteger p|arameter|
|00005820| 20 69 6e 20 74 68 65 0a | 60 73 27 20 63 6f 6d 6d | in the.|`s' comm|
|00005830| 61 6e 64 20 69 73 20 6f | 66 20 74 79 70 65 2c 20 |and is o|f type, |
|00005840| 75 6e 73 69 67 6e 65 64 | 2e 0a 0a 20 20 20 41 6e |unsigned|...   An|
|00005850| 6f 74 68 65 72 20 6d 65 | 74 68 6f 64 20 6f 66 20 |other me|thod of |
|00005860| 67 69 76 69 6e 67 20 61 | 20 6e 65 74 77 6f 72 6b |giving a| network|
|00005870| 20 61 20 6b 69 63 6b 20 | 69 73 20 74 6f 20 61 64 | a kick |is to ad|
|00005880| 64 20 68 69 64 64 65 6e | 20 6c 61 79 65 72 0a 75 |d hidden| layer.u|
|00005890| 6e 69 74 73 2e 20 20 54 | 68 65 20 63 6f 6d 6d 61 |nits.  T|he comma|
|000058a0| 6e 64 3a 0a 0a 2e 63 65 | 0a 48 20 32 20 30 2e 35 |nd:...ce|.H 2 0.5|
|000058b0| 0a 0a 61 64 64 73 20 6f | 6e 65 20 75 6e 69 74 20 |..adds o|ne unit |
|000058c0| 74 6f 20 6c 61 79 65 72 | 20 32 20 6f 66 20 74 68 |to layer| 2 of th|
|000058d0| 65 20 6e 65 74 77 6f 72 | 6b 20 61 6e 64 20 61 6c |e networ|k and al|
|000058e0| 6c 20 74 68 65 20 77 65 | 69 67 68 74 73 20 74 68 |l the we|ights th|
|000058f0| 61 74 20 61 72 65 0a 63 | 72 65 61 74 65 64 20 61 |at are.c|reated a|
|00005900| 72 65 20 69 6e 69 74 69 | 61 6c 69 7a 65 64 20 74 |re initi|alized t|
|00005910| 6f 20 62 65 74 77 65 65 | 6e 20 2d 20 30 2e 35 20 |o betwee|n - 0.5 |
|00005920| 61 6e 64 20 2b 20 30 2e | 35 2e 0a 0a 20 20 20 54 |and + 0.|5...   T|
|00005930| 68 65 20 73 75 62 6a 65 | 63 74 20 6f 66 20 6b 69 |he subje|ct of ki|
|00005940| 63 6b 69 6e 67 20 61 20 | 62 61 63 6b 2d 70 72 6f |cking a |back-pro|
|00005950| 70 61 67 61 74 69 6f 6e | 20 6e 65 74 77 6f 72 6b |pagation| network|
|00005960| 20 6f 75 74 20 6f 66 20 | 6c 6f 63 61 6c 20 6d 69 | out of |local mi|
|00005970| 6e 69 6d 61 0a 68 61 73 | 20 62 61 72 65 6c 79 20 |nima.has| barely |
|00005980| 62 65 65 6e 20 73 74 75 | 64 69 65 64 20 61 6e 64 |been stu|died and|
|00005990| 20 74 68 65 72 65 20 69 | 73 20 6e 6f 20 67 75 61 | there i|s no gua|
|000059a0| 72 61 6e 74 65 65 20 74 | 68 61 74 20 74 68 65 20 |rantee t|hat the |
|000059b0| 61 62 6f 76 65 20 6d 65 | 74 68 6f 64 73 0a 61 72 |above me|thods.ar|
|000059c0| 65 20 76 65 72 79 20 75 | 73 65 66 75 6c 20 69 6e |e very u|seful in|
|000059d0| 20 67 65 6e 65 72 61 6c | 2e 0a 0a 2e 75 6c 0a 53 | general|....ul.S|
|000059e0| 65 74 74 69 6e 67 20 74 | 68 65 20 41 6c 67 6f 72 |etting t|he Algor|
|000059f0| 69 74 68 6d 20 74 6f 20 | 55 73 65 0a 0a 20 20 20 |ithm to |Use..   |
|00005a00| 41 20 6e 75 6d 62 65 72 | 20 6f 66 20 64 69 66 66 |A number| of diff|
|00005a10| 65 72 65 6e 74 20 76 61 | 72 69 61 74 69 6f 6e 73 |erent va|riations|
|00005a20| 20 6f 6e 20 74 68 65 20 | 6f 72 69 67 69 6e 61 6c | on the |original|
|00005a30| 20 62 61 63 6b 2d 70 72 | 6f 70 61 67 61 74 69 6f | back-pr|opagatio|
|00005a40| 6e 0a 61 6c 67 6f 72 69 | 74 68 6d 20 68 61 76 65 |n.algori|thm have|
|00005a50| 20 62 65 65 6e 20 70 72 | 6f 70 6f 73 65 64 20 69 | been pr|oposed i|
|00005a60| 6e 20 6f 72 64 65 72 20 | 74 6f 20 73 70 65 65 64 |n order |to speed|
|00005a70| 20 75 70 20 63 6f 6e 76 | 65 72 67 65 6e 63 65 2e | up conv|ergence.|
|00005a80| 20 20 53 6f 6d 65 0a 6f | 66 20 74 68 65 73 65 20 |  Some.o|f these |
|00005a90| 68 61 76 65 20 62 65 65 | 6e 20 62 75 69 6c 74 20 |have bee|n built |
|00005aa0| 69 6e 74 6f 20 74 68 65 | 73 65 20 73 69 6d 75 6c |into the|se simul|
|00005ab0| 61 74 6f 72 73 2e 20 20 | 53 6f 6d 65 20 6f 66 20 |ators.  |Some of |
|00005ac0| 74 68 65 20 6d 65 74 68 | 6f 64 73 0a 63 61 6e 20 |the meth|ods.can |
|00005ad0| 62 65 20 6d 69 78 65 64 | 20 74 6f 67 65 74 68 65 |be mixed| togethe|
|00005ae0| 72 2e 20 20 54 68 65 20 | 74 77 6f 20 6d 6f 73 74 |r.  The |two most|
|00005af0| 20 69 6d 70 6f 72 74 61 | 6e 74 20 63 68 6f 69 63 | importa|nt choic|
|00005b00| 65 73 20 61 72 65 20 74 | 68 65 0a 64 65 72 69 76 |es are t|he.deriv|
|00005b10| 61 74 69 76 65 20 74 65 | 72 6d 20 74 6f 20 75 73 |ative te|rm to us|
|00005b20| 65 20 61 6e 64 20 74 68 | 65 20 75 70 64 61 74 65 |e and th|e update|
|00005b30| 20 6d 65 74 68 6f 64 20 | 74 6f 20 75 73 65 2e 20 | method |to use. |
|00005b40| 20 54 68 65 20 64 65 66 | 61 75 6c 74 0a 64 65 72 | The def|ault.der|
|00005b50| 69 76 61 74 69 76 65 20 | 69 73 20 74 68 65 20 6f |ivative |is the o|
|00005b60| 6e 65 20 64 65 76 69 73 | 65 64 20 62 79 20 46 61 |ne devis|ed by Fa|
|00005b70| 68 6c 6d 61 6e 3a 0a 0a | 2e 63 65 0a 30 2e 31 20 |hlman:..|.ce.0.1 |
|00005b80| 2b 20 73 28 31 2d 73 29 | 0a 0a 77 68 65 72 65 20 |+ s(1-s)|..where |
|00005b90| 73 20 69 73 20 74 68 65 | 20 61 63 74 69 76 61 74 |s is the| activat|
|00005ba0| 69 6f 6e 20 76 61 6c 75 | 65 20 6f 66 20 74 68 65 |ion valu|e of the|
|00005bb0| 20 75 6e 69 74 2e 20 20 | 54 68 65 20 72 65 61 73 | unit.  |The reas|
|00005bc0| 6f 6e 20 66 6f 72 20 61 | 64 64 69 6e 67 20 69 6e |on for a|dding in|
|00005bd0| 0a 74 68 65 20 30 2e 31 | 20 74 65 72 6d 20 74 6f |.the 0.1| term to|
|00005be0| 20 74 68 65 20 63 6f 72 | 72 65 63 74 20 66 6f 72 | the cor|rect for|
|00005bf0| 6d 75 6c 61 20 66 6f 72 | 20 74 68 65 20 64 65 72 |mula for| the der|
|00005c00| 69 76 61 74 69 76 65 20 | 69 73 20 74 68 61 74 20 |ivative |is that |
|00005c10| 77 68 65 6e 20 73 20 69 | 73 0a 63 6c 6f 73 65 20 |when s i|s.close |
|00005c20| 74 6f 20 30 20 6f 72 20 | 31 2c 20 74 68 65 20 61 |to 0 or |1, the a|
|00005c30| 6d 6f 75 6e 74 20 6f 66 | 20 65 72 72 6f 72 20 70 |mount of| error p|
|00005c40| 61 73 73 65 64 20 62 61 | 63 6b 20 69 73 20 76 65 |assed ba|ck is ve|
|00005c50| 72 79 20 73 6d 61 6c 6c | 20 61 6e 64 20 73 6f 0a |ry small| and so.|
|00005c60| 6c 65 61 72 6e 69 6e 67 | 20 69 73 20 76 65 72 79 |learning| is very|
|00005c70| 20 73 6c 6f 77 2e 20 20 | 41 64 64 69 6e 67 20 74 | slow.  |Adding t|
|00005c80| 68 65 20 30 2e 31 20 73 | 70 65 65 64 73 20 75 70 |he 0.1 s|peeds up|
|00005c90| 20 74 68 65 20 6c 65 61 | 72 6e 69 6e 67 20 70 72 | the lea|rning pr|
|00005ca0| 6f 63 65 73 73 2e 0a 28 | 46 6f 72 20 74 68 65 20 |ocess..(|For the |
|00005cb0| 6f 72 69 67 69 6e 61 6c | 20 64 65 73 63 72 69 70 |original| descrip|
|00005cc0| 74 69 6f 6e 20 6f 66 20 | 74 68 69 73 20 6d 65 74 |tion of |this met|
|00005cd0| 68 6f 64 2c 20 73 65 65 | 20 22 46 61 73 74 65 72 |hod, see| "Faster|
|00005ce0| 20 4c 65 61 72 6e 69 6e | 67 0a 56 61 72 69 61 74 | Learnin|g.Variat|
|00005cf0| 69 6f 6e 73 20 6f 66 20 | 42 61 63 6b 2d 50 72 6f |ions of |Back-Pro|
|00005d00| 70 61 67 61 74 69 6f 6e | 3a 20 20 41 6e 20 45 6d |pagation|:  An Em|
|00005d10| 70 69 72 69 63 61 6c 20 | 53 74 75 64 79 22 2c 20 |pirical |Study", |
|00005d20| 62 79 20 53 63 6f 74 74 | 20 45 2e 0a 46 61 68 6c |by Scott| E..Fahl|
|00005d30| 6d 61 6e 2c 20 69 6e 20 | 5c 66 49 50 72 6f 63 65 |man, in |\fIProce|
|00005d40| 65 64 69 6e 67 73 20 6f | 66 20 74 68 65 20 31 39 |edings o|f the 19|
|00005d50| 38 38 20 43 6f 6e 6e 65 | 63 74 69 6f 6e 69 73 74 |88 Conne|ctionist|
|00005d60| 20 4d 6f 64 65 6c 73 20 | 53 75 6d 6d 65 72 0a 53 | Models |Summer.S|
|00005d70| 63 68 6f 6f 6c 5c 66 52 | 2c 20 4d 6f 72 67 61 6e |chool\fR|, Morgan|
|00005d80| 20 4b 61 75 66 6d 61 6e | 6e 2c 20 31 39 38 39 2e | Kaufman|n, 1989.|
|00005d90| 29 20 20 42 65 73 69 64 | 65 73 20 46 61 68 6c 6d |)  Besid|es Fahlm|
|00005da0| 61 6e 27 73 20 64 65 72 | 69 76 61 74 69 76 65 20 |an's der|ivative |
|00005db0| 61 6e 64 20 74 68 65 0a | 6f 72 69 67 69 6e 61 6c |and the.|original|
|00005dc0| 20 6f 6e 65 2c 20 74 68 | 65 20 64 69 66 66 65 72 | one, th|e differ|
|00005dd0| 65 6e 74 69 61 6c 20 73 | 74 65 70 20 73 69 7a 65 |ential s|tep size|
|00005de0| 20 6d 65 74 68 6f 64 20 | 28 73 65 65 20 22 53 74 | method |(see "St|
|00005df0| 65 70 73 69 7a 65 20 56 | 61 72 69 61 74 69 6f 6e |epsize V|ariation|
|00005e00| 0a 4d 65 74 68 6f 64 73 | 20 66 6f 72 20 41 63 63 |.Methods| for Acc|
|00005e10| 65 6c 65 72 61 74 69 6e | 67 20 74 68 65 20 42 61 |eleratin|g the Ba|
|00005e20| 63 6b 2d 50 72 6f 70 61 | 67 61 74 69 6f 6e 20 41 |ck-Propa|gation A|
|00005e30| 6c 67 6f 72 69 74 68 6d | 22 2c 20 62 79 20 43 68 |lgorithm|", by Ch|
|00005e40| 65 6e 20 61 6e 64 0a 4d | 61 72 73 2c 20 69 6e 20 |en and.M|ars, in |
|00005e50| 5c 66 49 49 4a 43 4e 4e | 2d 39 30 2d 57 41 53 48 |\fIIJCNN|-90-WASH|
|00005e60| 2d 44 43 5c 66 52 2c 20 | 4c 61 77 72 65 6e 63 65 |-DC\fR, |Lawrence|
|00005e70| 20 45 72 6c 62 61 75 6d | 2c 20 31 39 39 30 29 20 | Erlbaum|, 1990) |
|00005e80| 74 61 6b 65 73 20 74 68 | 65 0a 64 65 72 69 76 61 |takes th|e.deriva|
|00005e90| 74 69 76 65 20 74 6f 20 | 62 65 20 31 20 69 6e 20 |tive to |be 1 in |
|00005ea0| 74 68 65 20 6c 61 79 65 | 72 20 67 6f 69 6e 67 20 |the laye|r going |
|00005eb0| 69 6e 74 6f 20 74 68 65 | 20 6f 75 74 70 75 74 20 |into the| output |
|00005ec0| 75 6e 69 74 73 20 61 6e | 64 20 75 73 65 73 20 74 |units an|d uses t|
|00005ed0| 68 65 0a 6f 72 69 67 69 | 6e 61 6c 20 64 65 72 69 |he.origi|nal deri|
|00005ee0| 76 61 74 69 76 65 20 66 | 6f 72 20 61 6c 6c 20 6f |vative f|or all o|
|00005ef0| 74 68 65 72 20 6c 61 79 | 65 72 73 2e 20 20 54 68 |ther lay|ers.  Th|
|00005f00| 65 20 6c 65 61 72 6e 69 | 6e 67 20 72 61 74 65 20 |e learni|ng rate |
|00005f10| 66 6f 72 20 74 68 65 0a | 69 6e 6e 65 72 20 6c 61 |for the.|inner la|
|00005f20| 79 65 72 73 20 69 73 20 | 6e 6f 72 6d 61 6c 6c 79 |yers is |normally|
|00005f30| 20 73 65 74 20 74 6f 20 | 31 2f 31 30 20 74 68 65 | set to |1/10 the|
|00005f40| 20 72 61 74 65 20 69 6e | 20 74 68 65 20 6f 75 74 | rate in| the out|
|00005f50| 65 72 20 6c 61 79 65 72 | 2e 20 20 54 6f 0a 73 65 |er layer|.  To.se|
|00005f60| 74 20 74 68 65 20 64 65 | 72 69 76 61 74 69 76 65 |t the de|rivative|
|00005f70| 2c 20 75 73 65 20 74 68 | 65 20 60 41 27 20 63 6f |, use th|e `A' co|
|00005f80| 6d 6d 61 6e 64 20 61 73 | 20 69 6e 3a 0a 0a 2e 6e |mmand as| in:...n|
|00005f90| 65 34 0a 20 20 20 41 20 | 64 6f 20 20 20 2a 20 75 |e4.   A |do   * u|
|00005fa0| 73 65 20 74 68 65 20 6f | 72 69 67 69 6e 61 6c 20 |se the o|riginal |
|00005fb0| 64 65 72 69 76 61 74 69 | 76 65 0a 20 20 20 41 20 |derivati|ve.   A |
|00005fc0| 64 66 20 20 20 2a 20 75 | 73 65 20 46 61 68 6c 6d |df   * u|se Fahlm|
|00005fd0| 61 6e 27 73 20 64 65 72 | 69 76 61 74 69 76 65 0a |an's der|ivative.|
|00005fe0| 20 20 20 41 20 64 64 20 | 20 20 2a 20 75 73 65 20 |   A dd |  * use |
|00005ff0| 74 68 65 20 64 69 66 66 | 65 72 65 6e 74 69 61 6c |the diff|erential|
|00006000| 20 73 74 65 70 20 73 69 | 7a 65 20 64 65 72 69 76 | step si|ze deriv|
|00006010| 61 74 69 76 65 0a 0a 20 | 20 20 54 68 65 20 61 6c |ative.. |  The al|
|00006020| 67 6f 72 69 74 68 6d 20 | 63 6f 6d 6d 61 6e 64 20 |gorithm |command |
|00006030| 63 61 6e 20 63 6f 6e 74 | 61 69 6e 20 6f 74 68 65 |can cont|ain othe|
|00006040| 72 20 73 75 62 2d 63 6f | 6d 6d 61 6e 64 73 20 62 |r sub-co|mmands b|
|00006050| 65 73 69 64 65 73 20 74 | 68 65 0a 73 65 74 74 69 |esides t|he.setti|
|00006060| 6e 67 20 6f 66 20 74 68 | 65 20 64 65 72 69 76 61 |ng of th|e deriva|
|00006070| 74 69 76 65 2e 20 20 54 | 68 65 20 6f 74 68 65 72 |tive.  T|he other|
|00006080| 20 6d 61 6a 6f 72 20 63 | 68 6f 69 63 65 20 69 73 | major c|hoice is|
|00006090| 20 74 68 65 20 75 70 64 | 61 74 65 20 6d 65 74 68 | the upd|ate meth|
|000060a0| 6f 64 2e 0a 54 68 65 20 | 63 68 6f 69 63 65 73 20 |od..The |choices |
|000060b0| 61 72 65 20 74 68 65 20 | 6f 72 69 67 69 6e 61 6c |are the |original|
|000060c0| 20 6f 6e 65 2c 20 74 68 | 65 20 64 69 66 66 65 72 | one, th|e differ|
|000060d0| 65 6e 74 69 61 6c 20 73 | 74 65 70 20 73 69 7a 65 |ential s|tep size|
|000060e0| 20 6d 65 74 68 6f 64 2c | 0a 4a 61 63 6f 62 27 73 | method,|.Jacob's|
|000060f0| 20 64 65 6c 74 61 2d 62 | 61 72 2d 64 65 6c 74 61 | delta-b|ar-delta|
|00006100| 20 6d 65 74 68 6f 64 2c | 20 74 68 65 20 63 6f 6e | method,| the con|
|00006110| 74 69 6e 75 6f 75 73 20 | 75 70 64 61 74 65 20 6d |tinuous |update m|
|00006120| 65 74 68 6f 64 20 61 6e | 64 20 74 68 65 0a 63 6f |ethod an|d the.co|
|00006130| 6e 74 69 6e 75 6f 75 73 | 20 75 70 64 61 74 65 20 |ntinuous| update |
|00006140| 6d 65 74 68 6f 64 20 77 | 69 74 68 20 74 68 65 20 |method w|ith the |
|00006150| 64 69 66 66 65 72 65 6e | 74 69 61 6c 20 73 74 65 |differen|tial ste|
|00006160| 70 20 73 69 7a 65 20 65 | 74 61 73 2e 20 20 54 6f |p size e|tas.  To|
|00006170| 20 73 65 74 0a 74 68 65 | 73 65 20 75 70 64 61 74 | set.the|se updat|
|00006180| 65 20 6d 65 74 68 6f 64 | 73 20 75 73 65 3a 0a 0a |e method|s use:..|
|00006190| 2e 6e 61 0a 2e 6e 66 0a | 2e 6e 65 36 0a 20 20 20 |.na..nf.|.ne6.   |
|000061a0| 41 20 75 6f 20 20 20 2a | 20 74 68 65 20 6f 72 69 |A uo   *| the ori|
|000061b0| 67 69 6e 61 6c 20 75 70 | 64 61 74 65 20 6d 65 74 |ginal up|date met|
|000061c0| 68 6f 64 0a 20 20 20 41 | 20 75 64 20 20 20 2a 20 |hod.   A| ud   * |
|000061d0| 74 68 65 20 64 69 66 66 | 65 72 65 6e 74 69 61 6c |the diff|erential|
|000061e0| 20 73 74 65 70 20 73 69 | 7a 65 20 6d 65 74 68 6f | step si|ze metho|
|000061f0| 64 0a 20 20 20 41 20 75 | 6a 20 20 20 2a 20 4a 61 |d.   A u|j   * Ja|
|00006200| 63 6f 62 27 73 20 64 65 | 6c 74 61 2d 62 61 72 2d |cob's de|lta-bar-|
|00006210| 64 65 6c 74 61 20 6d 65 | 74 68 6f 64 0a 20 20 20 |delta me|thod.   |
|00006220| 41 20 75 63 20 20 20 2a | 20 74 68 65 20 63 6f 6e |A uc   *| the con|
|00006230| 74 69 6e 75 6f 75 73 20 | 75 70 64 61 74 65 20 6d |tinuous |update m|
|00006240| 65 74 68 6f 64 0a 20 20 | 20 41 20 75 43 20 20 20 |ethod.  | A uC   |
|00006250| 2a 20 74 68 65 20 63 6f | 6e 74 69 6e 75 6f 75 73 |* the co|ntinuous|
|00006260| 20 75 70 64 61 74 65 20 | 6d 65 74 68 6f 64 20 77 | update |method w|
|00006270| 69 74 68 20 74 68 65 20 | 64 69 66 66 65 72 65 6e |ith the |differen|
|00006280| 74 69 61 6c 0a 20 20 20 | 20 20 20 20 20 20 20 2a |tial.   |       *|
|00006290| 20 73 74 65 70 20 73 69 | 7a 65 20 65 74 61 73 0a | step si|ze etas.|
|000062a0| 2e 61 64 0a 2e 66 69 0a | 0a 54 68 65 20 64 69 66 |.ad..fi.|.The dif|
|000062b0| 66 65 72 65 6e 74 69 61 | 6c 20 73 74 65 70 20 73 |ferentia|l step s|
|000062c0| 69 7a 65 20 6d 65 74 68 | 6f 64 20 75 73 65 73 20 |ize meth|od uses |
|000062d0| 74 68 65 20 73 74 61 6e | 64 61 72 64 20 65 74 61 |the stan|dard eta|
|000062e0| 20 77 68 65 6e 20 75 70 | 64 61 74 65 73 0a 61 72 | when up|dates.ar|
|000062f0| 65 20 6d 61 64 65 20 74 | 6f 20 74 68 65 20 75 6e |e made t|o the un|
|00006300| 69 74 73 20 6c 65 61 64 | 69 6e 67 20 69 6e 74 6f |its lead|ing into|
|00006310| 20 74 68 65 20 6f 75 74 | 70 75 74 20 6c 61 79 65 | the out|put laye|
|00006320| 72 2e 20 20 46 6f 72 20 | 64 65 65 70 65 72 20 6c |r.  For |deeper l|
|00006330| 61 79 65 72 73 2c 0a 61 | 6e 6f 74 68 65 72 20 76 |ayers,.a|nother v|
|00006340| 61 6c 75 65 20 77 69 6c | 6c 20 62 65 20 75 73 65 |alue wil|l be use|
|00006350| 64 2e 20 20 54 68 65 20 | 64 65 66 61 75 6c 74 20 |d.  The |default |
|00006360| 69 73 20 74 6f 20 75 73 | 65 20 61 6e 20 65 74 61 |is to us|e an eta|
|00006370| 2c 20 63 61 6c 6c 65 64 | 20 65 74 61 32 2c 0a 66 |, called| eta2,.f|
|00006380| 6f 72 20 74 68 65 20 69 | 6e 6e 65 72 20 6c 61 79 |or the i|nner lay|
|00006390| 65 72 73 20 74 68 61 74 | 20 69 73 20 6f 6e 65 2d |ers that| is one-|
|000063a0| 74 65 6e 74 68 20 74 68 | 65 20 73 74 61 6e 64 61 |tenth th|e standa|
|000063b0| 72 64 20 65 74 61 2e 20 | 20 54 68 65 73 65 20 65 |rd eta. | These e|
|000063c0| 74 61 73 0a 62 6f 74 68 | 20 67 65 74 20 73 65 74 |tas.both| get set|
|000063d0| 20 75 73 69 6e 67 20 74 | 68 65 20 60 65 27 20 63 | using t|he `e' c|
|000063e0| 6f 6d 6d 61 6e 64 20 28 | 6e 6f 74 20 61 20 73 75 |ommand (|not a su|
|000063f0| 62 2d 63 6f 6d 6d 61 6e | 64 20 6f 66 20 74 68 65 |b-comman|d of the|
+--------+-------------------------+-------------------------+--------+--------+