NetNews Usenet Archive 1993 #3/NN_1993_3.iso/spool/comp/ai/neuraln/5005

Internet Message Format | 1993-01-28 | 1.6 KB

Path: sparky!uunet!vnet.ibm.com
From: [e-mail] (Scott Fischthal)
Message-ID: <[e-mail]>
Date: Thu, 28 Jan 93 08:54:40 EST
Newsgroups: comp.ai.neural-nets
Subject: Re: a beginner's quiestion
Organization: IBM Federal Systems Co.
Disclaimer: This posting represents the poster's views, not those of IBM
News-Software: UReply 3.0
References: <[e-mail]>
Lines: 24

In <[e-mail]> <[e-mail]> writes:
> My question is: When I run the XOR problem with one hidden layer
> it converges (abeit slowly). But if I use more than one hidden
> layer, it gets stucked (in local maximum?).
> Is this supposed to happen or It's just my code...?

It probably is not your code; standard backprop nets with multiple
hidden layers are always more difficult to control.  How many nodes are
you using in each hidden layer, what is your learning rate and momentum
and what activation function are you using?  Also, how many epochs (runs
through the data set) are you giving it before you throw in the towel?
With some sets of random weights, I've found XOR-analogous sets to take
literally hundreds of thousands of iterations to learn (on the other
hand, sometimes the net converges very quickly).

Keep in mind that the XOR problem is not really an appropriate one to
solve with neural nets; it's important to know that it *can* be solved,
but such discrete, highly linearly non-separable problems are generally
better handled with other techniques.

Scott Fischthal
Artificial Intelligence Technology Center
IBM Federal Systems Company
Gaithersburg, MD