NetNews Usenet Archive 1993 #3/NN_1993_3.iso/spool/bit/listserv/csgl/2246

Text File | 1993-01-21 | 7.0 KB | 140 lines

Comments: Gated by [e-mail]
Path: sparky!uunet!gatech!europa.asd.contel.com!paladin.american.edu!auvm!BEN.DCIEM.DND.CA!MMT
Return-Path: <@VMD.CSO.UIUC.EDU:[e-mail]>
Message-ID: <[e-mail]>
Newsgroups: bit.listserv.csg-l
Date:         Thu, 21 Jan 1993 19:06:14 EST
Sender:       "Control Systems Group Network (CSGnet)" <[e-mail]>
From:         [e-mail]
Subject:      More on Occam
Lines: 128

[Martin Taylor 930121 18:30]

The following is a discussion draft paper used in a local seminar 2 Jan 1987.
In view of Bruce's postings on MDL, I think it worth putting out on CSG-L.

Specifically it follows up on (Bruce Nevin Thu 930121 08:24:53) quoting
Albert Boulanger:

>There is a pair of back-to-back papers in the Journal of the Royal
>Statistics Society B(1987), by Jorma Rissanen  and then C.S. Wallace &
>P.R. Freeman on their contributions to MDL-like metrics:
>
>"Stochastic Complexity", Jorma Rissanen, J. R. Statist. Soc B(1987)
>49, No 3, pp 223-239, and 252-265.
>
>"Estimation and Inference by Compact Coding", J. R. Statist.  Soc B(1987)
>49, No 3, pp 240-265.
>
>These two papers are of interest along with the discussion section on
>pages 252-265. The crux of the issue (besides the silly "who published
>first") is Rissanen use of a "universal" prior. Wallace does not,
>being a die-hard prior believer. (Personally I feel that often the
>choice of prior can lead to radically different answers, and any
>attempt at making a more "robust" method should be welcomed.

The discussion draft argues that there is no real conflict between these
two positions.  They represent different aspects of how an idea may be
considered.  And yes, indeed they lead to radically different answers,
consistent with the difficulties PCT has in being accepted in the
orthodox psychological community.

In this draft, the word "idea" is used exactly as "hypothesis" was used
in the Occam's Razor paper.

----------------------------------------
The Structure of an Idea

M. M. Taylor

(a discussion paper distributed for a seminar 2 Jan 1987)

How much structure is there in an idea? The answer depends on the frame
within which the idea is considered.  That frame includes the existing
conceptual structure of the idea-gatherer (IG), as well as the region of
the outer world within which the idea is considered,  There are thus
two environments within which the idea must be evaluated: the internal
environment of IG, and the part of the outer world within which the idea
is potentially valid.  The amount of structure in the idea may differ in
the two environments

The "credibility" of an idea (in the sense of the Occam's razor argument
of Taylor, 1972) depends on the minimum description of the idea that in
conjunction with the preconceptions of IG allows IG to use the idea.  It
depends also on the reduction in the size of the description required to
describe the outer world environment.  We can use the term "acceptability"
for the first term, which designates how easily the idea fits into the
existing conceptual structure of IG.  Small ideas, or ideas that agree
with  preconceptions, are readily added but make little change in IG's
conceptual structure.  Large ideas (those that require much description)
are hard to assimilate but add considerable structure.  From the viewpoint
of IG, the amount of structure in the idea is the amount it adds to the
minimum descripton of IG's concept structure.

The second component of the structure of an idea is its "value", which is
the amount by which use of the idea reduces the description of the outer
world from the basic "random-world" description.  This is different from
the value of the idea to IG, which is the amount by which use of the idea
reduces the description of the world from that based on IG's preconceptions.
An idea may have intrinsic value, but have negative value to IG because the
preconceptions already provide a simpler description of the world than the
idea can give.  In such a case, the idea must contradict some preconceptions,
since if it did not, adding it could not increase the complexity of the
description of the world

If an idea contradicts some of IG's preconceptions, it must have a longer
minimum description than the same idea presented to an IG with whose
preconceptions it agrees.  Hence an idea of negative value is less acceptable
as well as of less value to such an IG than to one with whose preconceptions
it does not disagree

An idea can have positive value for an IG, even when it does disagree with
some of IG's preconceptions.  This can occur when using the idea provides
a simpler description of the world than the preconceptions did, even
including the fact that the new description must include statements about
the contradicted preconceptions.  If the idea causes IG to alter
preconceptions, then the idea may also simplify IG's conceptual structure
and thereby add to the simplification of the description of the world.
But this is not required for incorporation of the idea into IG's cognitive
structure.  Contradictory concepts can be held, but a description of the
world using them will require some identification of which sides of the
contradiction are being used for any particular aspect of the world.  In
other words, a description that incorporates contradictions will be more
complex and less credible than an analogous one that does not.

It is possible for an idea to describe wrongly a part of the world, and
yet have positive value, since IG can use it together with a qualifier
"except under such-and-such circumstances."  Naturally, the real-world
description is lengthened by the addition of such qualifiers, but the
resulting descriptions may still be shorter than can be produced without
the idea.  For example, Newton's laws of motion are known to be quite
wrong at high velocities, but provided the qualifier "at low velocities"
is used, their descriptions are good enough for most purposes and much
simpler than the correct, relativistic, ones.


Incorporation of ideas

The acceptability of an idea determines how easily it is incorporated
into IG's conceptual structure, but its value determines how readily the
idea can be dislodged by competing ideas.  A competing idea is one that
provides an alternate description of the same part of the outer world.
Such an idea will contradict the original idea, in the sense that a world
description using both will require a statement as to which is being used.
Hence a competing idea will be less acceptable than it would have been in
the absence of the original idea.

In conjunction with the fractal diffusion model for the acquisition of
conceptual structure, the notion of acceptability as a "stickiness"
parameter, and value as a "fragility" parameter might provide an
interesting description of the learning process.
----------------------

The fractal diffusion model is probably not appropriate for description
on CSG-L, unless we get into a really serious discussion on reorganization
and learning.  It would take a great deal of precursor work to set up
what the above draft calls "preconceptions."

Martin