NetNews Usenet Archive 1992 #27/NN_1992_27.iso/spool/comp/parallel/2571

Text File | 1992-11-20 | 52.2 KB | 1,424 lines

Newsgroups: comp.parallel
Path: sparky!uunet!gatech!hubcap!fpst
From: [e-mail] (Scott Kohn)
Subject: [long] summary of scientific parallel computing requests
Message-ID: <[e-mail]>
Followup-To: comp.parallel
Keywords: parallel processing, scientific computing, models
Sender: [e-mail]
Nntp-Posting-Host: picard.ucsd.edu
Organization: UC-San Diego CSE Department
Date: 19 Nov 92 21:33:54 GMT
Approved: [e-mail]
Lines: 1409



Some time ago, I posted a request for information about parallel programming
models for scientific computing.  (By models, I really meant the different types
of abstractions presented to the programmer, so languages, environments, and
libraries would also apply.)

Many thanks to all who responded:

"Chad O. Yoshikawa" <[e-mail]>
"Christopher T. Faigle" <[e-mail]>
"L. V. Kale'" <[e-mail]>
"Neelakantan Sundaresan" <[e-mail]>
"Robert J. Harrison (509-375-2037)" <[e-mail]>
Brian Wylie <[e-mail]>
Carolyn Schauble <[e-mail]>
Gregory T. Byrd <[e-mail]>
[e-mail] (Jean Louis Pazat)
Jeff Martens <[e-mail]>
Mats Brorsson <[e-mail]>
Monika Haerdtner <[e-mail]>
Roland Ruehl <[e-mail]>
Steven Zenith <[e-mail]>
Thomas Schnekenburger <[e-mail]>
[e-mail]
[e-mail] (Kalhan Ajay)
[e-mail] (brent)
[e-mail] (Craig Chase)
[e-mail]
[e-mail]
[e-mail] (Alan Bryant Heirich)
[e-mail] (David Kaminsky)
[e-mail]
[e-mail]
[e-mail] (Maneesh Dhagat)
[e-mail]
[e-mail] ("Timothy G. Mattson")
[e-mail] (Kevin McCurley)
[e-mail]
[e-mail] (Robert Bernecky)
[e-mail] (Richard S. Brice)
[e-mail] (David Skillicorn)
[e-mail] (Scott Whitman)
[e-mail] (Stephen W Poole)
[e-mail] (Thomas Willis)
[e-mail] (Michael Pernice)
[e-mail] (Randy Winney)

********************************************************************************

From: Gregory T. Byrd <[e-mail]>

Sisal has been favored by some for scientific programming.
(There is a Cray version...)

...Greg Byrd             North Carolina Supercomputing Center
   [e-mail]        P.O. Box 12889
   [phone]         Research Triangle Park, NC  27709-2889

********************************************************************************

From: [e-mail] (Randy Winney)

In the February issue of Supercomputing there was an article about
the functional language Sisal that Lawrence Livermore is promoting
to replace Fortran. Contact [e-mail] for more info.

--
Randy Winney       [e-mail]
Mobil Exploration & Producing Technology Center
Dallas, Texas

********************************************************************************

From: Mats Brorsson <[e-mail]>

First I would like to say that I am intersted in all answers you get.

There is an article in the September 92 issue of IEEE Computer which deals
with programming paradigms, some of which parallel. One of the references
to this article deals with Linda which you might want to add to your list.
Also don't forget the Single-Program-Multiple-Data (SPMD) model which
was first proposed by Darema-Rogers et al. [1] and has since
been used for a variety of languages. The SPLASH suite [2] of parallel
benchmarks are written in this model using macros from ANL to express
parallelism.

I am particularly interested in if various paradigms for writing shared memory
parallel programs result in different sharing behaviour. If you get any
answers reflecting this particular problem please let me know.

Regards,
  /Mats

[1]  @article{Darema:EPEX,
            author="Darema F., George D. A., Norton V. A., and
            Pfister G. F.",
            title="A single-program-multiple-data computational model for
                    EPEX/FORTRAN.",
            journal="Parallel Computing",
            volume=7,
            number=1,
            year=1988,
            month="April",
            pages="11-24"}

[2]  @Book{Boyle:ANL,
            author="Boyle J., Butler R., Disz T., Glickfield B.,
            Lusk E., Overbeek R., Patterson J., and Stevens R.,",
            title="Portable Programs for Parallel Processors",
            publisher="Holt, Rinehart and Winston inc.",
            year=1987,
        city="New York"}


--
========================================================================
Mats Brorsson                              internet: [e-mail]
Dept. of Comp. Eng., Lund University       phone:    [phone]
P.O. Box 118, S-221 00 Lund, SWEDEN        fax:      [phone]

********************************************************************************

From: [e-mail] (Michael Pernice)

Scott,

Your list does not include Compositional C++ (CC++) which is being
developed by Carl Kesselman at CalTech.  It is an object-oriented
approach that hides the details of message-passing inside the object.
I don't have a pointer to a reference, but I believe you can contact
him at [e-mail], or at [phone].

Tony Skjellum at LLNL has been developing an interface called The
Multicomputer Toolbox.  Again, this appears to be an object-oriented
approach to achieving data distribution independence, portability,
and a level of abstraction from the hardware.  You can contact him
at [e-mail] or [phone].  He's published several
papers in this area; the one I've read (which contains references to
several other articles) is

  Falgout, Skjellum, Smith and Still, "The Multicomputer Toolbox
     Approach to Concurrent BLAS and LACS", Proceedings of the
     Scalable High Performance Computing Conference SHPCC-92,
     IEEE Computer Society Press, Los Alamitos, CA.

I presume you regard Vienna Fortran to be a variant of Fortran D.
It would be useful to find an elucidation of the relationships
among these variants, and of what elements are included in HPF.

I'd be interested in seeing a summary of the references that you
find.

----------------------------------------+--------------------------------------
Michael Pernice                         |    Utah Supercomputing Institute
                                        |    85 Student Services Building
Bell-net:  [phone]               |    University of Utah
Internet:  [e-mail] |    Salt Lake City, Utah 84112
----------------------------------------+--------------------------------------

********************************************************************************

From: [e-mail] (David Skillicorn)

You might be interested in my summary
paper in IJPP (April 1991, really April
1992). There's no emphasis on scientific
computation, but there are lots of
references.

You can pick up a large bibliography
from ftp.qucis.queensu.ca in directory
pub/skill.

           -david skillicorn

********************************************************************************

From: Jeff Martens <[e-mail]>

I'd include Sisal and Id.
--

-- Jeff ([e-mail])

********************************************************************************

From: [e-mail]

Scott --

I don't know enough about Fortran dialects to know how it relates
to Fortran-D, but MasPar has a parallel version of Fortran-90.
They also have a parallel C called MPL which is fairly distinct from
C*.

Thinking Machines has a parallel Lisp -- last I heard (quite a while
ago) it was called *Lisp.  An interesting article on a sizable
parallel scientific application written in this language is by Zhao
and Johnsson in SIAM J. Stat. Comput., Vol 12, Nov. 1991.

A language you definitely need to include is SISAL, a
functional/dataflow language developed jointly be participants from
MIT, Colorado State, Livermore Labs, and others.  Send e-mail to David
Cann, [e-mail], for more info.  You can FTP the manual, and
they have lots of papers showing some nice performance compared to
Fortran on real parallel machines.

Finally, Stephen Bevan of the University of Manchester
([e-mail]) has collected a bunch of numerical analysis
programs written in the functional language ML.  As far as I know
there are no high performance implementations of ML yet, but it's a
good bet someone will develop a port for a parallel machine soon.

Good luck with the survey -- I'm definitely interested in the results.

JC

********************************************************************************

From: [e-mail]

Although it is more of a "tool" than a "model", you might be interested in
p4:

                      p4

p4 is a library of macros and subroutines developed at Argonne National
Laboratory for programming a variety of parallel machines.  Its predecessor
was the m4-based "Argonne macros" system described in the Holt, Rinehart, and
Winston book "Portable Programs for Parallel Processors, by Lusk, Overbeek, et
al., from which p4 takes its name.  The current p4 system maintains the same
basic computational models described there (monitors for the shared-memory
model, message-passing for the distributed-memory model, and support for
combining the two models) while significantly increasing ease and flexibility
of use.

The current release is version 1.2  New features added since version 0.2
include:

  + manual enhanced and converted to latex format
  + Emacs info version of the manual for on-line help
  + SYSV IPC support added for several machines (for shared-memory
     multiprocessing on workstations that support multiple processors)
  + instrumentation added for automatic logging/tracing
  + provided better user-control of message-passing/buffer-management
  + high-resolution clock support added for several machines
  + improved error/interrupt handling
  + optional secure server for faster startup on networks
  + optional automatic logging of events for upshot tracing

New since version 0.1:

  + xdr for communication in a heterogeneous network
  + asynchronous communication of large messages
  + global operations (broadcast, global sum, max, etc.)
  + both master-slave and SPMD models for message-passing programs
  + an improved and simplified Fortran interface
  + an optional secure server
  + ports to more machines

p4 is intended to be portable, simple to install and use, and efficient.  It
can be used to program networks of workstations, advanced parallel
supercomputers like the Intel Touchstone Delta and the Alliant Campus
HiPPI-based system, and single shared-memory multiprocessors.  It has
currently been installed on the following list of machines: Sequent Symmetry,
Encore Multimax, Alliant FX/8, FX/800, and FX/2800, Cray X/MP, Sun, NeXT, DEC,
Silicon Graphics, and IBM RS6000 workstations, Stardent Titan, BBN GP-1000 and
TC-2000, Intel IPSC/860, Intel Touchstone Delta, and Alliant Campus.  It will
soon be ported to the CM-5 and to the Intel Paragon.  It is not difficult to
port to new systems.

A useful companion system is the upshot logging and X-based trace examination
facility.  The macros to create logs are included in p4.  Upshot (an X program
for graphically displaying the logs) is available separately.

You can obtain the complete distribution of p4 by anonymous ftp from
info.mcs.anl.gov.  Take the file p4t1.2.tar.Z from the directory pub/p4.  The
distribution contains all source code, installation instructions, a User's
Guide in both ascii text and latexinfo format, and a collection of examples in
both C and Fortran.

To ask questions about p4, report bugs, contribute examples, etc., send mail
to [e-mail].

                                                   Rusty Lusk
                                                   [e-mail]

********************************************************************************

From: "Robert J. Harrison (509-375-2037)" <[e-mail]>

Do you mean models or environments?  I include below a
mix of both that you might also want to look at.

 1) Linda

\journal{carriero}{N. Carriero and D. Gelernter}{Communications of the ACM}{32}{1989}{444}

\book{lindabook}{N. Carriero and D. Gelernter}{How To Write Parallel Programs. A First Course}{The MIT Press, Cambridge, MA}{1990}

\bibitem{sca} C-Linda is distributed commercially by Scientific Computing
              Associates, New Haven, Connecticut.

\bibitem{edinburgh} G. Wilson, editor (1991) Linda-Like Systems and Their
                    Implementation, Edinburgh Parallel Computing Centre
                    Technical Report 91-13.

\journal{interwork}{W.L. Bain}{ACM SIGPLAN Notices}{24}{1989}{95}

\bibitem{auclinda} C.J. Callsen, I. Cheng and P.L. Hagen, pp. 39 in \cite{edinburgh}.

\bibitem{mercury} G. Schoinas, pp. 105 in \cite{edinburgh}.


 2) strand

\book{strandbook}{I. Foster and S. Taylor}{Strand, New Concepts in Parallel Programming}{Prentice Hall, New Jersey}{1990}

 3) pcn


\journal{pcn}{K.M. Chandy, S. Taylor, C. Kesselman and I. Foster}{Caltech Comp. Sci. Tech. Report}{CS-TR-90-03}{February 1990}{1}

\journal{pcnruntime}{I. Foster and S. Taylor}{Argonne National Laboratory Report}{ANL/MCS-TM-137}{January 1990}{1}

 4) The multitude of message passing environments of which these
    are just a sample

\bibitem{express} Express is a product of ParaSoft, Mission Viejo, CA.

\bibitem{tcgmsg} R.J. Harrison (1991) Int. J. Quant Chem., in press.

\bibitem{p4} {\tt p4} are a set of portable message passing routines
             being distributed by Ewing Lusk of the Math. and Comp. Sci.
             division at Argonne.  They are the current version of the
             tools described in ref. \cite{parmacs}.

\book{parmacs}{J. Boyle, R. Butler, T. Disz, B. Glickfeld, E. Lusk,
               R. Overbeek, J. Patterson and R. Stevens}{Portable
               Programs for Parallel Processors}{Holt, Rinehart
                 and Winston, Inc., New York, NY}{1987}

\journal{picl}{G.A. Geist, M.T. Heath, B.W. Peyton and P.H. Worley}{Oak Ridge National Laboratory Tech. Report}{TM-11616}{1990}{1}

\journal{pvm}{A. Beguelin, J. Dongarra, Al Geist, R. Manchek and V. Sunderam}{Oak Ridge National Laboratory Tech. Report}{TM-11826}{1991}{1}

  5) occam is very popular with scientists using transupters in Europe


\journal{occam}{D. May}{ACM SIGPLAN Notices}{18}{1983}{69}

\book{occambook}{K.C. Bowler, R.D. Kenway, G.S. Pawley, D. Roweth}{Occam 2 Programming Language}{Prentice-Hall, Englewood Cliffs, NJ}{1984}



I'd be very interested in seeing a summary if you find it
possible to generate one ... you are probably getting much
longer messages than this from others on the net!

Robert J. Harrison

Mail Stop K1-90                             tel: [phone]
Battelle Pacific Northwest Laboratory       fax: [phone]
P.O. Box 999                             E-mail: [e-mail]
Richland WA 99352

********************************************************************************

From: [e-mail] (Scott Whitman)

Yes, there were several models developed here at LLNL.  They are called
PFP and PCP; portable (Fortran or C) pre-processors.  They are intended
for scalable shared memory architectures.  Also, I'm not sure if you include
items such as C-Threads or Linda.

        Scott

--------------------------------------------------------
Scott R. Whitman, Ph.D.        [e-mail]
LLNL, L-416, P.O. Box 808    [phone]
Livermore, CA 94550

********************************************************************************

From: [e-mail] (Alan Bryant Heirich)

It sounds like you are looking for parallel programming environments
or languages, from your list.  Don't overlook

Program Composition Notation (PCN)
Mani Chandy & Stephen Taylor
California Institute of Technology

PCN is a language based on the Unity model (see Chandy & Misra,
Parallel Program Design) and Concurrent Prolog (see Taylor,
Parallel Logic Programming).  It is generally intended for the
new class of multicomputer architectures, distributed memory message
passing machines with 100's, 1000's, or more nodes.  Examples of these
machines are: the J and M machines (MIT); the Mosaic (Cal Tech);
the Touchstone/Paragon project (Intel).  Main features are that
programs are declarative and asynchronous, and execution order is
determined by data dependencies, sort of like high level dataflow.
Communication and synchronization issues are entirely handled by
the language run time system.

I'm using PCN to model the transition from equilibrium to turbulence
in a clasic fluid mechanics problem.  Algorithmically this amounts to
solving the incompressible Navier-Stokes PDE's by Newton's method.
Each step of Newton's method is solved by a parallel red-black Gauss-Seidel
iteration.  These concurrent distributed algorithms have mapped
beautifully into the PCN programming model, which is very non-von Neumann.

For a reference on PCN

Chandy & Taylor (1991 or 1992)

Sorry I've forgotten the title, it's something like "Introduction
to Parallel Programming", publiushed by Bartlett & Jones.  They
use it in a 3rd term freshman course here at Cal Tech, so it's not
too advanced (but, frankly, Cal Tech freshmen or often like seniors
elsewhere!  I can comment honestly because I was never one).

For a ref on the work I'm doing, the previous publication from the
project is

Schroder & Keller
Journal of Computation Physics
vol 92, no 1, nov 1990, pp. 197-227
"Wavy Taylor-Vortex Flows via Multigrid-Continuation Methods"

I would be happy to provide more info on PCN or our work if you
have specific interests.

==================================================================
Alan Heirich                    internet: [e-mail]
256-80 California               office:   818/356 4600
   Institute of Technology    lab:      818/356 3903
Pasadena, CA 91125 USA

********************************************************************************

From: [e-mail]

We have developed Modula-2*, an extension of Modula-2 for portable
highly-parallel programming.

Modula-2* features machine independent data allocators and synchronous &
asynchronous FORALL statements with process alignment declarations.

We already have working Modula-2* compilers and libraries for the MasPar
MP-1, MP-2 series, for heterogenous LANs of Unix workstations (p4-based),
and for single Unix workstations (sequential simulation).

If you want to know more about our work I will be pleased to provide you
with more detailed information.

=Ernst=

--
+--------------------------------------------------------+-------------------+
| Ernst A. Heinz              (email: [e-mail]) |                   |
| Institut fuer Programmstrukturen und Datenorganisation | Make it as simple |
| Fakultaet fuer Informatik, Universitaet Karlsruhe      | as possible, but  |
| Postfach 6980, W-7500 Karlsruhe 1, Germany             | not simpler.      |
| (Voice: ++49/(0)721/6084386, FAX: ++49/(0)721/694092)  |                   |
+--------------------------------------------------------+-------------------+

********************************************************************************

From: "L. V. Kale'" <[e-mail]>

One more entry for your list:

Charm / (Chare Kernel)

This a parallel programming system we (me and my students)
have been building for past few years.
The current implementation is C-based, and runs on
iPSC/2, iPSC/860, NCUBE, sequent symmetry, Encore multimax,
Alliant FX/8, FX/2800, and a network of workstations.
The system supports portability, "specific information sharing modes",
and features for reusability of parallel software.
It provides libraries for scheduling, and dynamic load balancing, etc.

(This is just a brief outline. If you need more detailed
info, let me know.)

    L.V. Kale
    Dept. of Comp. Science
    University of Illinois at Urbana Champaign
     [e-mail]

********************************************************************************

From: [e-mail] (David Kaminsky)

    Linda has been used extensively for scientific computation.
See:

@techreport(BCG91,
  key = {BCG91},
  author = {R. Bjornson and  N. Carriero and D. Gelernter
            and T. Mattson and A. Sherman},
  title = {Experience with {L}inda: Notes from the bandwagon},
  institution = YALEU,
  type = {Reserach Report},
  month = aug,
  year = 1991
)

    There are other references in the literature

David

********************************************************************************

From: [e-mail] (brent)

  At Livermore we use a split-join model (Based on Harry Jordan's
Force), called PFP, the Parallel Fortran Preprocessor, and PCP, the C
equivalent.

  The model is shared memory and very lightweight in terms of overhead
  imposed on the programmer.  It has had considerable success localy,
  and is having an impact on some models we see being developed
  currently outside of the lab.

  The references are:

    \bibitem{pcp} Brent Gorda, Karen Warren, Eugene D. Brooks III,
    {\em Programming in PCP},
    Lawrence Livermore National Laboratory,
    UCRL-MA-107029,  April 1991.

    \bibitem{pfp} Karen Warren, Brent Gorda, Eugene D. Brooks III,
    {\em Programming in PFP},
    Lawrence Livermore National Laboratory,
    UCRL-MA-107028, April 1991.


 If you have any questions, please contact me at [e-mail].  We
  also have two years worth of "annual reports" with 30+ papers in each
  describing using parallel architectures, including programming in
  this model.


 Brent Gorda

********************************************************************************

From: [e-mail]

You may wish to add Hypertasking and APR's MIMDizer (Forge-90) to your
list.  Hypertasking is described in Nov. 91 Supercomputing Review.

-Marc Baber
[e-mail]

********************************************************************************

From: Carolyn Schauble <[e-mail]>

AND
    I forgot to mention

        The Force

            (for references, contact [e-mail])

        CJCSchauble
        [e-mail]

********************************************************************************

From: [e-mail] (Richard S. Brice)

There's a report called "A Survey of parallel Programming Tools" available
from NASA Ames which lists some of the parallel systems you posted and
includes a number of others.  The report is RND-91-005, May '91 and the
author is Doreen Cheng.

********************************************************************************

From: "Chad O. Yoshikawa" <[e-mail]>

Scott,
    I'm not quite sure what you mean by 'programming models inteneded
for use with scientific computing'.  By the list you have already given,
it seems that you want different parallel languages added to the list?
The only one on the list I am familiar with is a FORTRAN-D variant, which
I am programming on the I-Warp here at CMU(even though Intel has all
but dropped the project).  Another language is SISAL, which has a report
by David Cann right next to the FORTRAN D article in the ACM of
September 92, I believe.  Here is  a small description:
     It was developed by Lawrence Livermore and Colorado State U.  Like I said
in the description above, it is a functional language that exploits
parallelism without directives, just using the semantics of
the program.  It is hardware independent and is currently supported only
on shared memory machines(Cray,IRIS,etc..) but it is currently
being ported to run on distri. machines as well.

Chad

********************************************************************************

From: Thomas Schnekenburger <[e-mail]>

Hi,

I'm working in a research group where we developed
the parallel and distributed programming language
ParMod-C.

ParMod-C is an extension of ANSI-C. There exist
an implementation on a workstation net and
on a iPSC-Hypercube.

Descriptions of ParMod-C can be found in

@InProceedings{Weininger91,
  author =      "Weininger, A. and Schnekenburger, T. and
                 Friedrich, M.",
  title =       "Parallel and Distributed Programming with {ParMod-C}",
  booktitle =   {First International Conference of the Austrian Center
                 for Parallel Computation},
  year =        "1991",
  address =     "Salzburg",
  month =       "September",
  publisher =   "Springer, LNCS 591"
}

@TechReport{Schnekenburger91,
    author =      "Schnekenburger, Thomas and Weininger, Andreas and
Friedrich, Michael",
    title =       {Introduction to the Parallel and Distributed Programming
Language {ParMod-C}},
    institution = {Technische Universit\"at M\"unchen}},
    year =        "1991",
    number =      "SFB 342/27/91 A",
  month =     Okt
}

The description of the first version of ParMod, based on Pascal
can be found in:

@InProceedings{Eichholz87,
  author =      "Eichholz, S.",
  title =       "Parallel Programming with {ParMod}",
  booktitle =   "Proceedings of the 1987 International Conference on
                 Parallel Processing",
  year =        "1987",
  pages =       "377--380",
}

I'm very interested in a summary.

Thanks.

    Thomas Schnekenburger

--------------------------------------------------------------------
   Thomas Schnekenburger                 |       Orleansstr. 34
                                         |     D 8000 Muenchen 80
   Institut fuer Informatik              |        GERMANY
Technische Universitaet Muenchen         |
--------------------------------------------------------------------
Phone: [phone],            Fax: [phone]
e-mail: [e-mail]
--------------------------------------------------------------------

********************************************************************************

From: Brian Wylie <[e-mail]>

I think that such a summary would be a good idea if you felt up to it.
I've heard of the others, but not these two -- perhaps you could forward
some information if that wasn't too much to ask.

I'm not too sure how "intended for scientific computation" all of the
following are, but they fit in with the others on your list -- if you
need some references I can probably assist with most of them.

  Vienna Fortran from Hans Zima at U Vienna (HPF/Fortran-D -like)
  Dataparallel C from Mike Quinn at Oregon State U (from early C* work)
  Mentat/MPL from Andrew Grimshaw at U Virginia (extended C++ for MIMD)
  uC++/uSystem from Peter Buhr at U Waterloo (concurrent dialect of C++)
  PC/PC++/PF from Ridgway Scott at U Houston (PICL-based for DM&SM MIMD)

along with "vendor" languages such as CM Fortran, MasPar programming
language, DAP Fortran, etc.

  Slainte,                    "democracy: (n) separatism if we want it,     oo
    Brian.                                    but reform -- no deal!" __/\_/`'

Brian J N Wylie | EPCC, University of Edinburgh, JCMB, King's Bldgs, Edinburgh,
<[e-mail]> | EH9 3JZ, Scotland [Tel: [phone], Fax: [phone]]

********************************************************************************

From: [e-mail]

Dear Scott,

I'm really interested in your collecting references about
parallel programming models.

Here, in university of Lille (F), we are working on a new Data-parallel model,
namely HELP (Hyper-Espace et Language Parallele).
We will present a poster at

``Supercomputing'92 poster session''
(Minneapolis, November 16-20th)

and a paper at

``Euromicro Worshop on Parallel and Distributed Processing''
(Las Palmas, Spane ,January 27-29th 1993)

Please find forward a abstract of this paper,and an example of C-HELP program.
I will be very pleased to receive your collecting results.

bye,

    Dominique.


%------------------------------------------------------------------------------
Title:         Help for parallel scientific programming

Authors:       Akram BENALIA, Jean-Luc DEKEYSER,
               Dominique LAZURE, and Philippe MARQUET

Institution:   Laboratoire d'Informatique Fondamentale de Lille
               Universit\'e de Lille, France

Email:            {benalia,dekeyser,lazure,marquet}@lifl.fr

Address:       Laboratoire d'Informatique Fondamentale de Lille
               Universit\'e des Sciences et Technologies de  Lille
               B\^atiment M3, Cit\'e Scientifique,
               59655 Villeneuve d'Ascq cedex,
               France.

Abstract:

Massively parallel machines emergence leads to the definition of a new
programming model already stated with vector machines: the data
parallelism. This programming model standardizes all the parallel
machine programming, whatever they are vector machine (Cray Y-MP),
massively parallel SIMD (Connection Machine CM-200 or MasPar MP-1) or
MIMD (Connection Machine CM-5 or Intel Paragon). In this third
approach, only the data parallelism is able to manage a great number
of processors.

Data parallel language designers specially pay attention to the data
mapping onto elementary processors and/or data alignment, and
communication expression.

The HELP data parallel programming model is defined as a new original
massively parallel machine programming environment. On the opposite of
the current approaches consisting of adding extra features to handle
the data-parallel paradigm, HELP proposes a two complementary
programming level to express data-parallel processing and
data-parallel object migration.

HELP model is based on hyper-space notion. An HELP program defines one
or several hyper-spaces. The programmer-handled data parallel objects (DPO)
are placed in those hyper-spaces. In the HELP model, applications of
operators are clearly distinguished from data transfers because of the
two levels of programming: calculating operators on objects are
executed along a subset of hyper-space points, while communications
are realized in intra hyper-space DPO migration forms.

An hyper-space is defined as a cartesian reference of positive
coordinate points. A virtual processor (VP) is associated to each
point. Whereas informations about data distribution are linked to the
hyper-space axes, each hyper-space groups the DPO using the same
distribution.

The DPO are multi-dimensional arrays allocated on compact sets of
hyper-space points. Two DPO allocated on the same hyper-space point set
are said to be ``conforms''. The microscopic level allows the programming
of arithmetic and logical operators; they will be executed concurrently
on the DPO element values. An operator is processed on each point: it
does not generate any communications.

The macroscopic view allows to obtain DPO conformity. Migration
primitives are apllied to a complete DPO.  At this level, the
programmer indivisibly handles objects in his hyper-space.

Different macroscopic operation classes coexist:
1- Geometrical data parallel object migration in the hyper-space.
2- Data parallel object reshape: expand...
3- Data parallel object duplicate and extract: produces a copy of a DPO
or a sub-DPO.
4- Data parallel object shake: the elements of the DPO are interchanged.

The HELP model relies on a graphical resolution of data-parallel
algorithms. A workstation environment is suited to interactively
specify data migration via graphical tools, especially for 2 and 3-D
data parallel object. A prototype, HelpDraw, has been defined over the
X window environment. The poster will present differents programs
developed on the HelpDraw environment.

This project is supported by Digital Equipment Corporation in the
frame of a Data-Parallel Research Initiative.

%------------------------------------------------------------------------------


/************************************************************************/
/*    Square matrix inversion with GAUSS-JORDAN algorithm        */
/************************************************************************/
/*     ______________________________________________\ x        */
/*    |     _________________ _________________   /        */
/*    |    |          |1                |            */
/*    |    |          |  1              |            */
/*    |    |          |    1            |            */
/*    |    |          |      1     0    |            */
/*    |    |    (A)      |        1        |            */
/*    |    |          |          1        |            */
/*    |    |          |    0       1    |            */
/*    |    |          |             1  |            */
/*    |    |_________________|________________1|            */
/*   y \ /                                */
/*    v                                */
/************************************************************************/

#define N    100
hspace planar x = 2*N , y = N ;

DPO float[N,*] at planar(1,1) A;
DPO float[N,*] at planar(N,1) A_INV;

#define row(i)            extract(y,i)
#define col(i)            extract(x,i)
#define diag(i)            extract(x,i).extract(y,i).scalar

void gauss_jordan(){
   int i ;
   DPO float[*,*] at planar(1,1) M;

   /********************* Initialization *************************************/
   M = 0;
   where( ipoint(x)-N == ipoint(y)-1 )
      M = 1;
   on(A) M = A ;

   /********************* Diagonalization ************************************/
   for (i=1;i<N;i++)
      where ( ipoint(y) <> i )
     M -= M.row(i).expand(y) * M.col(i).expand(x) / M.diag(i) ;

   /********************* Compute result on X ********************************/
   on(A_INV)
      A_INV = M / M.dup.shift(x,N+1-ipoint(y)).col(N).expand(x,N-1);
}
%------------------------------------------------------------------------------

********************************************************************************

From: [e-mail]

Scott,
    Try ordinary C++. All of the following applications  were
    (or are being) written with parallel computers in mind.
    The all use C++ as is without extensions.

    BTW: I will be giving a tutorial at Supercomputing92 on
    this stuff.  You might be interested.

/ian


@INPROCEEDINGS{dscfd,
    AUTHOR = "I. G. Angus and W. T. Thompkins",
    TITLE = {{Data Storage, Concurrency, and Portability: An Object
        Oriented Approach to Fluid Dynamics}.},
    BOOKTITLE = {Fourth Conference on Hypercubes, Concurrent
        Computers, and Applications},
    YEAR = 1989
    }

@INPROCEEDINGS{ftoc++,
    AUTHOR = "I. G. Angus and Janice L. Stolzy",
    TITLE = {{Experiences Converting an Application from Fortran to
        C++: Beyond f2c}.},
    BOOKTITLE = {C++ at Work Conference},
    MONTH = "November",
    YEAR = 1991
    }

@INPROCEEDINGS{image_a,
    AUTHOR = "I. G. Angus",
    TITLE = {{Image Algebra: An Object Oriented Approach to Transparently
            Concurrent Image Processing}.},
    BOOKTITLE = {Scalable High Performance Computing Conference},
    YEAR = 1992
    }

@INPROCEEDINGS{cfdbc,
    AUTHOR = "I. G. Angus",
    TITLE = {{An Object Oriented Approach to Boundary Conditions
        in Finite Difference Fluid Dynamics Codes}.},
    BOOKTITLE = {Scalable High Performance Computing Conference},
    YEAR = 1992
    }

@INPROCEEDINGS{dwf1,
    AUTHOR = "D. W. Forslund and others",
    TITLE = {{Experiences in Writing a Distributed Particle Simulation
        Code in C++}.},
    BOOKTITLE = {USENIX C++ Conference},
    YEAR = 1990
    }

@INPROCEEDINGS{dwf2,
    AUTHOR = "David Forslund and others",
    TITLE = {{A Distributed Particle Simulation Code in C++}},
    BOOKTITLE = "Computing in Civil Engineering",
    YEAR = 1992,
    MONTH = "June",
    }

@INPROCEEDINGS{budge_uc++,
    AUTHOR = "K. G. Budge, J. S. Peery, A. C. Robinson",
    TITLE = {{High Performance Scientific Computing Using C++}.},
    BOOKTITLE = {USENIX C++ Conference},
    YEAR = 1992
    }

@INPROCEEDINGS{acr_asce,
    AUTHOR = "Allen Robinson and others",
    TITLE = {{Massively Parallel Computing, C++ and Hydrocode Algorithms}},
    BOOKTITLE = "Computing in Civil Engineering",
    YEAR = 1992,
    MONTH = "June",
    }

@INPROCEEDINGS{rhale1,
    AUTHOR = "James Peery and Kent Budge",
    TITLE = {{Experiences in Using C++ to Develop a Next Generation
        Strong Shock Physics Code}},
    BOOKTITLE = "Computing in Civil Engineering",
    YEAR = 1992,
    MONTH = "June",
    }

@INPROCEEDINGS{paragon,
    AUTHOR = "C. M. Chase and others",
    TITLE = {{Paragon: A Parallel Programming Environment for Scientific
        Computing Using Communications Structures}.},
    BOOKTITLE = "International Conference on Parallel Processing",
    YEAR = 1991,
    }

********************************************************************************

From: [e-mail] (Robert Bernecky)

You might want to look at APL and J.

References include (somewhat outdated):

a. ACM SIGAPL Quote Quad Conference Proceedings for 1990, 1991, 1992.

b. IBM Journal of Research and Development. (Most recent issue):
    "25th Anniversary of APL"

Bob

********************************************************************************

From: Roland Ruehl <[e-mail]>

We have developed the parallelizing Fortran compiler Oxygen, which
substantially differs from Fortran D / HPF in its run-time support. There was
a paper on performance of compiler generated parallel code on various
platforms in ICS 92 (Washington).

---------------------------------------------------------------------------

Roland Ruehl                                     Phone:     [phone]
Integrated Systems Laboratory                    Telex:     53 178 ethbi ch
ETH-Zuerich                                    Telefax:     [phone]
Switzerland                                     E-mail:   [e-mail]

********************************************************************************

I don't mean to offend, but the list from your comp.parallel
posting:


        Fortran-D (and variants)
        HPF
        DINO
        P++
        Parti and Block-Structured Parti
        pC++
        GenMP
        LPAR
        KALI
        C*

consists of systems that are rather obscure or still
under development.  I have organized a workshop with
Cherri Pancake where we will present what I like to
call "mainstream systems" for parallel computing.  These
are systems that are robust, supported, and heavilly used
outside of the groups that created them.  These are issues
of top priority for scientific programmers who are more
concerned with the nature of the application than with
the computer science (which may not apply to you).

None of these mainstream systems appear in your list.  They
are:

   Express
   PVM
   Linda

These are by far the most commonly used tools for parallel
scientific computing.   If you want more invormation on
these tools, contact their developers.  I don't have email
address for the other two, but you cant get Linda information
mailed to you if you contact Sudy B. at

   [e-mail]

By the way, some other very important tools are:

   SISAL
   PCN

They are quite different than the ones you list above in that
they are nonimperitive languages designed to support concurrent
processing.  They are well worth checking out though they will
never see heavy use (in my not-so-humble opinion) because they
are so different from C or Fortran.

--Tim

********************************************************************************

From: [e-mail] (Stephen W Poole)

About a year ago a CS student from Canada (Ontario ?) wrote part of
a class on this subject. I will try and remember and find out the
paper. It was available via the net. You might want to look for grins at
some of the work done at IBM , also in the early days of HPFF there
was a raft of papers published on Data Parallel Fortran. You might also
look at SISAL and Pandore'.

Steve...

********************************************************************************

From: "Neelakantan Sundaresan" <[e-mail]>

Hi,
I would be very much interested in the summary that you make.

By the way, pC++ is from Indiana university and references include
1. Lee and Gannon - Object-Oriented Parallel Programming-Supercomputing '91
2. Gannon and Lee - On Using Object Oriented Parallel Prog to build
Distributed Algebraic Abstractions, CONPAR '92.
3. Gannon and Lee - Object oriented parallelism: pC++ ideas and
experiments, Proceedings of joint symposium on parallel processing '91,  Japan.
thanks
Neel

********************************************************************************

From: [e-mail] (Kevin McCurley)

I was amused to see that the two most commonly used models were not on
your list, namely CM-Fortran (not used much here), and message passing
in either C or Fortran-77.

Kevin McCurley
Sandia National Laboratories

********************************************************************************

From: Steven Zenith <[e-mail]>

You have missed out EASE, SR, Orca, Occam at least.

--
Steven Ericsson Zenith
Disclaimer: Opinions expressed are my own and not necessarily those of KAI.

********************************************************************************

From: [e-mail] (Thomas Willis)

F77 (with parallel extentions)
ansi c (with parallel extentions)
ada (with parallel extentions)
parallel prolog
dataparallel C

I think a case could be made for each as a scientific programming
language...

--
-----------------------------------------------------------------------------
o-----o     T.J. Willis
|\    |\                                                   Pressure(t)
| o---+-o   [e-mail]              lim    -------------
o-+---o |   Research Assistant                     t->oo    Grace(t)
 \|    \|   Syracuse Center for Computational Science
  o-----o   Northeast Parallel Architectures Center          << Disclaimer.m
-----------------------------------------------------------------------------

********************************************************************************

From: [e-mail] (Craig Chase)

Yes, I have one to add, the "Paragon" programming environment developed
at Cornell (not to be confused with the Paragon computer marketed by
Intel).

Currently, the best reference on the Paragon environment is the ICPP-91

Here's my bibtex entry

@Inproceedings{CCRS91,
        Author = {Craig Chase and Alex Cheung and Anthony Reeves and
                Mark Smith},
        Title = {Programming For Scientific Computation Using Communication
                Structures},
        booktitle = {International Conference on Parallel Processing},
        year = 1991
}

Paragon is available via anonymous ftp from tesla.ee.cornell.edu in
pub/paragon.  It is a programming environment based on a data-parallel
programming paradigm augmented by programming constructs to better support
multicomputers.  Paragon is written in C++ and requires a C++ compiler
The AT&T C++-2.x and g++-1.XX compilers are known to work.

for more information please contact [e-mail]

Craig Chase

--
"They [La Prensa] accused us of suppressing freedom of
 expression.  This was a lie and we could not let them
 publish it."
 - Nelba Blandon, Interior Ministry Director of Censorship, quoted in
   The New York Times, 1984

********************************************************************************

From: "Christopher T. Faigle" <[e-mail]>

Scott,
    Hi.  One parallel programming model that is intended for scientific
computation, but is not on your list is the MOVIE project, from the Northeast
Parallel Architectures Center at Syracuse University.  This project is
based on the idea of in interpreted language (MovieScript) that can be passed
around MOVIE servers on a network of workstations or massively parallel machines.

    The MovieScript language is based on PostScript, but is extended to
include Fortran-90 style index free matrix algebra, among other things, and
is completely object oriented.  The server is extensible, and has a set of
CASE tools called MetaShell which ease server extension.  The interpreter uses
multi-threading with pre-emptive scheduling to achieve real-time properties.

    Future projects may include a Fortran 90->mps translator, so that
Fortran-90 programs can be dynamically load balanced, etc.

    If you want, I can send you the paper we (the MOVIE group) presented
at the HPDC-1 conference.  Please, however, do not redistribute the paper.
Although I am listed as the first author, the project has really been
developed by Dr. Wojtek Furmanski (of the CalTech Concurrent Compter Project);
the names are simply listed alphabetically.

    Further, the code is not at release stage yet, and I am not sure what
format a release will take.  The code currently runs in various forms on
DecStations, Suns, and our nCube.  Ports to other toys lying around here include:
CM-5, DecMpp 12000, IBM RS-6000s and SGIs.

Regards,
Chris Faigle
Northeast Parallel Architectures Center
Syracuse Center for Computational Science
Syracuse University
[e-mail]

********************************************************************************

From: [e-mail] (Maneesh Dhagat)

Hi Scott, the following language has a different programming
model from the ones you listed (as far as I can tell).  Like
FortranD, Dino, Kali, it also has data distribution primitives.
Let me know how you would compare it to the others, and could
you please send me a copy of your summary.

                  Thanks,
                            Maneesh Dhagat (CS dept, UCLA)

------------------
@INPROCEEDINGS{Bagrodia:UC,
    AUTHOR = "Rajive L. Bagrodia and K. Mani Chandy and Edmund Kwan",
    TITLE = "{UC}:  A Language for the {Connection} {Machine}",
    BOOKTITLE = "Proceedings of Supercomputing '90",
    PAGES = "525--534",
    ORGANIZATION = "IEEE Computer Society and ACM SIGARCH",
    PUBLISHER = "IEEE Computer Society Press",
    ADDRESS = "New York, NY",
    MONTH = Nov,
    YEAR = 1990,
    DATEREAD = "7-17-91",
    HAVECOPY = "yes",
    BIBENTERED = "yes"
    }
@ARTICLE{Bagrodia:efficient-implementation,
    AUTHOR = "Rajive L. Bagrodia and Sharad Mathur",
    TITLE = "Efficient Implementation of High-Level Parallel Programs",
    JOURNAL = sigplan,
    NOTE = "In Proceedings of the 4th International Conference on Architectural Support for Programming Languages",
    PAGES = "142--153",
    VOLUME = 26,
    NUMBER = 4,
    MONTH = Apr,
    YEAR = 1991,
    MYLIB = "yes",
    DATEREAD = "7-18-91",
    PRIVATENOTE = "Mapping transformations in UC",
    HAVECOPY = "yes",
    REFERENCEDFROM = "Bagrodia:UC"
    }

********************************************************************************

From: [e-mail]

Scott,

You seem to have essentially a list of programming languages here.
If you wish to distinguish Fortran-D from HPF, then I would strongly
suggest that Vienna Fortran, developed by Hans Zima and colleagues at
Vienna, should be included too.  HPF can be viewed as building on both
Fortran-D and Vienna Fortran.  I can put you in touch with Hans if you
have difficulty finding references or pointers, but Vienna Fortran is
fairly heavily referenced in the compiler and language literature.

I also note that you do not cite any message-passing interfaces in your
list of programming models.  I believe that the vast majority of scientific
applications running on MIMD machines are written in this manner.  Examples
of interfaces would be ORNL's PVM, GMD's PARMAC's and EPCC's CHIMP.

There has also been quite a lot of work done to support particular
restricted classes of scientific computations on parallel
machines.  Examples of this sort of tool which have been developed
at EPCC include:

CAPE --- A Cellular Automata Programming Environment, which ran on a
large transputer-based machine;

PUL-RD (A library to support regular domain decomposition of grid-based
problems on parallel machines, which runs on top of our CHIMP message-passing
interface on Sun, SGI and IBM Unix workstations, T800 and i860-based
Meiko Computing Surfaces, PC-hosted transputers, and Fujitsu's AP1000, with
ports to the CM5 and Intel kit planned.  Other PUL (Parallel Utility Library)
projects include: PUL-EM for extended message passing functionality such as
parallel prefix operations; PUL-TF for task farming; PUL-GF for parallel
file access from applications.  The majority of these utilities offer the
programmer both a skeletal (ie template-based) interface or a procedural
(ie library) interface.

Recent work in the last few months has led to the implementation of a
library for handling (irregular) mesh-based problems on (distributed memory)
parallel machines in a transparent fashion.  A library for this purpose was
implemented by a student in our Summer Scholarship Programme this year.

I would be happy to supply you with further information on any of the above,
or about EPCC and its other activities.  I hope you find this of interest.

Neil MacDonald
Edinburgh Parallel Computing Centre
The University of Edinburgh

********************************************************************************

From: [e-mail] (Jean Louis Pazat)

PANDORE II is a  C based langage with data distribution annotations.
The Pandore language is based on a sequential imperative language. The
design of powerful compile-time techniques is the main purpose of our
study, so the input language has been restricted to a subset of C,
without pointers and side-effect functions. Indeed the parallelization
rules in the compiler would be of no use for such dynamic aspects of
the C language.  With similar restrictions it is possible to define a
subset of FORTRAN, so that our technique could easily be applied to
this language.

REFS:

@InProceedings{Pandore91,
  author =       "Fran{\c c}oise Andr\'{e} and Olivier Ch\'{e}ron and
                  Jean-Louis Pazat and Henry Thomas",
  title =        "Efficient Code Generation for Distributed Memory Machines",
  booktitle =    "Parallel Computing '91",
  year =         "1992",
  organization = "Parallel Computing Society",
  publisher =    "Elsevier Science Publishers B.V.",
  month =        "September"
}

%------------------------------- 1992 ------------------------------------
@TechReport{Pandore92,
  author =      "Fran{\c c}oise Andr\'{e} and Olivier Ch\'{e}ron and
                 Jean-Louis Pazat",
  title =       "Compiling Sequential Programs for Distributed Memory
                 Parallel Computers with Pandore II",
  institution = "IRISA/INRIA",
  year =        "1992",
  number =      "651",
  month =       "April"
}

Jean-Louis Pazat.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
FORTRAN "the infantile disorder", by now 20 years old, is  hoplessly
inadequate  for  whatever  computer  application  you have in mind today:
it is now too clumsy, too risky, and too expensive to use. (E.W.Dijkstra)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jean-Louis PAZAT, IRISA 35042 RENNES CEDEX FRANCE        ([e-mail])
Phone  [phone]    |    Fax [phone]    |    Telex UNIRISA 950 473F
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

********************************************************************************

From: Monika Haerdtner <[e-mail]>

Dear Scott,

We are developing a programming environment called GRIDS. It is based
on its own grid based computation model. Find enclosed an abstract of
the GRIDS idea. If you are interested in more technical information,
we can send you the project specification. It will be released for
distribution probably end of november.

Regards
Monika

------------------------------------------------------------------------

GRIDS - An Environment for Parallel Execution of Grid Based Applications

Currently the situation in scientific and engineering computing can be
described as follows:

1. Numerically intensive applications are preferably executed on
mainframes or supercomputers, all of them with at most low parallelism
and shared memory.

2. The currently usable compute power does not meet the needs of
scientific and engineering applications.

The tempting solution to exploit parallel systems to meet the
performance needs of scientific applications currently fails due to
complexity of explicit parallel programming. The wide success of
parallel computing systems depends on the development of parallel
problem solvers, which are usable without parallel programming but
exploit the known parallelism inherent in the problem.

GRIDS is a specialized solution for the application field of
grid-based algorithms. Irregular grids are supported. GRIDS offers a
simple computing model to the user, which allows the use of grid
constructs. The technical difficulties of parallel execution of grid
methods are implemented in the GRIDS system and hidden from the user.

********************************************************************************

From: [e-mail] (Maneesh Dhagat)

...is that of Linda.  Here's the reference:

Gelernter, David.  Generative Communication in Linda
ACM Transactions on Programming Languages and Systems,
vol.7,no.1,Jan.'85

Again, when you are ready, could you send me a copy
of your references.

        --Maneesh Dhagat (CS dept, UCLA)

********************************************************************************

From: [e-mail] (Kalhan Ajay)

Some more:
1. Chare Kernel System - Kale et al, UIUC.
2. Chores - Eager, Zahorjan, Lazowska UofWashington

More later (maybe)
/ajay
email: [e-mail]

********************************************************************************