This an excellent set up from the traditional C++/pthreads methods. It gives you the ability to explicitly define processes and tasks (threads) along with handling IPC shared resources via keywords. It also gives you the ability to control the assignment of processes to phsyical CPUs. More Information
Message Passing Interface is a protocol for passing messages between processes. Since it's implemented at a lower level, it is more suitable for high performance computing. It is an excellent way to communicate between processes and processors because the various implementations of the protocol allow for flexible usage.
More information on Message Passing Interface.The OpenMP Application Program Interface (API) supports multi-platform shared-memory parallel programming in C/C++ and Fortran on all architectures, including Unix platforms and Windows NT platforms. Jointly defined by a group of major computer hardware and software vendors, OpenMP is a portable, scalable model that gives shared-memory parallel programmers a simple and flexible interface for developing parallel applications for platforms ranging from the desktop to the supercomputer.
This is an interesting project as there is one of the most active projects researched so far. The project has been around for over 10 years and has high goals and expectations. The API is implemented at the preprocessor level before compilation of your application to define the parallel aspects of the application. It's worth a look.
More information on OpenMP and the OpenMP 2.5 Specification.LAM/MPI is a implementation of MPI that allow for communication over TCP/IP networks using existing protocols for communication. The implementation allows for low overhead in terms of encapsulation of data, but does not out weight the high latency involved with packet switching networks.
There are similar projects with similar goal to LAN/MPI, including LA-MPI, FT-MPI and PACX-MPI. However after sometime, these projects formed a merger and created the OpenMPI. The OpenMPI is an effort to maintain a single MPI implementation that is open, portable and flexible. One interesting mandate of the OpenMPI project is to keep the project from forking into other projects.
More information on LAM/MPI and OpenMPIPVM (Parallel Virtual Machine) is a software package that permits a heterogeneous collection of Unix and/or Windows computers hooked together by a network to be used as a single large parallel computer. Thus large computational problems can be solved more cost effectively by using the aggregate power and memory of many computers. The software is very portable. The source, which is available free thru netlib, has been compiled on everything from laptops to CRAYs.
PVM enables users to exploit their existing computer hardware to solve much larger problems at minimal additional cost. Hundreds of sites around the world are using PVM to solve important scientific, industrial, and medical problems in addition to PVM's use as an educational tool to teach parallel programming. With tens of thousands of users, PVM has become the de facto standard for distributed computing world-wide.
More information about Parallel Virtual Machine"Cluster is a widely-used term meaning independent computers combined into a unified system through software and networking. At the most fundamental level, when two or more computers are used together to solve a problem, it is considered a cluster. Clusters are typically used for High Availability (HA) for greater reliability or High Performance Computing (HPC) to provide greater computational power than a single computer can provide.
Beowulf Clusters are scalable performance clusters based on commodity hardware, on a private system network, with open source software (Linux) infrastructure. The designer can improve performance proportionally with added machines. The commodity hardware can be any of a number of mass-market, stand-alone compute nodes as simple as two networked computers each running Linux and sharing a file system or as complex as 1024 nodes with a high-speed, low-latency network.
Class I clusters are built entirely using commodity hardware and software using standard technology such as SCSI, Ethernet, and IDE. They are typically less expensive than Class II clusters which may use specialized hardware to achieve higher performance.
Common uses are traditional technical applications such as simulations, biotechnology, and petro-clusters; financial market modeling, data mining and stream processing; and Internet servers for audio and games.
Beowulf programs are usually written using languages such as C and FORTRAN. They use message passing to achieve parallel computations. See Beowulf History for more information on the development of the Beowulf architecture."
Grid Computing is the idea of joining multiple SISD systems as one "super computer" for computation intensive tasks where each node would do computations independently from each other. This is commonly used these days in scientific computations such as Folding@home.
Apple's Mac OS X (based on Darwin/Mach, variant of Unix) has built in capabilities to create a scalable Grid with no technical knowledge. This technology is called Xgrid. It uses web services to communicate between the master node and its agents. It was designed to take advantage of the Xgrid with little to no-modification to your desktop application. Message passing between the master node and its agents are done via SOAP/Web Services over a secure channel. More information can be found here
The Open Grid Services Architecture (OGSA) is an initiative to standardize grid computing protocols to allow for more compatibility between systems and not limited to homogeneous systems (IE: Unix). The standard specifies that message passing uses Web Services/SOAP. It also specifies the architecture of the systems to allow for platform independence. More information can be found here.