AMDC: Using AMDC in the CSAM Department at IIT

AMDC is a parallel programming paradigm based on active messages. In the Computer Science and Applied Mathematics Department at IIT, AMDC is available on the cluster of SGI workstations known as the deltas cluster: the implementation is done on top of PVM, though the user does not have to know PVM programming to be able to use AMDC.

This page contains information about the MDC setup for users of the Deltas Cluster.

Changes to your environment

Here are the steps you have to follow to be ready to run AMDC applications on the deltas (for csh users). Most of these changes are due to the fact that the implementation runs on top of PVM.

  • Add these lines to your .cshrc file: 
     setenv PVM_ROOT /a/apps/pvm3 
     setenv PVM_ARCH SGI5 
     set path=($path $PVM_ROOT/lib/SGI5)
  • Make sure you don't have anything in .cshrc that will attempt to do I/O (read from the keyboard or write something on the console), since this will interfere with the PVM start-up process. If, for instance, you have a line that reads 
     news
then move this into the .login file since `news' will attempt to print a message.
  • Create a file called .rhosts in your home directory with the name(s) of the host(s) from which you will start pvm: if you plan to start pvm from delta0, for instance, then the file .rhosts will contain: 
     delta0
If you plan to be able to start PVM from more than one host, then add the name of each of them in the .rhosts file, one per line.
  • Set proper permissions for the .rhosts file: 
     chmod go-rwx .rhosts
  • Create the directory where you plan to do the AMDC related work (you may choose any name you want): 
     cd
     mkdir  amdc-work
  • In the directory where you do your AMDC work (amdc-work in the above example) create a host file (the name is up to you, but let's say it is myhostfile) which lists all the hosts on which the PVM daemon will be started: 
     delta0   ep=$HOME/amdc-work   wd=$HOME/amdc-work 
     delta1   ep=$HOME/amdc-work   wd=$HOME/amdc-work 
     delta2   ep=$HOME/amdc-work   wd=$HOME/amdc-work
The above host file assumes you'll be using up to three hosts, with their names listed at the beginning of each line. If you start the PVM daemons using pvmd3, then a daemon will be started on each host listed in myhostfile.

Creating an AMDC executable

The process of creating an AMDC executable is as simple as creating any executable starting from a C file. The only exception is that the object file must be linked with the AMDC library.

  • You must be in the directory where you do AMDC work: 
       cd ~/amdc-work
  • Compile your application making sure that:
    • the AMDC include files are found (-I/a/apps/AMDC/src)
    • the AMDC library is searched before anything else (-L/a/apps/AMDC/lib/pvm_sgi)
    • the AMDC library is used (-lmdc)
    • the PVM directories are searched before standard directories ( -L/a/apps/pvm3/lib/SGI5 )
    • the PVM library is available for linking ( -lpvm3 ) 
   cc myfile.c -I/a/apps/AMDC/src  
     -L/a/apps/AMDC/lib/pvm_sgi -lamdc  \
     -L/a/apps/pvm3/lib/SGI5 -lpvm3 -o myfile
  • The executable is myfile. Before you can run it you will need to start the PVM daemon on the machines where you want to have the program execute
  • A makefile which builds an executable from an AMDC source file can be downloaded. Save it with the name makefile. Edit the file makefile and add the name of the executable to build on the line that starts with TESTS
  • Build the executable: 
    make myfile

Run the AMDC executable

  • You must be in the directory where you do AMDC work: 
       cd ~/amdc-work
  • Start the PVM daemon on all machines listed in myhostfile 
     pvmd3 myhostfile &
Remember that you have to be logged on the machine(s) you have listed in the ~/.rhosts file.
  • Type the name of the executable at your UNIX prompt: 
       myfile -n2
This will run myfile on two nodes (a node is a workstation in this case). Part of the output will go to your terminal, the rest in the file /tmp/pvml.my_user_id where my_user_id is what the computer returns when you type id at the command prompt.
The file /tmp/pvml.my_user_id will be created on the system where the PVM daemon was started.

An example

  • Login on some system on the deltas cluster. Let's assume this is delta0.
  • Change your working directory to the directory where you do your AMDC work 
     cd ~/amdc-work

     make t1
  • Start the PVM daemon, on the hosts listed in your hostfile (let's assume it is called myhostfile). The daemon has to be started only once, and it will keep running even if you log out, unless you specifically stop it. 
     pvmd3 myhostfile &
  • Run the example using a single node 
     t1
  • Run the example using three nodes 
       t1 -n3
  • Find your user ID 
       id
  • Let's call the first number the previous command returns my_user_id, You can now see where the rest of the output went when you did t1 -n3 
    cat /tmp/pvml.my_user_id
  • You can stop the PVM daemons if you want to: 
    %    pvm
pvm> halt

PVM help

  • For online PVM help try: 
     man -M /a/apps/pvm3/man pvm_intro
Some PVM documentation is available in /a/apps/pvm3/doc/

Frequently Asked Questions

Q: I use the makefile to compile a test (like primes.c) but I keep getting errors like the one below. Why:

Error: primes.c, line 45: Type int of rhs 
       of assignment expression is 
       incompatible with type struct of lhs
    LN = amdc_getNameOfLoc(L);
 ------^

A: Most likely you have old amdc header files in your directory. The only files that you must have in your working directory are your amdc program (with the .c extension), the makefile and the pvm hostfile (called myhostfile in the section that describes how to setup your environment).

Q: How do I know how many nodes (workstations are available to my computation)?

A: The nodes used in computation will be those listed in the pvm hostfile ( called myhostfile in the section that describes how to setup your environment), unless you decide to run your application with fewer nodes. When you type

    pvmd3 myhostfile &

the pvm daemon will be started on all nodes that are listed in myhostfile. Let's say there are 4 nodes listed in myhostfile. You can run the executable t1 using all nodes

    t1 -n4

or only three of the nodes

    t1 -n3

To see what computers are available type pvm on the machine where you run pvmd3; then type conf to see a list of active computers.

Q: I use pvmd3 to start the pvm daemons but I never get to have more than one host running, why?

A: When pvm starts it creates two files in the /tmp directory, pvmd.my_user_id and pvml.my_user_id. The former is a lock indicating pvm is running, thus preventing the user to start another daemon on the same machine. When pvm exits gracefully it removes the pvmd.my_user_id file. If for some reason the daemon dies without removing the file, then pvm won't start on that machine. To fix the problem you have to login on the machine(s) where pvm does not start and manually remove the file.
AMDC & Web Content Created by
Thomas W. Christopher, George K. Thiruvathukal
&
Virgil Bistriceanu

Web Site Designed and Created by
Lance Larsen & Emad Shawakfa