A typical TOP-C application has the following structure:
#include <topc.h>
... define four callback functions for TOPC_master_slave() ...
int main( int argc, char **argv ) {
TOPC_init( &argc, &argv );
...
TOPC_master_slave( generate_task_input, do_task, check_task_result,
update_shared_data )
...
TOPC_finalize();
}
Every `TOP-C' application must include a `topc.h' header, open
with TOPC_init(), call TOPC_master_slave() one or more
times, and then close with TOPC_finalize().
#include <topc.h>
Required at head of any file using TOPC library calls.
TOPC_master_slave();
Recommended to place this as first executable statement in
main(). It will strip off extra `TOP-C' arguments
such as --TOPC_verbose, which are added by `TOP-C'.
See command line options for extended syntax.
TOPC_master_slave(), each invoking different callback functions,
between TOPC_init()
and TOPC_finalize().
A task input or task output is simply a buffer of bytes, specified by
TOPC_MSG().
GenerateTaskInput() and DoTask().
Specifies arbitrary user data structure. In DoTask(),
ptr must not reference a buffer on stack; Declare buffer static
to avoid this. (This requirement is relaxed in the experimental
version.) `TOP-C' will copy data to `TOP-C' space.
Application can free it later via a static local variable in
GenerateTaskInput()/DoTask().
static void *mydata = NULL;
if ( mydata != NULL ) { free(mydata); mydata = NULL; }
EXAMPLE:
TOPC_BUF convert_string_to_msg( char *mystring ) {
if (mystring == NULL) return TOPC_MSG(NULL,0);
else return TOPC_MSG(mystring, strlen(mystring)+1);
}
TOPC_master_slave()
The application writer must defined the following four callback
functions (although the last can be NULL).
TOPC_MSG(ptr, size).
It should return NOTASK, when there are no more tasks, and it
should continue to return NOTASK if invoked again.
GenerateTaskInput();
returns another arbitrary user data structure, RESULT.
RESULT must
be a static or global user buffer specified by
TOPC_MSG(ptr, size).
DoTask();
returns an ACTION that determines what happens to the task next.
An easy way to write CheckTaskResult() appears in the
example for the utility TOPC_is_up_to_date().
See the section section TOP-C Utilities, for more details.
DoTask(), and the original task returned by
GenerateTaskInput();
called only if CheckTaskResult() returned UPDATE;
useful for updating non-shared, global variables in all processes;
The pointer argument, update_shared_data, may be NULL if
an application never requests an UPDATE action.
In a shared memory environment, only the master calls
UpdateSharedData(). See the section section Modifying TOP-C Code for the Shared Memory Model, for more details.
CheckTaskResult()
The actions returned by CheckTaskResult() are:
UpdateSharedData( void *task) (see below)
also updates bookkeeping for sake of TOPC_is_up_to_date() (see below)
DoTask() on original task input again, and on
original slave; useful if shared data has changed since original
invocation of DoTask(); see TOPC_is_up_to_date(), below
CONTINUATION() is a parametrized action that may be returned.
It's like REDO, but if the result of
CONTINUATION( next_input ) is returned by
CheckTaskResult(),
then DoTask( next_input ) is called on the original slave.
useful if only the master can decide whether task is complete.
Note that any pending calls to UpdateSharedData() will have
occurred on the slave before the new call to DoTask().
Hence, this allows an extended conversation between master and
slave, in which the slave continues to receive updates of the
shared data before each new input from the master.
`TOP-C' also defines some utilities.
CheckTaskResult()
has not returned the result UPDATE (invoking
UpdateSharedData())
between the time when GenerateTaskInput() was originally
called on the
current task, and the time when the corresponding
CheckTaskResult()
was called.
Typical usage:
TOPC_ACTION CheckTaskResult( void *input, void *output )
{ if (input == NULL) return NO_ACTION;
else if (! TOPC_is_up_to_date()) return NO_ACTION;
else return REDO;
}
TOPC_rank() == 0.
/* NOT CURRENTLY SUPPORTED */
returns unique id, as a C int, for the last process from
which a message was received. This is trivial, when called on
a slave, but it is useful on the master.
TOPC_MSG(): Memory Allocation of TOP-C Message Buffers
Recall the syntax for creating a message buffer of type TOPC_BUF:
TOPC_MSG(ptr, size). The two callback functions
GenerateTaskInput() and DoTask() both return such a
message buffer. In both cases, the callback functions block until the
corresponding message has been sent. In the case of
GenerateTaskInput(), `TOP-C' saves a copy of the buffer,
which becomes an input argument to CheckTaskResult().
Hence, if ptr points to a temporarily allocated buffer,
it is the responsibility of the `TOP-C' callback function to free the
buffer only after the callback function has returned.
This seeming contradiction can be easily handled by the following code.
TOPC_BUF GenerateTaskInput() {
static char *input_task_buf = NULL;
... compute buf_size, size of input task buffer ...
if ( input_task_buf != NULL ) free(input_task_buf);
malloc( input_task_buf, buf_size );
... write input task data into input_task_buf ...
return TOPC_MSG( input_task_buf, buf_sze );
}
Note that input_task_buf is allocated as a static local
variable. Currently, `TOP-C' restricts the ptr of
TOPC_MSG(ptr, size) to point to a buffer that is in
the heap (not on the stack). Hence, ptr must not point to
non-static local data. This restriction may be alleviated in a future
version of `TOP-C'.
IMPORTANT:
`src/comm-mpi.c' sets alarm() before waiting to receive
message from master.
If master does not reply in one hour, slave receives SIGALRM and dies.
This is to prevent runaway processes in dist. memory version when master dies
without killing all slaves. If you don't like the one hour default,
modify the source in `src/comm-mpi.c'.
GenerateTaskInput() and DoTask()
This memory is managed by `TOP-C'.
The slave process attempts to set current directory to same as master
inside TOPC_init() and produces a warning if unsuccessful.
/* Experimental version only: */
void TOPC_init(int *argc, char ***argv, TOPC_OPTION(string,val), ...);
string may be one of: "trace", "trace_input", "trace_output",
"trace_action", "num_threads", "agglom_count".
"statistics" is another intended option
The final set of options is still under development, and
we hope to also pass them in either from a command line
or from an environment variable. For now,
each option is available as a C variable:
e.g.: TOPC_trace = 0;
turns off all tracing.
/* EXPERIMENTAL: not currently supported */
declare_recv_buf(char *buf, int size) - useful only for distributed memory.
If message size of your application is too large, error message will
ask you to declare your own receive buffer on master and/or slave.
The `TOP-C' programmer's model changes slightly for shared
memory. With careful design, one can use the same application source
code both for distributed memory and shared memory architectures.
Processes are replaced by threads. UpdateSharedData() is
executed only by the master thread, and not by any slave thread. As
with distributed memory, TOPC_MSG() buffers are copied to
`TOP-C' space (shallow copy), and it is the responsibility of the
application to free any application buffers that it may have created.
Furthermore, since the master and slaves share memory, `TOP-C' creates the
slaves only during the first call to master_slave. If a slave needs to
initialize any private data (see TOPC_private_global, below), then this
can be done by the slave the first time that it gains control through
do_task(). Two issues arise in porting a distributed memory
`TOP-C' application to shared memory.
First, there can be consistency problems if a slave thread executes
DoTask() while the master thread executes
UpdateSharedData(). To avoid this, `TOP-C' forces slave
threads to wait when the master wishes to execute
UpdateSharedData(), and the master begins only when all currently
running threads have exited DoTask(). `TOP-C' provides
additional constructs with which to specify finer grained
synchronization. The synchronization primitives are used by
`TOP-C' in shared memory mode, and ignored in all other modes.
Second, the only variables that are thread-private by default in shared memory are those on the stack (non-static, local variables). All other variables exist as a single copy, shared by all threads. `TOP-C' provides primitives to declare a single global variable, for which each thread has a private, global copy. `TOP-C' allows the application programmer to declare the type of that variable. If more than one variable is desired, this can be emulated by declaring a struct.
By default, DoTask() and UpdateSharedData() never run
simultaneously. Optionally, one can achieve greater concurrency through
a finer level of granularity than the procedure level. This is done by
declaring to `TOP-C' which sections of code read or write shared
data.
TOPC_ATOMIC_READ(0) { ... C code ... }
TOPC_ATOMIC_WRITE(0) { ... C code ... }
The number 0 refers to page 0 of shared data. `TOP-C' currently
supports only a single common page of shared data, but future versions
will support multiple pages. If TOPC_ATOMIC_READ() and
TOPC_ATOMIC_WRITE() are ommitted, the default behavior of
`TOP-C' is equivalent to having defined DoTask() and
UpdateSharedData() as follows:
TOPC_BUF DoTask( void *input ) {
TOPC_ATOMIC_READ(0) {
... body of DoTask() ...
}
}
void UpdateSharedData( void *input, void *output ) {
TOPC_ATOMIC_WRITE(0) {
... body of UpdateSharedData() ...
}
}
Also, TOPC_BEGIN_ATOMIC_READ(0), TOPC_BEGIN_ATOMIC_WRITE(0),
TOPC_END_ATOMIC_READ() and TOPC_END_ATOMIC_WRITE() exist.
The syntax may change based on future experience with the experimental
version.
`TOP-C' provides for a single, global, thread-private variable using the primitives below.
TOPC_private_global_t. It may be
used like any C variable, and each thread has its own private
copy that will not be shared.
typedef
if TOPC_private_global is used.
If you need more than one global, thread-private variable, define
TOPC_private_global_t as a struct, and use each
field as a separate thread-private variable.
EXAMPLE:
/* The pre-defined thread-private TOP_C variable, TOPC_private_global,
* will have the type: struct {int my_rank; int rnd;}
*/
typedef struct {int my_rank; int rnd;} TOPC_private_global_t;
/* Re-define TOPC_private_global to more meaningful name for application */
#define mystruct TOPC_private_global
void foo() {
mystruct.my_rank = TOPC_rank();
mystruct.rnd = rand();
printf("Slave %d random number: %d\n", mystruct.my_rank, mystruct.rnd);
}
void bar() {
foo();
if (mystruct.my_rank != TOPC_rank()) printf("ERROR\n");
printf("Slave %d random number: %d\n", mystruct.my_rank, mystruct.rnd);
}
Note that `SMP' involves certain performance issues that do not arise
in other modes. If you find a lack of performance, please read
section Improving Performance. Also, note that the vendor-supplied
compiler, cc, is often recommended over gcc for
`SMP', due to specialized vendor-specific architectural issues.
`TOP-C' also provides a sequential memory model. That model is useful for first debugging an application in a sequential context, and then re-compiling it with one of the parallel `TOP-C' libraries for production use. The application code for the sequential library is usually both source and object compatible with the application code for a parallel library. The sequential library emulates an application with a single `TOP-C' library.
The sequential memory model emulates an application in which
DoTask() is executed in the context of the single slave
process/thread, and all other code is executed in the context of the
master process/thread. This affects the values returned by
TOPC_is_master() and TOPC_rank(). In particular,
conditional code for execution on the master will work
correctly in the sequential memory model, but the following conditional
code for execution on the slave will probably not work correctly.
int main( int argc, char *argv[] ) {
TOPC_init( &argc, &argv );
if ( TOPC_is_master() )
...; /* is executed in sequential model */
else
...; /* is never executed in sequential model */
TOPC_master_slave( ..., ..., ..., ...);
TOPC_finalize();
}
Go to the first, previous, next, last section, table of contents.