Cuneiform (programming language)

Parallel Execution

The task applications in a Cuneiform script form a data dependency graph. This dependency graph constrains the order in which tasks can be evaluated. Apart from data dependencies tasks can be evaluated in any order, assuming tasks are always side effect-free and deterministic.

Map

Applies a task to each element in a list. Each task applications can run in parallel.

Cartesian product

Takes the Cartesian product of several lists and applies a task to each combination. Each task application can run in parallel.

Dot product

Given a pair of lists of equal sizes, each element of the first list is combined with its corresponding element in the second list. A task is applied to each combination. Each task application can run in parallel.

Aggregate

Applies a task to the list as a whole without decomposing it. Since the task is applied only once for the whole list, this skeleton leaves the parallelism potential unchanged.

Conditional

Evaluates a program branch, depending on a condition computed at runtime. This skeleton leaves the parallelism potential unchanged.

By partitioning input data and using parallelizable skeletons to process partitions the interpreter can exploit data parallelism even if the integrated tools are single-threaded. Workflows can be executed also in distributed compute environments.

Examples

A hello-world script:

deftask greet( out : person )in python *{ out = "Hello "+person }* greet( person: "Peter" "Robert" );

This script defines a task greet in Python which prepends the string "Hello " to its argument person. The task has one output variable out. Applying the task greet, binding the argument person to the two-element list "Peter" "Robert" implicitly maps the task greet to each element of the input list. The workflow result is the two-element list "Hello Peter" "Hello Robert".

Command line tools can be integrated by defining a task in Bash:

deftask samtools-view( bam( File ) : sam( File ) )in bash *{ samtools view -bS $sam > $bam }*

In this example a task samtools-view is defined. It calls the tool SAMtools, consuming an input file in SAM format and producing an output file in BAM format. If this task is applied, binding the argument sam to a list of SAM files, the task is mapped to each element of that list.