A collective operation is a concept in parallel computing in which data is simultaneously sent to or received from many nodes.
Common examples of collective operations are gather (in which data is collected from all nodes), scatter (in which a set of data is broken up into pieces, and a different piece is sent to all nodes), and broadcast (in which the same data is sent to all nodes).
Parallel computing frameworks with collective operations
Collective operations are present in the following parallel programming frameworks:
References
Collective operation Wikipedia(Text) CC BY-SA