Reducing Layering Overhead in Collective Communication Operations

1. A method for message transfer, in a data processing system, between tasks in support of a collective operation, said method comprising: responsive to invoking at least one collective operation by tasks executing in an application layer, updating, by the tasks executing in the application layer, state information for said at least one collective operation in a control structure accessible to said tasks executing in the application layer, wherein the at least one collective operation comprises processing parceled out cooperatively among the tasks, and wherein the updating comprises: initiating, by an initiating task of the tasks executing, on a processor, in the application layer, a message passing request directly to a communication layer interface in a communication layer, the communication layer having a point-to-point, message passing communication protocol, the message passing request for passing at least one message to at least one targeted task of the tasks executing in the application layer; delivering, by said communication layer interface, said at least one message to said at least one targeted task executing in said application layer; triggering, in said application layer, execution of at least one user defined routine upon said delivering, or upon completion of sending, said at least one message, said execution triggered to asynchronously handle said at least one message; updating, by said at least one user defined routine, state information of said at least one targeted task for said at least one collective communication in said control structure upon arrival of said at least one message; and further updating, by said at least one user defined routine, state information of the initiating task for said at least one collective communication in said control structure upon completion of sending said at least one message.

2. The method of claim 1 in which said application layer is Message Passing Interface, and wherein the message passing request is initiated directly to the communication layer interface in the communication layer absent passage of the message passing request through an intermediary layer between the Message Passing Interface and the communication layer.

3. The method of claim 1 in which said communication layer is Low-level Application Programming Interface.

4. The method of claim 1 in which said control structure includes two bit vectors for tracking progress of the at least one collective operation, the two bit vectors being updated by the at least one user defined routine as part of the updating the state information of the at least one targeted task in said control structure and the updating the state information of the initiating task in said control structure.

5. The method of claim 4 in which a first one of said two bit vectors is used to track messages from which a task has arrived for a current barrier operation of the at least one collective operation.

6. The method of claim 4 in which a second one of said two bit vectors is used to track early arrival for a next barrier operation of the at least one collective operation.

7. The method of claim 4 in which each one of said two bit vectors is approximately log 2 N in length, as measured in bits, where N is the number of tasks involved.

8. The method of claim 4 in which said two bit vectors are cleared as part of said initiating.

9. The method of claim 1 in which said control structure includes an integer counter used to keep track of synchronization points.

10. The method of claim 1 in which said at least one user defined routine is registered with the communication layer to be executed asynchronously so as to further the progress of said at least one collective operation.

11. The method of claim 1 in which, for messages of said at least one message smaller than a threshold message size, data is cached in early arrival buffers maintained by the application layer.

12. The method of claim 11 in which said early arrival buffers are managed using tokens.

13. The method of claim 11 in which, upon message arrival earlier than the targeted task, an address of an early arrival buffer is stored in the control structure.

14. The method of claim 13 in which, said early arrived data is consumed as part of synchronization advancement.

15. The method of claim 1 in which, for a message of said at least one message greater than a threshold message size, the message is processed as a smaller message without a token.

16. The method of claim 15 in which a data address is stored in the control structure.

17. The method of claim 16 in which a control message triggers data transfer to a target.

18. The method of claim 1 wherein said application layer is Message Passing Interface, and wherein said communication layer is Low-level Application Programming Interface.

19. A computer system for message transfer between tasks in support of a collective operation, said computer system comprising: a memory; and a processor in communications with the memory, wherein the computer system is configured to perform: responsive to invoking at least one collective operation by tasks executing in an application layer, updating, by the tasks executing in the application layer, state information for said at least one collective operation in a control structure accessible to said tasks executing in the application layer, wherein the at least one collective operation comprises processing parceled out cooperatively among the tasks, and wherein the updating comprises: initiating, by an initiating task of the tasks executing in the application layer, a message passing request directly to a communication layer interface in a communication layer, the communication layer having a point-to-point, message passing communication protocol, the message passing request for passing at least one message to at least one targeted task of the tasks executing in the application layer; delivering, by said communication layer interface, said at least one message to said at least one targeted task executing in said application layer; triggering, in said application layer, execution of at least one user defined routine upon said delivering, or upon completion of sending, said at least one message, said execution triggered to asynchronously handle said at least one message; updating, by said at least one user defined routine, state information of said at least one targeted task for said at least one collective communication in said control structure upon arrival of said at least one message; and further updating, by said at least one user defined routine, state information of the initiating task for said at least one collective communication in said control structure upon completion of sending said at least one message.

20. The computer system of claim 19 , wherein said application layer is Message Passing Interface, and wherein said communication layer is Low-level Application Programming Interface.