Message Passing Interface (MPI) MPI is the standard of programming parallel applications using message passing. Processes run on network distributed hosts and communicate by exchanging messages. A community of communicating processes is called a Communicator. The default communicator is called MPI_COMM_WORLD. Inside a communicator, each process is attributed a "process ID", called rank.