MPI_Send does not work with higher buffer size?

Question

When MPI_Send buffer size is 100 program works, but it stucks when it is 1000 or greater. Why?

if(id == 0){
    rgb_image = stbi_load(argv[1], &width, &height, &bpp, CHANNEL_NUM);

    for(int i = 0; i < size -1; i++)
        MPI_Send(rgb_image,1000,MPI_UINT8_T,i,0,MPI_COMM_WORLD);
        
}
uint8_t *part = (uint8_t*) malloc(sizeof(uint8_t)*(1000));

if(id != size-1 && size > 1)
    MPI_Recv(part,1000,MPI_UINT8_T,0,0,MPI_COMM_WORLD,MPI_STATUS_IGNORE); 

Answer 1

This program is not valid w.r.t. MPI Standard since there is no matching receive (on rank 0) for

MPI_Send(..., dest=0, ...)

MPI_Send() is allowed to block until a matching receive is posted (and that generally happens when the message is "large") ... and the required matching receive never gets posted.

A typical fix would be to issue a MPI_Irecv(...,src = 0,...) on rank 0 before the MPI_Send() (and MPI_Wait() after), or to handle 0 -> 0 communication with MPI_Sendrecv().

That being said, it would likely more efficient to create a communicator will all the ranks minus the last one, and MPI_Bcast() in this communicator.

Answer 2

If a program works for small buffers but not for large, you are probably running into "eager sends". Normally, a send & receive transaction involves the sender & receiver talking back and forth, confirming that the data went across. This is overhead, so for small messages, many MPIs will just send the data, without confirmation. The data then goes into some secret buffer on the receiver.

But this means that your program will "succeed" if it's not a correct program. As is the case here. See @Giles answer.