The single global struct
struct cell
{
double a, b, c, *aa, *bb;
} * Send_l, *Send_r;
is a bit fragile:
aa and bb are allocated as arrays of double but the subCount -1 size is not there. It is buried into the code.Send_l and Send_r are also pointers to arrays of struct cell but the count size is not there. It is also buried into the code. The single struct is global and it is also weak.
This makes hard to test, allocate or free data. I will left a C example using a bit of encapsulation and that you can adapt to your case under MPI. I will use your code and functions with a bit of OOP orientation :)
The example includes 2 programs and functions to serialize and deserialize the data. For testing, the data is written to a file by the 1st program and read back by the second one. The same printSendRecv() shows the data before and after the data is written to disk.
A Cell structure
typedef struct
{
double a;
double b;
double c;
double* aa;
double* bb;
} Cell;
The Send structure
typedef struct
{
Cell l;
Cell r;
} Send;
The Set structure
typedef struct
{
unsigned count;
unsigned subCount;
Send* send;
} Set;
So a Set has all that is needed to describe its contents.
function prototypes
Set* allocateSendRecv(size_t, size_t);
int fillSendRecv(Set*);
Set* freeSendRecv(Set*);
int printSendRecv(Set*, const char*);
Using encapsulation and a bit of RAII from C++ you can rewrite allocateSendRecv() and freeSendRecv() as constructor and destructor of the struct as:
Set* allocateSendRecv(size_t count, size_t subCount)
{
// count is the number of send buffers
// subcount is the size of the arrays inside each cell
printf(
"AllocateSendRecv(count = %llu, subCount = %llu)\n", count,
subCount);
Set* nw = (Set*)malloc(sizeof(Set));
nw->count = count;
nw->subCount = subCount;
nw->send = (Send*)malloc(count * sizeof(Send));
// now that we have Send allocate the Cell arrays
for (size_t i = 0; i < count; i++)
{
nw->send[i].l.aa =
(double*)malloc(subCount * sizeof(double));
nw->send[i].l.bb =
(double*)malloc(subCount * sizeof(double));
nw->send[i].r.aa =
(double*)malloc(subCount * sizeof(double));
nw->send[i].r.bb =
(double*)malloc(subCount * sizeof(double));
}
return nw;
}
Set* freeSendRecv(Set* set)
{
if (set == NULL) return NULL;
printf(
"\nDeallocating(count = %llu, subCount = %llu)\n",
set->count, set->subCount);
for (size_t i = 0; i < set->count; i++)
{
free(set->send[i].l.aa);
free(set->send[i].l.bb);
}
free(set->send);
free(set);
return NULL;
}
Writing this way the tst pointer is invalidated in the call to freeSendRecv(). In this case tst is allocated with count and subCount as 2 and 5 and this goes inside the Set.
fillSendRecv() uses incremental fill values to make it easy to pinpoint some eventual displacement. printSendRecv() accpets a string for an optional message. Values are printed before and after the creation of the Set.
Example: serialize and deserialize a buffer
serialize()
In order to write to disk or to transmit the data first aa and bb arrays must be expanded. The example uses v2-out x y 4 file to create and show a struct using these values and then write if to file
int main(int argc, char** argv)
{
char f_name[256] = {0};
if (argc < 3) usage();
strcpy(f_name, argv[3]);
size_t count = atoll(argv[1]);
size_t subCount = atoll(argv[2]);
Set* tst = allocateSendRecv(count,subCount);
fillSendRecv(tst);
printSendRecv(tst, "printSendRecv(): ");
to_disk(tst, f_name);
tst = freeSendRecv(tst);
return 0;
}
These functions take a Set and write to a file:
int to_disk(Set*, const char*);
int write_cell(Cell*, const size_t, FILE*);
deserialize()
Since the Set has all that is needed to recreate the Set just the file name is needed. The example uses v2-in file to read back the data from file and show it on screen
int main(int argc,char** argv)
{
char f_name[256] = {0};
if (argc < 2) usage();
strcpy(f_name, argv[1]);
Set* tst = from_disk(f_name);
printSendRecv(tst, "As read from disk: ");
tst = freeSendRecv(tst);
return 0;
}
These functions read a file and return a pointer to a Set with the data:
Set* from_disk(const char*);
int read_cell(FILE*, Cell*, const size_t);
output of an example
Here the programs are
v2-out to create a Set and write to a file in diskv2-in to read a file created by v2-out and load into a new Setdump.bin is created and Set has count = 2 and subCount = 4
PS C:\SO>
PS C:\SO> .\v2-out 2 4 dump-2-4.bin
AllocateSendRecv(count = 2, subCount = 4)
FillSendRecv()
printSendRecv(): Count is 2, subCount is 4
Set 1 of 2
l:
[a,b,c] = [ 42.001, 42.002, 42.003]
aa: 42.004 42.005 42.006 42.007
bb: 42.008 42.009 42.010 42.011
r:
[a,b,c] = [ 42.012, 42.013, 42.014]
aa: 42.015 42.016 42.017 42.018
bb: 42.019 42.020 42.021 42.022
Set 2 of 2
l:
[a,b,c] = [ 42.023, 42.024, 42.025]
aa: 42.026 42.027 42.028 42.029
bb: 42.030 42.031 42.032 42.033
r:
[a,b,c] = [ 42.034, 42.035, 42.036]
aa: 42.037 42.038 42.039 42.040
bb: 42.041 42.042 42.043 42.044
writing 'Set' to "dump-2-4.bin"
Deallocating(count = 2, subCount = 4)
PS C:\SO> .\v2-in dump-2-4.bin
read 'Set' from "dump-2-4.bin"
From disk: Count = 2, SubCount = 4
AllocateSendRecv(count = 2, subCount = 4)
new 'Set' created
As read from disk: Count is 2, subCount is 4
Set 1 of 2
l:
[a,b,c] = [ 42.001, 42.002, 42.003]
aa: 42.004 42.005 42.006 42.007
bb: 42.008 42.009 42.010 42.011
r:
[a,b,c] = [ 42.012, 42.013, 42.014]
aa: 42.015 42.016 42.017 42.018
bb: 42.019 42.020 42.021 42.022
Set 2 of 2
l:
[a,b,c] = [ 42.023, 42.024, 42.025]
aa: 42.026 42.027 42.028 42.029
bb: 42.030 42.031 42.032 42.033
r:
[a,b,c] = [ 42.034, 42.035, 42.036]
aa: 42.037 42.038 42.039 42.040
bb: 42.041 42.042 42.043 42.044
Deallocating(count = 2, subCount = 4)
The example in 2 files
a header v2.h
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
typedef struct
{
double a;
double b;
double c;
double* aa;
double* bb;
} Cell;
typedef struct
{
Cell l;
Cell r;
} Send;
typedef struct
{
size_t count;
size_t subCount;
Send* send;
} Set;
Set* allocateSendRecv(size_t, size_t);
int fillSendRecv(Set*);
Set* freeSendRecv(Set*);
int printSendRecv(Set*, const char*);
// helpers
Set* from_disk(const char*);
double get_next(void);
int print_cell(Cell*, size_t, const char*);
int read_cell(FILE*, Cell*, const size_t);
int to_disk(Set*, const char*);
int write_cell(Cell*, const size_t, FILE*);
code in file v2.c
#include "v2.h"
#include <stdio.h>
#pragma pack(show)
Set* allocateSendRecv(size_t count, size_t subCount)
{
// count is the number of send buffers
// subcount is the size of the arrays inside each cell
printf(
"AllocateSendRecv(count = %llu, subCount = %llu)\n", count,
subCount);
Set* nw = (Set*)malloc(sizeof(Set));
nw->count = count;
nw->subCount = subCount;
nw->send = (Send*)malloc(count * sizeof(Send));
// now that we have Send allocate the Cell arrays
for (size_t i = 0; i < count; i++)
{
nw->send[i].l.aa =
(double*)malloc(subCount * sizeof(double));
nw->send[i].l.bb =
(double*)malloc(subCount * sizeof(double));
nw->send[i].r.aa =
(double*)malloc(subCount * sizeof(double));
nw->send[i].r.bb =
(double*)malloc(subCount * sizeof(double));
}
return nw;
}
int fillSendRecv(Set* s)
{
printf("FillSendRecv()\n");
if (s == NULL) return -1;
for (size_t i = 0; i < s->count; i += 1)
{
// l
s->send[i].l.a = get_next();
s->send[i].l.b = get_next();
s->send[i].l.c = get_next();
for (size_t j = 0; j < s->subCount; j += 1)
s->send[i].l.aa[j] = get_next();
for (size_t j = 0; j < s->subCount; j += 1)
s->send[i].l.bb[j] = get_next();
// r
s->send[i].r.a = get_next();
s->send[i].r.b = get_next();
s->send[i].r.c = get_next();
for (size_t j = 0; j < s->subCount; j += 1)
s->send[i].r.aa[j] = get_next();
for (size_t j = 0; j < s->subCount; j += 1)
s->send[i].r.bb[j] = get_next();
}
return 0;
}
Set* freeSendRecv(Set* set)
{
if (set == NULL) return NULL;
printf(
"\nDeallocating(count = %llu, subCount = %llu)\n",
set->count, set->subCount);
for (size_t i = 0; i < set->count; i++)
{
free(set->send[i].l.aa);
free(set->send[i].l.bb);
}
free(set->send);
free(set);
return NULL;
}
int printSendRecv(Set* s, const char* msg)
{
if (s == NULL) return -1;
if (msg != NULL) printf("%s", msg);
printf(
" Count is %llu, subCount is %llu\n", s->count,
s->subCount);
for (size_t i = 0; i < s->count; i += 1)
{
printf("\tSet %llu of %llu\n", 1 + i, s->count);
print_cell(&s->send[i].l, s->subCount, "\tl:\n");
print_cell(&s->send[i].r, s->subCount, "\tr:\n");
printf("\n");
}
printf("\n");
return 0;
}
// helpers
Set* from_disk(const char* file)
{
printf("read 'Set' from \"%s\"\n", file);
FILE* in = fopen(file, "rb");
if (in == NULL) return NULL;
size_t res = 0;
size_t count = 0;
res = fread(&count, sizeof(count), 1, in);
size_t subCount = 0;
res = fread(&subCount, sizeof(subCount), 1, in);
printf("From disk: Count = %llu, SubCount = %llu\n",
count,subCount);
Set* nw = allocateSendRecv(count, subCount);
if (nw == NULL)
{
fclose(in);
return NULL; // could not alloc
}
printf("new 'Set' created\n");
nw->count = count;
nw->subCount = subCount;
// so we have the exact structure to hold ALL data
for (size_t i = 0; i < nw->count; i += 1)
{
read_cell(in, &nw->send[i].l, nw->subCount);
read_cell(in, &nw->send[i].r, nw->subCount);
}
fclose(in);
return nw;
}
double get_next(void)
{
static double ix = 42.;
ix += .001;
return ix;
}
int print_cell(Cell* cell, size_t sz, const char* msg)
{
printf(
"%s\t[a,b,c] = [%10.3f,%10.3f,%10.3f]\n", msg,
cell->a, cell->b, cell->c);
printf("\taa: ");
for (size_t j = 0; j < sz; j += 1)
printf("%10.3f ", cell->aa[j]);
printf("\n\tbb: ");
for (size_t j = 0; j < sz; j += 1)
printf("%10.3f ", cell->bb[j]);
printf("\n\n");
return 0;
}
int read_cell(FILE* in, Cell* cell, const size_t size)
{
if (in == NULL) return -2;
if (cell == NULL) return -1;
size_t res = 0;
// a,b,c,aa,bb
res += fread(&cell->a, 1, 3 * sizeof(double), in);
res += fread(cell->aa, 1, size * sizeof(double), in);
res += fread(cell->bb, 1, size * sizeof(double), in);
return 0;
}
int to_disk(Set* set, const char* file)
{
printf("writing 'Set' to \"%s\"\n", file);
FILE* out = fopen(file, "wb");
if (out == NULL) return -1;
size_t res = 0;
res = fwrite(&set->count, sizeof(set->count), 1, out);
res = fwrite(&set->subCount, sizeof(set->subCount), 1, out);
for (size_t i = 0; i < set->count; i += 1)
{
write_cell(&set->send[i].l, set->subCount, out);
write_cell(&set->send[i].r, set->subCount, out);
}
fclose(out);
return 0;
}
int write_cell(Cell* cell, const size_t size, FILE* out)
{
if (cell == NULL) return -1;
if (out == NULL) return -2;
size_t res = 0;
// a,b,c, aa, bb
res += fwrite(&cell->a, 1, 3 * sizeof(double), out);
res += fwrite(cell->aa, 1, size * sizeof(double), out);
res += fwrite(cell->bb, 1, size * sizeof(double), out);
//printf("write_cell(): %llu bytes written to disk\n", res);
return 0;
}
main() for the 2 examples is above in text
casting the return for malloc()
Yes, I always cast the return of malloc() as I and many others do no like anything implicit. And also because malloc() accepts any expression that evaluates to a size an lloking at the expression not always say something about the area. Many times the program allocates data for many structures, some enclosed. This little program has 3. So using the cast works as a reminder for the programmmers of what the program intends to allocate, and can avoid many bugs, since the expression many times is not sufficient to show what is what.
This thing about malloc() and cast comes from the C-FAQ, an old never-updated thing that is a compilation of articles from usenet all dating before 2000. And even in that time people wrote there about the possible reasons to CAST the pointer.
One of the reason pro-casting in the (C-FAQ)[https://c-faq.com/malloc/sd3.html] is that it could alert the programmer for have forgotten to use an include for stdlib.h. I mean it:
Suppose that you call malloc but forget to #include <stdlib.h>.
The compiler is likely to assume that malloc is a function
returning int, which is of course incorrect, and will lead to trouble
Therefore, the seemingly redundant casts are used by people who are
(a) concerned with portability to all pre-ANSI compilers, or
(b) of the opinion that implicit conversions are a bad thing.
I would add the reason I described above.
aaandbbfields (e.g. have them point into the allocated memory), and only after you can populate them. But if applicable, keep it easy and declarestruct cell { double a; double b; double a[sAS-1]; double b[sAS-1];}void compute_aa_and_bb(struct cell * Cells, int count, int sAS), and invoke it right afterallocateInOneSendRecv()on the sender size, and right afterMPI_Recv()on the receiver size.allocateInOneSendRecv()is the one crashing onx64-Releaseon my machine. UsingallocateSendRecv()runs ok. The structure only have pointer and double so it should not have problems with alignment.