Segmentation fault in C while declaring large pointer array

Question

I run in a problem with a program and I'm not sure how to solve it. I'm processing a file and to do so I get the size with ftell and store it in M_size. After that I declare a unsigned char pointer array with N. The array is then used in two functions a() and b().

...
unsigned long N = (M_size/ x);
int LstElemSize = M_size % x;
if(LstElemSize != 0){
    N += 1;
}    
unsigned char *ptr_M[N]
a(ptr_M)
b(ptr_M)
...

Function a() actually initializes each element of ptr_M in a for loop:

a(){
  int i;
  for(i = 0; i < N-1; i ++){
     ptr_M[i] = malloc(sizeof(unsigned char) * x);
  }
 }

Function b() iterates then over each element and calculates stuff and at the end each element is freed.

My problem is now that when I try to process a file e.g. 1 GB the array size will be around 4 000 000 and a Segmentation error occurs (In the line i declare my array). If I calculated it correctly that is 8 byte (char pointer) times 4 000 000 = 32MB. The server running the program has enough memory to hold the file, but i guess as mentioned in Response 1 the stack space is not enough.

What can I do to solve my problem? Increase my stack space? Thanks!

Are you trying to create a 32MB array on the stack? The stack is usually in the single-digit MB range, on Windows using VC++ the default is only 1MB. — Some programmer dude
– Some programmer dude, Commented Sep 26, 2014 at 18:23

Macil · Accepted Answer · 2014-09-26 18:26:16Z

1

The problem is that you're defining ptr_M in the stack, which has a small size limit. The heap does not have such a small size limit and is able to use more of your system's memory. You need to use malloc() to allocate ptr_M just like you allocate the subarrays. (Make sure to free it at some point too along with all those subarrays!)

unsigned char **ptr_M = malloc(sizeof(unsigned char*) * N);

Also, your a() has an off-by-one error. It ignores the last item of the array. Use this:

  for(i = 0; i < N; i ++){

answered Sep 26, 2014 at 18:26

Macil

3,7261 gold badge22 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

m_hau Over a year ago

You're right about the off-by-one error but actually I'm doing something else with the last element. But without any context you're absolutely right!

Joe · Accepted Answer · 2014-09-26 18:28:41Z

1

unsigned char *ptr_M[N] is a variable-length array declaring N number of unsigned char on the stack in your case. You should dynamically allocate the space for the array as well.

unsigned char **ptr_M = malloc(sizeof(unsigned char*) * N);
a(ptr_M);
b(ptr_M);
...
//After you free each element in ptr_M
free(ptr_M);

edited Sep 26, 2014 at 18:28

answered Sep 26, 2014 at 18:23

Joe

57.2k9 gold badges130 silver badges136 bronze badges

1 Comment

m_hau Over a year ago

I would upvote this too for the free hints, but I can't. Hope someone else will do it! Thanks!

Dr. Debasish Jana · Accepted Answer · 2014-09-26 18:22:58Z

0

malloc allocates space from heap, not from stack. You may try increasing your heapsize looking at the compiler option. Check the upper limit of heapsize that is supported there.

answered Sep 26, 2014 at 18:22

Dr. Debasish Jana

7,1386 gold badges40 silver badges76 bronze badges

Collectives™ on Stack Overflow

Segmentation fault in C while declaring large pointer array

3 Answers 3

1 Comment

1 Comment

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Related