Skip to content

memory leak in verify_tuple #440

Description

@mdcallag

Describe the bug

verify_tuple allocates memory for expected_msg but does not free it. I assume that is also the root cause for #306

Updated that diff just now to remove the "//"

This is fixed by:

diff --git a/tests/test_common.c b/tests/test_common.c
index b57cbc4..69c5995 100644
--- a/tests/test_common.c
+++ b/tests/test_common.c
@@ -55,6 +55,7 @@ verify_tuple(trunk_handle           *spl,
          platform_error_log("expected data: %s\n", data_str);
          platform_assert(FALSE);
       }
+      merge_accumulator_deinit(&expected_msg);
    }
 }

Reproduction steps

Run this and watch process RSS. It is stable during the insert stage, close to 5G but grows once the lookup stage begins. With a large enough number of lookups I get OOM on my 16G server.

build/release/bin/driver_test splinter_test --perf --max-async-inflight 0 --num-insert-threads 1 --num-lookup-threads 1 --num-range-lookup-threads 0 --db-capacity-gib 250 --key-size 20 --data-size 200 --db-location /data/m/db  --cache-capacity-gib 5  --num-inserts 50000000

Run with valgrind --tool=massif and I get reports like that show the sources of the allocations and test_common.c is verify_tuple.

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 87 490,274,083,607    3,804,596,048    3,354,196,432   450,399,616            0
 88 494,269,690,703    3,876,520,272    3,416,870,212   459,650,060            0
 89 496,409,647,753    3,917,957,056    3,452,978,272   464,978,784            0
88.13% (3,452,978,272B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->80.41% (3,150,335,904B) 0x4892587: UnknownInlinedFun (platform_inline.h:430)
| ->80.41% (3,150,335,904B) 0x4892587: UnknownInlinedFun (util.c:24)
|   ->80.41% (3,150,335,904B) 0x4892587: writable_buffer_resize (util.c:43)
|     ->80.41% (3,150,335,904B) 0x488E3C0: merge_accumulator_resize (data_internal.c:50)
|     | ->80.41% (3,150,335,904B) 0x116BF3: UnknownInlinedFun (test.h:177)
|     | | ->80.41% (3,150,335,904B) 0x116BF3: UnknownInlinedFun (test_common.c:48)
|     | |   ->80.41% (3,150,335,904B) 0x116BF3: test_trunk_lookup_thread (splinter_test.c:302)
|     | |     ->80.41% (3,150,335,904B) 0x48954E3: task_invoke_with_hooks (task.c:181)
|     | |       ->80.41% (3,150,335,904B) 0x498AB42: start_thread (pthread_create.c:442)
|     | |         ->80.41% (3,150,335,904B) 0x4A1BBB3: clone (clone.S:100)
|     | |
|     | ->00.00% (0B) in 1+ places, all below ms_print's threshold (01.00%)
|     |
|     ->00.00% (0B) in 1+ places, all below ms_print's threshold (01.00%)
|
->06.69% (262,144,000B) 0x488A0EE: UnknownInlinedFun (platform_inline.h:419)
| ->06.69% (262,144,000B) 0x488A0EE: clockcache_init (clockcache.c:1850)
|   ->06.69% (262,144,000B) 0x11EF82: splinter_test (splinter_test.c:2806)
|     ->06.69% (262,144,000B) 0x491FD8F: (below main) (libc_start_call_main.h:58)
|
->01.03% (40,498,368B) in 42 places, all below massif's threshold (1.00%)


### Expected behavior

I expect process RSS to be not much larger than sizeof(write buffer) + sizeof(block cache), regardless it shouldn't grow forever.

### Additional context

_No response_

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions