- Richard Feynman -
Chapter:C memory tools
Contents
Meta information about this chapter
Expand using link to the right to see the full content.
Introduction
Purpose
Requirements
The student shall be familiar with the following concepts:
Goal
The student shall be able use the functions:
Instructions to the teacher
Common problems
Chapter videos
All videos in this chapter:
- C Memory tools [ (eng)]
See below for individual links to the videos.
Memory leaks
Description
In computer science, a memory leak is a type of resource leak that occurs when a computer program incorrectly manages memory allocations[1] in such a way that memory which is no longer needed is not released. In object-oriented programming, a memory leak may happen when an object is stored in memory but cannot be accessed by the running code.[2] A memory leak has symptoms similar to a number of other problems and generally can only be diagnosed by a programmer with access to the program's source code.
Memory_leak (wkipedia link)
Look at the following source code:
char* str_dup(char* str)
{
char *new = (char*) calloc(strlen(str)+1, sizeof(char));
if (new==NULL)
{
return NULL;
}
return strncpy(new,str, strlen(str));
}
int main(void)
{
char *name = str_dup("Henrik");
printf("name: %s\n", name);
/* free(name); */
return 0;
}
It's easy to see that the memory allocated using calloc is never returned (using free). And in this simple example it is not the end of the world since the memory will be returned once the program exits. But imagne you leaked memory in a loop and that your program should run for some years without crashing. It simply doesn't do. So, how do we find memory leaks? Enter memory tools.
Tools
Valgrind / memcheck
Valgrind is a tool to help you analysememory management and profile your program (among other things).
Memcheck detects memory-management problems, and is aimed primarily at C and C++ programs. When a program is run under Memcheck's supervision, all reads and writes of memory are checked, and calls to malloc/new/free/delete are intercepted. As a result, Memcheck can detect if your program:
- Accesses memory it shouldn't (areas not yet allocated, areas that have been freed, areas past the end of heap blocks, inaccessible areas of the stack).
- Uses uninitialised values in dangerous ways.
- Leaks memory.
- Does bad frees of heap blocks (double frees, mismatched frees).
- Passes overlapping source and destination memory blocks to memcpy() and related functions.
Memcheck reports these errors as soon as they occur, giving the source line number at which it occurred, and also a stack trace of the functions called to reach that line. Memcheck tracks addressability at the byte-level, and initialisation of values at the bit-level. As a result, it can detect the use of single uninitialised bits, and does not report spurious errors on bitfield operations. Memcheck runs programs about 10--30x slower than normal.
Example use of valgrind:
$ gcc -g allocer.c -o allocer
$ valgrind --leak-check=full ./allocer
==9579== Memcheck, a memory error detector
==9579== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==9579== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==9579== Command: ./allocer
==9579==
name: Henrik
==9579==
==9579== HEAP SUMMARY:
==9579== in use at exit: 7 bytes in 1 blocks
==9579== total heap usage: 2 allocs, 1 frees, 1,031 bytes allocated
==9579==
==9579== 7 bytes in 1 blocks are definitely lost in loss record 1 of 1
==9579== at 0x4C2FA50: calloc (vg_replace_malloc.c:711)
==9579== by 0x4005FE: str_dup (allocer.c:8)
==9579== by 0x400646: main (allocer.c:19)
==9579==
==9579== LEAK SUMMARY:
==9579== definitely lost: 7 bytes in 1 blocks
==9579== indirectly lost: 0 bytes in 0 blocks
==9579== possibly lost: 0 bytes in 0 blocks
==9579== still reachable: 0 bytes in 0 blocks
==9579== suppressed: 0 bytes in 0 blocks
==9579==
==9579== For counts of detected and suppressed errors, rerun with: -v
==9579== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Compiling and then running your program inside valgrind makes it a piece of cake to 1) find out the program above leaked memory and 2) where it leaked.
Videos
Memory management error
Description
Reading or writing to memory not "belonging to you", reading unitialised memory, reading/writing beyond an arrays limit etc are examples of memory management errors. As an example consider the code below:
int *values = calloc(100, sizeof(int));
if (values==NULL)
{
fprintf(stderr, "Failed allocating memory\n");
}
values[123]=123;
printf ("val %d\n", values[178]);
It is easy to see that the code above will write outside of the memory we (most liklely) want to use. It is equally easy to fix the problem, either by allocating a bigger array or writing inside the limits of the array. But how do we detect these kind of problems in a big program?
Same as above, we use memory tools:
The file can be found here: outside.c
Tools
There's a variety of tools out there for this purpose. We will give you some examples here:
Valgrind
See previous introduction to valgrind.
Example use of valgrind:
$ gcc -g outside.c -o outside
$ valgrind ./outside
==10358== Memcheck, a memory error detector
==10358== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==10358== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==10358== Command: ./outside
==10358==
==10358== Invalid write of size 4
==10358== at 0x400620: main (outside.c:14)
==10358== Address 0x520022c is 28 bytes inside an unallocated block of size 4,193,744 in arena "client"
==10358==
==10358== Invalid read of size 4
==10358== at 0x400630: main (outside.c:15)
==10358== Address 0x5200308 is 248 bytes inside an unallocated block of size 4,193,744 in arena "client"
==10358==
val 0
==10358==
==10358== HEAP SUMMARY:
==10358== in use at exit: 400 bytes in 1 blocks
==10358== total heap usage: 2 allocs, 1 frees, 1,424 bytes allocated
==10358==
==10358== LEAK SUMMARY:
==10358== definitely lost: 400 bytes in 1 blocks
==10358== indirectly lost: 0 bytes in 0 blocks
==10358== possibly lost: 0 bytes in 0 blocks
==10358== still reachable: 0 bytes in 0 blocks
==10358== suppressed: 0 bytes in 0 blocks
==10358== Rerun with --leak-check=full to see details of leaked memory
==10358==
==10358== For counts of detected and suppressed errors, rerun with: -v
==10358== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Videos
Performance
Description
In software engineering, performance testing is in general, a testing practice performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage. Software_performance_testing
In previous chapter we've looked at a function called mean. We will take a look at this code again, slightly modified:
#include <stdio.h>
#include <limits.h>
#include <math.h>
#include "simple-math.h"
#define PRINT_INT_VAR(a) { print_int_var(#a, a); }
static int math_debug_mode = 0;
void
set_math_debug(void)
{
math_debug_mode=1;
}
void
unset_math_debug(void)
{
math_debug_mode=0;
}
void
print_int_var(char *str, int val)
{
if (math_debug_mode)
{
fprintf(stderr, "%s=%d\n", str, val);
}
}
int
mean(int *array, int size, int *result)
{
int i ;
// Store the sum in a long long instead
long long sum;
if ( (array==NULL) || (result==NULL) )
{
return 1;
}
if (size<0)
{
return 2;
}
// initialise sum to 0
sum = 0 ;
for (i=0; i<size; i++)
{
int val = array[i];
PRINT_INT_VAR(val);
// add the value to sum
sum = sum + val;
// Check if the sum fits in an int
if ((sum>INT_MAX) || (sum<INT_MIN) )
{
printf ("sum: %lld\n", sum);
return 3;
}
PRINT_INT_VAR(sum);
}
*result = (int)round(((double)sum/(double)size));
return 0;
}
We have added a macro PRINT_INT_VAR to print both the variable name and its value. We use this macro in the function mean to print the value and the sum since as debug printouts.
We also have a short main program:
#include <time.h>
#include <stdlib.h>
#include <stdio.h>
#include "simple-math.h"
#define ARRAY_SIZE 20000
int main(int argc, char **argv)
{
int ints[ARRAY_SIZE];
int cnt;
int i;
int r;
int res;
int ret;
set_math_debug();
if (argc>1)
{
fprintf(stderr, "Using debug mode\n");
set_math_debug();
}
srand(time(NULL));
for (i=0; i<ARRAY_SIZE; i++)
{
r = rand()%100000;
ints[i]=r;
}
for (cnt=0; cnt < 10 ; cnt ++)
{
// READ ERROR - don't go for 101
ret = mean(ints, ARRAY_SIZE, &res);
if (ret != 0)
{
fprintf(stderr, "Failed calculating mean value (%d)\n", ret);
return 1;
}
}
fprintf(stdout, "Mean value: %d\n", res);
return 0;
}
If we compile run the program as it is, we will see:
$ gcc -g -c -o main.o main.c
$ gcc -g -c -o simple-math.o simple-math.c
$ ggcc -g -g -lm main.o simple-math.o -o ./mean
$ ./mean
followed by lots of printouts ....
Mean value: 453
Measuring the execution time
How long time did it take? Let's ise the program time to check this. Start our program (mean) via time like this:
$gcc -g -c -o main.o main.c
$gcc -g -c -o simple-math.o simple-math.c
$gcc -g -g -lm main.o simple-math.o -o ./mean
$time ./mean
.... snip
real 0m1.186s
user 0m0.119s
sys 0m0.518s
If we remove the debug print statements, by unsetting debug mode in main.c:
unset_math_debug();
$ gcc -g -c -o main.o main.c
$ gcc -g -c -o simple-math.o simple-math.c
$ gcc -g -g -lm main.o simple-math.o -o ./mean
$ time ./mean
Mean value: 49788
real 0m0.005s
user 0m0.003s
sys 0m0.002s
Let's compare the user time user 0m0.119s and user 0m0.003s. Uh oh, much faster. So print statements are "expensive". Still, it would be interesting to see where in the code we spend the most time. We want to do this to see if we can make our program faster (perform better). To do this we use performance tools:
Note: The file above, in its final version, can be downloaded here: simple-math.c with its coresponding header file simple-math.h and the test file main.c.
Tools
Valgrind / callgrind
To check where we spend our time we simply invoke valgrind like this:
valgrind --tool=callgrind ./mean
which will generate files called callgrind.out.12743 (where 12743 is different between runs) or similar. And then run the program kcachegrind:
kcachegrind callgrind.out.12743
which produces a graph like this:
From this picture we can see that we spend roughly 50% of our time in the function print_int_var. So even if only checking the value of a variable (math_debug_mode) we seem to waste a lot of time. Can we conditionally discard the debug printout statements. Of course we can, let's write the macro like this instead:
#ifdef MATH_DEBUG
#define PRINT_INT_VAR(a) { print_int_var(#a, a); }
#else
#define PRINT_INT_VAR(a)
#endif
Let's run the program in time with the mcro discarded:
$ gcc -g -c -o main.o main.c
$ gcc -g -c -o simple-math.o simple-math.c
$ gcc -g -g -lm main.o simple-math.o -o ./mean
Mean value: 49829
real 0m0.001s
user 0m0.001s
sys 0m0.001s
Our program is now even faster. And note that we can get the debug printout statements basck by compiling using gcc -DMATH_DEBUG.
