View source for Chapter:C memory tools

[[Category:Functions]]

==Meta information about this chapter==
<div class="mw-collapsible mw-collapsed">

Expand using link to the right to see the full content.
<div class="mw-collapsible-content">

=== Introduction ===


==== Purpose ====


=== Requirements ===

The student shall be familiar with the following concepts:

==== Goal ==== 

The student shall be able use the functions:

===Instructions to the teacher===

====Common problems====

</div>
</div>

==Chapter videos==

All videos in this chapter:

* C Memory tools [ (eng)]  

See below for individual links to the videos.

== Memory leaks ==

===Description===

''In computer science, a memory leak is a type of resource leak that occurs when a computer program incorrectly manages memory allocations[1] in such a way that memory which is no longer needed is not released. In object-oriented programming, a memory leak may happen when an object is stored in memory but cannot be accessed by the running code.[2] A memory leak has symptoms similar to a number of other problems and generally can only be diagnosed by a programmer with access to the program's source code.''

[https://en.wikipedia.org/wiki/Memory_leak Memory_leak] (wkipedia link)


Look at the following source code:
<source lang="c">
char* str_dup(char* str)
{
  char *new = (char*) calloc(strlen(str)+1, sizeof(char));
  if (new==NULL)
    {
      return NULL;
    }
  return strncpy(new,str, strlen(str));
}

int main(void)
{
  char *name = str_dup("Henrik");


  printf("name: %s\n", name);


  /* free(name); */
  return 0;
}
</source>

It's easy to see that the memory allocated using <code>calloc</code> is never returned (using <code>free</code>). And in this simple example it is not the end of the world since the memory will be returned once the program exits. But imagne you leaked memory in a loop and that your program should run for some years without crashing. It simply doesn't do. So, how do we find memory leaks? Enter memory tools.

===Tools===

====Valgrind / memcheck====

Valgrind is a tool to help you analysememory management and profile your program (among other things).

''Memcheck detects memory-management problems, and is aimed primarily at C and C++ programs. When a program is run under Memcheck's supervision, all reads and writes of memory are checked, and calls to malloc/new/free/delete are intercepted. As a result, Memcheck can detect if your program:''

* Accesses memory it shouldn't (areas not yet allocated, areas that have been freed, areas past the end of heap blocks, inaccessible areas of the stack).
* Uses uninitialised values in dangerous ways.
* Leaks memory.
* Does bad frees of heap blocks (double frees, mismatched frees).
* Passes overlapping source and destination memory blocks to memcpy() and related functions.

''Memcheck reports these errors as soon as they occur, giving the source line number at which it occurred, and also a stack trace of the functions called to reach that line. Memcheck tracks addressability at the byte-level, and initialisation of values at the bit-level. As a result, it can detect the use of single uninitialised bits, and does not report spurious errors on bitfield operations. Memcheck runs programs about 10--30x slower than normal.''

[http://valgrind.org/info/tools.html Valgrind tools]

Example use of valgrind:
<source lang="bash">
$ gcc -g allocer.c -o allocer 
$ valgrind --leak-check=full ./allocer 
==9579== Memcheck, a memory error detector
==9579== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==9579== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==9579== Command: ./allocer
==9579== 
name: Henrik
==9579== 
==9579== HEAP SUMMARY:
==9579==     in use at exit: 7 bytes in 1 blocks
==9579==   total heap usage: 2 allocs, 1 frees, 1,031 bytes allocated
==9579== 
==9579== 7 bytes in 1 blocks are definitely lost in loss record 1 of 1
==9579==    at 0x4C2FA50: calloc (vg_replace_malloc.c:711)
==9579==    by 0x4005FE: str_dup (allocer.c:8)
==9579==    by 0x400646: main (allocer.c:19)
==9579== 
==9579== LEAK SUMMARY:
==9579==    definitely lost: 7 bytes in 1 blocks
==9579==    indirectly lost: 0 bytes in 0 blocks
==9579==      possibly lost: 0 bytes in 0 blocks
==9579==    still reachable: 0 bytes in 0 blocks
==9579==         suppressed: 0 bytes in 0 blocks
==9579== 
==9579== For counts of detected and suppressed errors, rerun with: -v
==9579== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
</source>

Compiling and then running your program inside valgrind makes it a piece of cake to 1) find out the program above leaked memory and 2) where it leaked.

===Videos===

== Memory management error ==

===Description===

Reading or writing to memory not "belonging to you", reading unitialised memory, reading/writing beyond an arrays limit etc are examples of memory management errors. As an example consider the code below:

<source lang="c">
  int *values = calloc(100, sizeof(int));

  if (values==NULL)
    {
      fprintf(stderr, "Failed allocating memory\n");
    }

  values[123]=123;
  printf ("val %d\n", values[178]);

</source>

It is easy to see that the code above will write outside of the memory we (most liklely) want to use. It is equally easy to fix the problem, either by allocating a bigger array or writing inside the limits of the array. But how do we detect these kind of problems in a big program?

Same as above, we use memory tools:

The file can be found here: [https://github.com/progund/programming-with-c/blob/master/mem-tools/wiki-material/memory/outside.c outside.c]

===Tools===

There's a variety of tools out there for this purpose. We will give you some examples here:

====Valgrind====

See previous introduction to valgrind.

Example use of valgrind:
<source lang="bash">
$ gcc -g outside.c -o outside
$ valgrind ./outside 
==10358== Memcheck, a memory error detector
==10358== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==10358== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==10358== Command: ./outside
==10358== 
==10358== Invalid write of size 4
==10358==    at 0x400620: main (outside.c:14)
==10358==  Address 0x520022c is 28 bytes inside an unallocated block of size 4,193,744 in arena "client"
==10358== 
==10358== Invalid read of size 4
==10358==    at 0x400630: main (outside.c:15)
==10358==  Address 0x5200308 is 248 bytes inside an unallocated block of size 4,193,744 in arena "client"
==10358== 
val 0
==10358== 
==10358== HEAP SUMMARY:
==10358==     in use at exit: 400 bytes in 1 blocks
==10358==   total heap usage: 2 allocs, 1 frees, 1,424 bytes allocated
==10358== 
==10358== LEAK SUMMARY:
==10358==    definitely lost: 400 bytes in 1 blocks
==10358==    indirectly lost: 0 bytes in 0 blocks
==10358==      possibly lost: 0 bytes in 0 blocks
==10358==    still reachable: 0 bytes in 0 blocks
==10358==         suppressed: 0 bytes in 0 blocks
==10358== Rerun with --leak-check=full to see details of leaked memory
==10358== 
==10358== For counts of detected and suppressed errors, rerun with: -v
==10358== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
</source>

===Videos===

== Performance ==

===Description===

''In software engineering, performance testing is in general, a testing practice performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.'' [https://en.wikipedia.org/wiki/Software_performance_testing Software_performance_testing]

In previous chapter we've looked at a function called <code>mean</code>. We will take a look at this code again, slightly modified:

<source lang="c">
#include <stdio.h>
#include <limits.h>
#include <math.h>

#include "simple-math.h"

#define PRINT_INT_VAR(a) { print_int_var(#a, a); }

static int math_debug_mode = 0;

void
set_math_debug(void)
{
  math_debug_mode=1;
}

void
unset_math_debug(void)
{
  math_debug_mode=0;
}

void
print_int_var(char *str, int val)
{
  if (math_debug_mode)
    {
      fprintf(stderr, "%s=%d\n", str, val);
    }
}


int
mean(int *array, int size, int *result)
{
  int i ;
  // Store the sum in a long long instead
  long long sum;

  if ( (array==NULL) || (result==NULL) )
    {
      return 1;
    }

  if (size<0)
    {
      return 2;
    }

  // initialise sum to 0
  sum = 0 ;
  for (i=0; i<size; i++)
    {
      int val = array[i];
      PRINT_INT_VAR(val);
      // add the value to sum
      sum = sum + val;

      // Check if the sum fits in an int
      if ((sum>INT_MAX) || (sum<INT_MIN) )
        {
          printf ("sum: %lld\n", sum);
          return 3;
        }
      PRINT_INT_VAR(sum);
    }

  *result = (int)round(((double)sum/(double)size));
  return 0;
}
</source>

We have added a macro <code>PRINT_INT_VAR</code> to print both the variable name and its value. We use this macro in the function <code>mean</code> to print the value and the sum since as debug printouts. 

We also have a short main program:
<source lang="c">
#include <time.h>
#include <stdlib.h>
#include <stdio.h>

#include "simple-math.h"

#define ARRAY_SIZE 20000

int main(int argc, char **argv)
{
  int ints[ARRAY_SIZE];
  int cnt;
  int i;
  int r;
  int res;
  int ret;


  set_math_debug();
  if (argc>1)
    {
      fprintf(stderr, "Using debug mode\n");
      set_math_debug();
    }
  
  srand(time(NULL));
  for (i=0; i<ARRAY_SIZE; i++)
    {
      r = rand()%100000;
      ints[i]=r;
    }

  for (cnt=0; cnt < 10 ; cnt ++)
    {
      // READ ERROR - don't go for 101
      ret = mean(ints, ARRAY_SIZE, &res);
      if (ret != 0)
        {
          fprintf(stderr, "Failed calculating mean value (%d)\n", ret);
          return 1;
        }
    }
  
  fprintf(stdout, "Mean value: %d\n", res);
  return 0;
}
</source>

If we compile run the program as it is, we will see:
<source lang="bash">
$ gcc -g   -c -o main.o main.c
$ gcc -g   -c -o simple-math.o simple-math.c
$ ggcc -g -g -lm main.o simple-math.o -o ./mean
$ ./mean
</source>
followed by lots of printouts ....
<source lang="bash">
Mean value: 453
</source>

=== Measuring the execution time ===

How long time did it take? Let's ise the program <code>time</code> to check this. Start our program (<code>mean</code>) via <code>time</code> like this:

<source lang="bash">
$gcc -g   -c -o main.o main.c
$gcc -g   -c -o simple-math.o simple-math.c
$gcc -g -g -lm main.o simple-math.o -o ./mean
$time ./mean
.... snip
real	0m1.186s
user	0m0.119s
sys	0m0.518s
</source>

If we remove the debug print statements, by unsetting debug mode in <code>main.c</code>:
<source lang="c">
  unset_math_debug();
</source>

<source lang="bash">
$ gcc -g   -c -o main.o main.c
$ gcc -g   -c -o simple-math.o simple-math.c
$ gcc -g -g -lm main.o simple-math.o -o ./mean
$ time ./mean
Mean value: 49788

real	0m0.005s
user	0m0.003s
sys	0m0.002s
</source>

Let's compare the user time <code>user	0m0.119s</code> and <code>user	0m0.003s</code>. Uh oh, much faster. So print statements are "expensive". Still, it would be interesting to see where in the code we spend the most time. We want to do this to see if we can make our program faster (perform better). To do this we use performance tools:

''Note: The file above, in its final version, can be downloaded here: [https://github.com/progund/programming-with-c/blob/master/mem-tools/wiki-material/performance/simple-math.c simple-math.c] with its coresponding header file [https://github.com/progund/programming-with-c/blob/master/mem-tools/wiki-material/performance/simple-math.h simple-math.h] and the test file [https://github.com/progund/programming-with-c/blob/master/mem-tools/wiki-material/performance/main.c main.c].''

===Tools===

====Valgrind / callgrind ====

To check where we spend our time we simply invoke valgrind like this:

<source lang="bash>
valgrind --tool=callgrind ./mean
</source>

which will generate files called <code>callgrind.out.12743</code> (where 12743 is different between runs) or similar. And then run the program <code>kcachegrind</code>:

<source lang="bash>
kcachegrind callgrind.out.12743
</source>

which produces a graph like this:

[[File:kcachegrind.png|800px|Kcachegrind]]

From this picture we can see that we spend roughly 50% of our time in the function <code>print_int_var</code>. So even if only checking the value of a variable (<code>math_debug_mode</code>) we seem to waste a lot of time. Can we conditionally discard the debug printout statements. Of course we can, let's write the macro like this instead:

<source lang="c">
#ifdef MATH_DEBUG
#define PRINT_INT_VAR(a) { print_int_var(#a, a); }
#else
#define PRINT_INT_VAR(a)
#endif
</source>

Let's run the program in time with the mcro discarded:

<source lang="text">
$ gcc -g   -c -o main.o main.c
$ gcc -g   -c -o simple-math.o simple-math.c
$ gcc -g -g -lm main.o simple-math.o -o ./mean
Mean value: 49829
real	0m0.001s
user	0m0.001s
sys	0m0.001s
</source>

Our program is now even faster. And note that we can get the debug printout statements basck by compiling using <code>gcc -DMATH_DEBUG</code>.