Chapter:C dynamically loaded libraries

From Juneday education
Jump to: navigation, search

Dynamically loaded libraries

We asumme you're familiar with:

Have you ever wondered how VLC loads various codecs? If not, think about it for a while. VLC can load a piece of software written by someone without the VLC project having any knowledge about this. How is this possible? By using an interface (specified in a header file) and using the so called dl functions. In this chapter we will write a small chat client that specifies an interface (again, as a header file) and a main function that can talk to some software implementing this interface.

Videos

C - Dynamically Loadable Library (Full playlist) | C - Dynamically Loadable Library 1/2 | C - Dynamically Loadable Library 2/2 | Dynamically Loadable Library (PDF)

Example library

You can use code from libraries by linking your program against these libraries. The linking can be done statically (using an archive, e g libcode.a) or dynamically (using a sharead object, e g libcode.so). We have written some simple piece of code to repeat libraries briefly and after that use dynamically loaded libraries.

Let's look at the code of the library first.

Library source code

code.h:

#ifndef CODE_H
#define CODE_H

void fancy_printing(char *str);

#endif /* CODE_H */

code.c:

#include <stdio.h>

#include "code.h"

void fancy_printing(char *str)
{
  fprintf(stdout,
          "%s %s():%d | %s\n",
          __FILE__, __func__, __LINE__,
          str);
}

and a simple main file, classic-main.c, using the function in code.c:

#include "code.h"

int main(void)
{
  fancy_printing("Hello old classic world\n");
  return 0;
}


Repeating archive

First we need to create an archive:

$ gcc -c -Wall -pedantic -Werror -Wextra -fno-strict-aliasing -g code.c
$ ar rcv libcode.a code.o
a - code.o

Let's check it by using the file command:

$ file libcode.a 
libcode.a: current ar archive

Ok, it seems to be an archive, let's compile the main file and link against the archive:

gcc classic-main.c -L. -lcode -o static-main

Let's check the progrm and see how it is linked:

$ ldd static-main 
	linux-vdso.so.1 (0x00007ffd59dee000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fb962542000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fb962925000)

and then execute the program

$ ./static-main 
code.c fancy_printing():9 | Hello old classic world

Repeating shared object file

First we need to create a shared object file:

gcc -c -Wall -pedantic -Werror -Wextra -fno-strict-aliasing -g -fPIC code.c
gcc -fPIC -shared code.o -o libcode.so

Let's check it by using the file command:

$ file libcode.so 
libcode.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=6102108ac327a972951c888291037dfa9c0ef302, with debug_info, not stripped

Ok, it seems to be a shred object file, let's compile the main file and link against the shared object file:

gcc classic-main.c -L. -lcode -o shared-main

Let's check the progrm and see how it is linked:

$ file libcode.so 
libcode.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=6102108ac327a972951c888291037dfa9c0ef302, with debug_info, not stripped
[hesa@bartok example]$ ldd shared-main 
	linux-vdso.so.1 (0x00007ffec65cb000)
	libcode.so => not found
	libc.so.6 => /lib64/libc.so.6 (0x00007f4811a37000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f4811e1a000)

We can see that the program is linked against the shared object file libcode.so

Now, it's time to execute the program

$ ./shared-main 
./shared-main: error while loading shared libraries: libcode.so: cannot open shared object file: No such file or directory

Oh yeah, we need to make sure the dynamic linker finds the shared object file by adding current directory to the linker's path where it looks for shared object files:

$ LD_LIBRARY_PATH=. ./shared-main 
code.c fancy_printing():9 | Hello old classic world

Note: The name of the function, fancy_printing, has to be the same in both the library and the program using the library.

If But what if we want to to link against a library

Dynamically Loaded Libraries

In the previous section we link against a shared object file libcode.so. What if we don't know the name of the shared object file. Enter dl. Let's first get to know the functions and then use the main program from the previous section to link against a shared obect file using the dl functions. The dl functions need to be included using dlfcn.h.

dlopen

This function opens a shared object file and prepares it for you to use.

  void * dlopen(const char *filename, int flag);

where filename is the name of the shared object file and flag either RTLD_LAZY, RTLD_NOW or RTLD_GOBAL. We refer to the manual for a complete description and will focus on RTLD_LAZY which manual entry says:

  Perform lazy binding.  Resolve symbols only as the code that references them is executed.  If the symbol is never  referenced,
  then  it is never resolved.  (Lazy binding is performed only for function references; references to variables are always imme‐
  diately bound when the shared object is loaded.)  Since glibc 2.1.1, this flag is overridden by the effect of the  LD_BIND_NOW
  environment variable.

Note: with shared object files you need to specify where dl should look for files just as you need to do with the linker when linking against shared object files. We do this via the environment variable LD_LIBRARY_PATH or give the path the shared object file (either absolute or relative).

dlsym

This function looks up a function (value of a symbol) in a shared object file.

 void * dlsym(void *handle, char *symbol);

dlclose

This function closes the library.

 int dlclose(void *handle);

dlerror and more

dlerror is used to report errors. This chapter does not aim at giving a complete guid to dynamically loaded libraries. Rather it introduces the concept. You need to read a bit more if you want to master dl. See below for useful links.

A new main program using dl functions

Let's write a new main program using dl (and not linking against a shared object file we know the name of).

dlopen

  void *handle;
  handle = dlopen(libname, RTLD_LAZY);
  if (!handle) {
    fputs (dlerror(), stderr);
    return 1;
  }

dlsym

/* pinter to the print function the so file */
fancy_printing_fun fancy_printing;
  fancy_printing = dlsym(handle, "fancy_printing");
  if ((error = dlerror()) != NULL)  {
    fputs(error, stderr);
    return 2;
  }

invoking the symbol/function

  fancy_printing("Wow, we called a function in a file this program had no idea existed\n");

dlclose

  dlclose(handle);

complete main

#include <dlfcn.h>
#include <stdio.h>

typedef void (*fancy_printing_fun) (char *str);

/* kind of awkward to place these in main
   but let's do that for now and be a bit
   better in the coming chat example */

/* pinter to the print function the so file */
fancy_printing_fun fancy_printing;

/* hande to the so file */
void *handle;

int load_lib(char *libname)
{
  char *error;

  handle = dlopen(libname, RTLD_LAZY);
  if (!handle) {
    fputs (dlerror(), stderr);
    return 1;
  }  

  fancy_printing = dlsym(handle, "fancy_printing");
  if ((error = dlerror()) != NULL)  {
    fputs(error, stderr);
    return 2;
  }

  return 0;
}


int main(int argc, char **argv)
{
  int ret;
  
  if (argc<2)
    {
      fprintf(stderr, "Missing arguments... you must supply the name of a shared object file\n");
      return 1;
    }
  
  ret = load_lib(argv[1]);
  if (ret!=0)
    {
      fprintf(stderr, "Failed reading file '%s'\n", argv[1]);
      return 2;
    }

  
  fancy_printing("Wow, we called a function in a file this program had no idea existed\n");


  dlclose(handle);
  return 0;
}

Compiling and executing

Let's compile the new main file:

gcc -Wall -Werror -Wextra -fno-strict-aliasing -g dyndl-main.c -o dyn-main -ldl

Let's see what libraries the program is linked against:

$ ldd dyn-main 
	linux-vdso.so.1 (0x00007ffcb7155000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f1b41950000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f1b4156d000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1b41b54000)

Ok, so it is not linked against libcode.so. Insted of using dynamic linking and letting the dynamic loader find the shared object file libcode.so we're now using the dl functions to find the file and load the function. Let's execute

$ ./dyn-main 
Missing arguments... you must supply the name of a shared object file

Uh oh, we mys supply a name.

$ ./dyn-main ./libcod
./libcod: cannot open shared object file: No such file or directory
Failed reading file './libcod'

... and perhaps we should give a name of a valid shared object file

$ ./dyn-main ./libcode.so
code.c fancy_printing():9 | Wow, we called a function in a file this program had no idea existed

Nice, isn't it. You can change the name of the shared object file if you want to make sure we're not fooling you.

$ mv libcode.so libdonkey.so 
$ ./dyn-main ./libcode.so
./libcode.so: cannot open shared object file: No such file or directory
Failed reading file './libcode.so'
$ ./dyn-main ./libdonkey.so
code.c fancy_printing():9 | Wow, we called a function in a file this program had no idea existed

Note: just as when using shared object files or static archives the name of the function, fancy_printing has to be the same in the shared object file libcode.so as in the pogram using it.

Source code to the examples above

You can find the source code to the examples above here: https://github.com/progund/programming-with-c/tree/master/dyn-loaded-libs/example. Here's a short introduction to the files:

  • code.c - the library containing the function fancy_printing
  • code.h - coresponding header file to code.c
  • classic-main.c - file with a main function using the name of the function fancy_printing
  • dyndl-main.c - file with a main function using dl to find and use libcode.so
  • Makefile - makefile to ease things up a bit

make targets

  • make clean - cleans up
  • make dyn-main - creates a program using dl to find and use a shared object file
  • make libcode.a - create the static archive
  • make libcode.so - creates the shared object file
  • make shared-main - creates a program using classical shared linking
  • make static-main - creates a program using classical static linking

Chat client

We should now write a poor man's version of pidgin. Well actually a very poor man's version. Anyhow, the idea is to have a program that can talk/communicate with any kind of chat (be it IRC, jabber or what ever). The program should not know of any chat protocols. How is this possible? We define an interface of expected behaviour and let others implement this interface. We can, using the dl functions, load this code into our program. Et voila, our chat program supports the protocol.

Supported chat protocol

We will actually not write support for a proper chat or IM protocol, rather we will create a very limited protocol which supports:

  • sending text from a client and the text appears in all connected clients
  • sending the text .quit leaves the chat

Chat server

You can either launch a server using nc or a java server that we (Juneday) have written.

nc

Start nc in listeing mode, in port 9999

$ nc -l -p 9999

Every time our chat programs connects and leaves the chat you need to restart nc, so a good idea is to do:

$ while(true); do nc -l -p 9999; done

java server

We've written a small chat server in Java that you can use. Download the following file https://raw.githubusercontent.com/progund/java-extra-lectures/master/networking/chat/SimpleChatServer.java, compile it and execute it.

$ wget https://raw.githubusercontent.com/progund/java-extra-lectures/master/networking/chat/SimpleChatServer.java
$ javac SimpleChatServer.java
$ java SimpleChatServer

Note: you need to start the server in a separate terminal.

Defining an interface

Based on our experience from writing a chat client in C see https://github.com/progund/web-misc/tree/master/chat-client/c we specify the following interface.

We would for the software implementing our interface to implement functions as below:

int input_handler (const char *str, void *cc);
void chat_close_ptr (void *data);
int chat_loop_ptr (void *data);
void chat_set_feedback_fun_ptr(void *data, input_handler_ptr fun);
int chat_handle_input (void *data, char *msg);

If someone wants to implement our chat interface they must implement the functions above.

void* data

Why this void* data? The idea is to give the implementing software a chance to pass around some data structure we know nothing about. By providing void* data in our interface we can let the implementing software pass around a pointer to what ever they like.

Implementing the interface

The software implementing the interface need to:

  • implement the functions in the interface
  • create a shared library (with the functions)

Implement the functions in the interface

All we have to do to achieve this is to write functions with exactly the same name as in the interface. It is also important to use the return values as provided by the interface, otherwise the chat program will have no chance of knowing what the return values from an implementing piece of software means.

Include the header file

So we need to start of by including the interface file

#include "chat.h"


Internal data structure

Internally the chat library uses a data structure called chat_client.

typedef struct chat_client_
{
  int sockfd;
  unsigned int port;
  struct sockaddr_in serveraddr;
  struct hostent *server;
  char *host_name;
  int running;
  pthread_t chat_thread;

  fd_set read_fds;
  unsigned nfds;

  input_handler feedback;

} chat_client ;

This structure contains all the information that the chat library (NOT the chat program, which later on will load this library) needs to perform its task. The variable cc with the type chat_client is the one being passed around by the chat program as void *

chat_set_feedback_fun

Next, Let's implement the functions:

void chat_set_feedback_fun(chat_client *cc, input_handler fun)
{
  cc->feedback = fun;
}

This functions simply sets the function, given as an argument of type input_handler fun which as a function pointer we typedefed before.

chat_init

void*
chat_init(char *hostname, int port)
{
  chat_client* cc =
    calloc(sizeof(chat_client), 1);

  if (cc==NULL || hostname==NULL)
    {
      return NULL;
    }

  cc->host_name = strdup(hostname);
  cc->port = port ; //port; 

  fprintf(stderr, "host: %s:%d  (%p)\n",
          hostname, port, cc);

  chat_set_feedback_fun(cc, (input_handler)print_msg);

  return cc;
}

We need not fully understand this code. The important thing is to see that the chat library really implements this function.

chat_loop

int
chat_loop(chat_client *cc)
{
  int ret;

  cc->running = 1;
  ret = chat_loop_impl(cc);
  return ret;
}

This function implements the interface function chat_loop and in turn invokes the function chat_loop_impl. Again, the important thing is not to understand HOW this functions imelements the function in the interface, rather it is the fact that it DOES implement the function chat_loop. We've put the details in a separate function chat_loop_impl which hopefully makes it easier for you to read and focus on the important things for this chapter.

chat_handle_input

int chat_handle_input(chat_client *cc, char *msg)
{
  int written;

  if (cc==NULL || msg==NULL)
    {
      return CHAT_CLIENT_BAD_ARG;
    }

  if (COMP_STR(".quit", msg)==0)
    {
      cc->feedback("Leaving since user typed '.quit'", cc);
      cc->running = 0;
      return CHAT_CLIENT_LEAVE;
    }

  written = (int)write(cc->sockfd, msg, strlen(msg));
  if (written < 0)
    {
      fprintf(stderr,
              "Failed writing to socket: '%s'\n",
              msg);
      return CHAT_CLIENT_ERROR;
    }
  return CHAT_CLIENT_OK;
}

chat_close

void chat_close(chat_client *cc)
{
  cc->running=0;

  close(cc->sockfd);
  if (cc!=NULL) {
    if( cc->host_name!=NULL) {
      free(cc->host_name);
    }
    free(cc);
  }
}

Create a shared library (with the functions)

Let's compile the chat library and create a shared library libchat.so:

gcc -Wall -Werror -Wextra -fno-strict-aliasing -I../chat-program/interface -DDEBUG -shared -fPIC  chat-lib.c -o libchat.so

For details about the above command line we ask you to read C libraries. We now have a shared library implementing the chat interface. This means that we should able to plug this in to a chat program. But hey, we don't have such a program. Ok, so let's write one.

Chat program

chat_functions

typedef struct _chat_functions {
  input_handler_ptr input_handler;
  chat_init_ptr chat_init;
  chat_close_ptr chat_close;
  chat_handle_input_ptr chat_handle_input;
  chat_set_feedback_fun_ptr chat_set_feedback_fun;
  chat_loop_ptr chat_loop;
} chat_functions;

The structure above can store a function pointer to each of the functions in the interface. We need to somehow set these variables to point to the coresponding functions in the library but let's focus on the overall picture first.

The function pointers are defined as follows:

typedef int (*input_handler_ptr)(const char *str, void *cc);
typedef void* (*chat_init_ptr)(const char *str, int port);
typedef void (*chat_close_ptr)(void *data);
typedef int (*chat_loop_ptr)(void *data);
typedef void (*chat_set_feedback_fun_ptr)(void *data, input_handler_ptr fun);
typedef int (*chat_handle_input_ptr)(void *data, char *msg);

main function

We can write a main program using the functions declared in the interface. We can not run the program since we don't have any code implementing the interface - well, we do have but we don't know how to load that library in to our chat program. So let's start of by writing code using the interface functions.

int main(int argc, char **argv)
{
  int port;
  char *host;
  char *lib;

  /*
   *
   * Rudimentary parser
   *
   */
  if (argc < 4)
    {
      fprintf(stderr,"Usage:\n");
      fprintf(stderr,"  %s chat-library <hostname> <port>\n", argv[0]);
      return 1;
    }
  /* turn port number into an (unsigned) int */
  if (sscanf(argv[3], "%ud", &port)!=1)
    {
      fprintf(stderr,"invalid port: %s\n", argv[3]);
      return 2;
    }

  host = argv[2];
  lib = argv[1];

  start_chat(host, port, lib);

  return 0;
}

Finding the functions in the library

static void * load_chat_lib(char *name, chat_functions *cf)
{
  void *handle;

  handle = dlopen (name, RTLD_LAZY);
  char *error;
  if (!handle) {
    fputs (dlerror(), stderr);
    return NULL;
  }
  fprintf(stderr, "handle: %p\n", handle);

  cf->chat_init = dlsym(handle, "chat_init");
  if ((error = dlerror()) != NULL)  {
    fputs(error, stderr);
    fprintf(stderr, "could not load chat_init from lib\n");
    return NULL;
  }

  cf->chat_close = dlsym(handle, "chat_close");
  if ((error = dlerror()) != NULL)  {
    fputs(error, stderr);
    fprintf(stderr, "could not load chat_close from lib\n");
    return NULL;
  }

  cf->chat_loop = dlsym(handle, "chat_loop");
  if ((error = dlerror()) != NULL)  {
    fputs(error, stderr);
    fprintf(stderr, "could not load chat_loop from lib\n");
    return NULL;
  }

  cf->chat_set_feedback_fun = dlsym(handle, "chat_set_feedback_fun");
  if ((error = dlerror()) != NULL)  {
    fputs(error, stderr);
    fprintf(stderr, "could not load chat_set_feedback_fun from lib\n");
    return NULL;
  }

  fprintf(stderr, "chat_functions: %p\n", cf);

  return handle;
}

Concluding remarks

The chat client can only communicate with one chat per time which seems pretty lame. It's easy to add code to let each chat execute in one thread - and add some way for the user to tell the chat program which chat to send text to. The latter must be decided since all chats (running in separate threads) share one and the same stdin (standard input).

Source code

Links