just do IT

Monday, March 31, 2014

install zeal on ubuntu 12.04

dash is an excellent API documentation browser on OSX. It supports offline documentation sets for lots of programming language, and search in dash is a lot faster than search on web. You can have access to documentation instantly, even if you don't have network access. Fortunately, programmers working on windows, linux platform, can use zeal, which is a opensource clone of dash.
The easy way to install zeal on Ubuntu is to install from the PPA . But we can't use this option, because zeal depends on Qt5, but we can't install Qt5 on system standard location to avoid confication with Qt4, which is used for our product development. So, we choose to install Qt5 in /opt directory and build zeal ourselves.
A problem with build zeal is it requires c++11 support, which isn't supported by gcc v4.6. But we can't upgrade to newer version. So, we choose to use clang v3.3 (or newer version) to build zeal.
  1. Download qt5 installer from https://qt-project.org/downloads and install to /opt
  2. Install required packages and clang:
    sudo apt-get install libgstreamer-plugins-base0.10-dev libxslt-dev libxml2-dev libxcb-keysyms1-dev bsdtar clang-3.3 libclang1-3.3
  3. Download zeal source code:
    git clone https://github.com/jkozera/zeal.git
  4. Run
    /opt/Qt5.2.1/5.2.1/gcc_64/bin/qmake -spec linux-clang && make && sudo make install

Friday, July 5, 2013

access github through proxy

In China mainland, you never know for what reason a website gets blocked by the fucking GFW. Even if the site has nothing to do with politics. github is a example, many developers in China are victims. This post is about how to access github via a proxy, such as goagent.
The git pull/push command can be instructed to access remote server via proxy by setting https_proxy environment variable. So, we can run commands below use proxy.
https_proxy= git pull
export https_proxy=
git push
But we may get below error because the goagent ssl certificate can't be verified.
error: SSL certificate problem: unable to get local issuer certificate while accessing https://github.com/xxxxxxxxxxx
fatal: HTTP request failed
To resolve this problem, we can force git not to verify ssl ceritificate by setting GIT_SSL_NO_VERIFY environment variable.
export GIT_SSL_NO_VERIFY=true
git pull
To make these settings perminately for a git project or globally. We can write them to git config file.
git config http.proxy
git config http.sslVerify false
The resulting .git/config file is like this:
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
    ignorecase = true
[remote "origin"]
    url = https://github.com/rxwen/my_posts.git
    fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
    remote = origin
    merge = refs/heads/master
    proxy =
    sslVerify = false
BTW, there is a rumour that Fang BingXing gets ill badly recently. Wish Death can conquer him as soon as possible, amen.

Friday, May 3, 2013

install gevent on mac os

This post is for Mac users in China who wants to fuck the gfw.
The goagent has been rewritten based on gevent. Though it still supported to run without gevent, it's recommended to install gevent.
It's pretty easy to install gevent on mac os with brew and pip tools.
  1. brew install libevent
  2. sudo pip install gevent greenlet

Tuesday, March 5, 2013

use wifi on raspberrypi

The raspberrypi has two usb ports, so I bought a usb wifi dongle to get rid of network cable.
This dongle uses RTL8188CUS chipset, which has built-in support on the raspberrypi linux kernel. So it just worked after I plugged it to raspberrypi.
The next step is to configure the dongle so that it can connect to my wifi router. There is GUI based connection manager if you're using raspbian system image. But in my case, I use a small system image built with built from builtroot with only command line interface, I have to setup manually.
I enabled wireless tools in buildroot config, which provides tools such as iwlist, iwconfig. My wifi router, which is configured with dhcp server enabled, doesn't required password to connect. So I can setup the network with commands below:
ifconfig wlan0 up
iwconfig wlan0 essid WIFI_ESSID
udhcpc -i wlan0
Having tested the network run ok, I updated the /etc/network/interfaces file so that network would be setup automatically.
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

auto wlan0
iface wlan0 inet dhcp
    pre-up ifconfig $IFACE up
    pre-up iwconfig $IFACE essid WIFI_ESSID
    down ifconfig $IFACE down
I struggled a lot with the interfaces file due to the dhcp always failed to work. And I finally found that the connman package caused the conflict. Disable it solved the problem.

Thursday, February 28, 2013

buildroot for raspberrypi

I bought a raspberrypi to try some interseting ideas on.
raspberrypi provides several images for downloading, but they are too large and exceeds my needs. My desired raspberrypi image should meet:
  1. fully customizing capabilities on kernel and user applications
  2. python support
  3. standard c&c++ libiaries
  4. serial port communication with peripheral devices
Luckily, here is a buildroot project customized for raspberrypi. With buildroot, all my requirements can be easily satisfied. So I forked from the repository and made some changes, to make the targe image minimal.
  1. disable Qt5 library
  2. disable boost library
  3. change kernel to the official one maintained by raspberrypi team
To build the target image, run commands below:
cp configs/raspberry_simple_defconfig .config
After the build is finished. I just need to copy the output/build/linux-rpi-3.6.y/arch/arm/boot/Image to kernel.img on boot partition on the sd card, and untar output/images/rootfs.tar to root filesystem partition.
Power on, and now raspberry pi should be running our own system.

Sunday, January 20, 2013

understanding android build system out directory

The android build system keeps a clean seperation of source and output. All intermediates and final output are placed in the directory named out. So, the simple way to fully clean a build is deleting the out directory.
The hierarchy of the out directory is shown below:
|-- host/                           # the directory containing all tools and libraries of build system
 -- target/product/generic/         # the root of this product's out directory
    |-- data                        # the directory for creating data file system image
    |-- obj                         # the root directory of build process
    |   |-- APPS                    # android application 
    |   |-- ETC
    |   |-- EXECUTABLES             # the root directory containing all native executable build output
    |   |-- include
    |   |-- JAVA_LIBRARIES
    |   |-- lib                     # the directory containing copies of stripped shared libraries,
    |   |                           # other modules will search this directory for libraries to resolve linkage
    |   |-- PACKAGING
    |   |-- SHARED_LIBRARIES        # the root directory containing all native shared library build output
    |   |   |-- {LOCAL_MODULE_NAME}_intermediates    # the direcotry containing all build output for {LOCAL_MODULE_NAME} module
    |   |       |                                    # this naming convention is followed by all subdirectories of module
    |   |        -- LINKED          # the directory containing the linked binary file, e.g, .so file
    |    -- STATIC_LIBRARIES        # the root directory containing all native static library build output
    |-- root                        # the directory for creating root file system, ramdisk image
    |   |-- data
    |   |-- dev
    |   |-- proc
    |   |-- sbin
    |   |-- sys
    |    -- system
    |-- symbols                     # the directory contains all binary images that has debugging symbols
    |   |-- data
    |   |-- sbin
    |    -- system
     -- system                      # the directory for creating system.img, where most of appications and libraries reside
        |-- app
        |-- bin
        |-- etc
        |-- fonts
        |-- framework
        |-- lib
        |-- media
        |-- tts
        |-- usr
         -- xbin
Under the out/target/product/generic/obj directory, there are several subdirectories, APPS, EXECUTABLES, SHARED_LIBRARIES, STATIC_LIBRARIES. They contain build output for modules of different type, java application, native executable, shared libraries and static libraries, respectively. Under the module type's directory, there is a directory for each module of corresponding type, named with the module's name catenating _intermediates. So, when we need to clean a specific module, we can simply delete the intermediate directory for the module.
For example, in the Android.mk for stlport, there is a LOCAL_MODULE defined as libstlport, which is a shared library (by including $(BUILD_SHARED_LIBRARY)). The output of this module will be placed in SHARED_LIBRARIES/libstlport_intermediates directory. The linker will generate the final shared library in the SHARED_LIBRARIES/libstlport_intermediates/LINKED directory.
After a module has been compiled the linked, it's to be stripped and copied to directory for creating file system image. The build system doesn't perform stripping in place. Instead, it will first copy the file with debugging symbol information (the file under LINKED directory) to correct place in symbols directory. Then strip the file and save in intermediate directory (for executable) or obj/lib directory (for shared library), meanwhile, the file without symbol and the file with symbol are associated with 'objcopy --add-gnu-debuglink' command. Finally, the stripped file will be copied to system directory.
Once all modules are built, the system directory should have been populated with necessary files. The build system will create three file system images, ramdisk.img, userdata.img, and system.img with system, root and data as source directories respectively. The default choice of file system is yaffs2.

Tuesday, December 25, 2012

examine macro definition in gdb

When debugging a c++ application, I used to refer to source code to find out the actual definition of a macro. If the macro is not a simple one, I had to perform the expansion on a paper or use the 'gcc -E' command to find out what's the actual result. This is a tedious task. The gdb macro command helps examine the macro value, as long as the application under debugging contains information about preprocessor macros. This can be satisfied by passing -g3 option to the gcc compiler. As an example, we need to debug the application below
 1 #define FOO foo_value
 2 #define STR(val) "STR of "#val
 3 #define VAL STR(FOO)
 5 int main(int argc, const char *argv[])
 6 {
 7     const char* t = VAL;
 8 #undef VAL
 9 #define VAL "test" // define VAL to a different value 
10     const char* t2 = VAL;
12     return 0;
13 }
We compile the code with gcc -g3 command, and debug it in gdb. Then we can examine the actual value of VAL macro with macro exp command.
(gdb) macro exp VAL  // run when break on line 7
expands to: "STR of ""FOO"
(gdb) macro exp VAL  // run when break on line 10
expands to: "test"
It's worthy of note that the macro expansion is context awareness in gdb, so we can get different value when the application breaks on line 7 and 10.

Sunday, November 25, 2012

user event in libevent

libevent is usually used as substitution for select system call to write efficient and portable code. A benefit of libevent is, besides normal fd, it enables monitoring signal, timeout and user supplied event in a consistent manner. Typically, the code makes use of libevent has the below structure:
In the final step, we call event_base_dispatch function which will run a loop on current thread until there is no more events to handle. Since the thread is busy running the loop, if we want to active an user event, we must do it in a new thread. We should notice that in libevent, we must explicitly set threading support via the evthread_use_pthreads or evthread_use_windows_threads, and we should call evthread_make_base_notifiable function so that event_base can be notified by events on another thread. So, in order to use user event, we need to following things:
  1. Create an event struct for user event, and add it to the event_base
  2. Prepare threading for event_base by calling evthread_use_pthreads/evthread_use_windows_threads and evthread_make_base_notifiable
  3. Start a new thread, and monitor if the user event should be fired
  4. Call event_active on the user event if the firing condition has been satisfied The structure is shown below:
And here is sample code.
  1 /*
  2   This exmple program provides a trivial server program that listens for TCP
  3   connections on port 9995.  When they arrive, it writes a short message to
  4   each client connection, and closes each connection once it is flushed.
  6   Where possible, it exits cleanly in response to a SIGINT (ctrl-c).
  7 */
 10 #include <string.h>
 11 #include <errno.h>
 12 #include <stdio.h>
 13 #include <signal.h>
 14 #ifndef _WIN32
 15 #include <netinet/in.h>
 17 #  include <arpa/inet.h>
 18 # endif
 19 #include <sys/socket.h>
 20 #endif
 21 #include <pthread.h>
 23 #include <event2/bufferevent.h>
 24 #include <event2/buffer.h>
 25 #include <event2/listener.h>
 26 #include <event2/util.h>
 27 #include <event2/event.h>
 28 #include <event2/thread.h>
 30 static const char MESSAGE[] = "Hello, World!\n";
 32 static const int PORT = 9995;
 34 static void listener_cb(struct evconnlistener *, evutil_socket_t,
 35     struct sockaddr *, int socklen, void *);
 36 static void conn_readcb(struct bufferevent *, void *);
 37 static void conn_writecb(struct bufferevent *, void *);
 38 static void conn_eventcb(struct bufferevent *, shortvoid *);
 39 static void signal_cb(evutil_socket_t, shortvoid *);
 41 static struct event* init_user_event(struct event_base*);
 42 static void* user_event_proc(void*);
 44 int main(int argc, char **argv) {
 45     struct event_base *base;
 46     struct evconnlistener *listener;
 47     struct event *signal_event, *user_event;
 48     pthread_t th;
 50     struct sockaddr_in sin;
 51     int rc = 0;
 52 #ifdef _WIN32
 53     WSADATA wsa_data;
 54     WSAStartup(0x0201, &wsa_data);
 55 #endif
 57     base = event_base_new();
 58     if (!base) {
 59         fprintf(stderr"Could not initialize libevent!\n");
 60         return 1;
 61     }
 63     evthread_use_pthreads();
 64     if (evthread_make_base_notifiable(base)<0) {
 65         printf("Couldn't make base notifiable!");
 66         return 1;
 67     }
 68     memset(&sin, 0sizeof(sin));
 69     sin.sin_family = AF_INET;
 70     sin.sin_port = htons(PORT);
 72     listener = evconnlistener_new_bind(base, listener_cb, (void *)base,
 74         (struct sockaddr*)&sin,
 75         sizeof(sin));
 77     if (!listener) {
 78         fprintf(stderr"Could not create a listener!\n");
 79         return 1;
 80     }
 82     signal_event = evsignal_new(base, SIGINT, signal_cb, (void *)base);
 84     if (!signal_event || event_add(signal_event, NULL)<0) {
 85         fprintf(stderr"Could not create/add a signal event!\n");
 86         return 1;
 87     }
 88     user_event = init_user_event(base);
 89     pthread_create(&th, NULL, user_event_proc, user_event);
 91     /*rc = event_base_loop(base, EVLOOP_NO_EXIT_ON_EMPTY);*/
 92     event_base_dispatch(base);
 94     evconnlistener_free(listener);
 95     event_free(signal_event);
 96     event_free(user_event);
 97     event_base_free(base);
 99     printf("done\n");
100     return 0;
101 }
103 static void listener_cb(struct evconnlistener *listener, evutil_socket_t fd,
104     struct sockaddr *sa, int socklen, void *user_data) {
105     struct event_base *base = (struct event_base*)user_data;
106     struct bufferevent *bev;
108     bev = bufferevent_socket_new(base, fd, BEV_OPT_CLOSE_ON_FREE);
109     if (!bev) {
110         fprintf(stderr"Error constructing bufferevent!");
111         event_base_loopbreak(base);
112         return;
113     }
114     bufferevent_setcb(bev, conn_readcb, conn_writecb, conn_eventcb, NULL);
115     bufferevent_enable(bev, EV_WRITE);
116     bufferevent_enable(bev, EV_READ);
118     /*bufferevent_write(bev, MESSAGE, strlen(MESSAGE));*/
119 }
121 static void conn_readcb(struct bufferevent *bev, void *) {
122     struct evbuffer *input = bufferevent_get_input(bev);
123     printf("readcb\n");
124     int len = evbuffer_get_length(input);
125     if (len != 0) {
126         printf("readcb parse_message\n");
127         char* buf = new char[len]();
128         evbuffer_copyout(input, buf, len);
129         delete[] buf;
130     }
131     bufferevent_write(bev, MESSAGE, strlen(MESSAGE));
132 }
134 static void conn_writecb(struct bufferevent *bev, void *user_data) {
135     struct evbuffer *output = bufferevent_get_output(bev);
136     if (evbuffer_get_length(output) == 0) {
137         printf("flushed answer\n");
138         /*bufferevent_free(bev);*/
139     }
140 }
142 static void conn_eventcb(struct bufferevent *bev, short events, void *user_data) {
143     if (events & BEV_EVENT_EOF) {
144         printf("Connection closed.\n");
145     } else if (events & BEV_EVENT_ERROR) {
146         printf("Got an error on the connection: %s\n",
147             strerror(errno));/*XXX win32*/
148     }
149     /* None of the other events can happen here, since we haven't enabled
150      * timeouts */
151     bufferevent_free(bev);
152 }
154 static void signal_cb(evutil_socket_t sig, short events, void *user_data) {
155     struct event_base *base = (struct event_base*)user_data;
156     struct timeval delay = { 10 };
158     printf("Caught an interrupt signal; exiting cleanly in one second.\n");
160     event_base_loopexit(base, &delay);
161 }
163 static void user_event_cb(evutil_socket_t, short events, void *user_data) {
164     printf("user event %04x fired!!!!!\n", events);
165     struct event_base *base = (struct event_base*)user_data;
166     /*event_base_dump_events(base, stdout);*/
167 }
169 static struct event* init_user_event(struct event_base* base) {
170     struct event *ev_user = NULL;
171     struct timeval timeout = { 20 };
172     ev_user = event_new(base, -1, EV_TIMEOUT|EV_READ, user_event_cb, base);
173     /*event_add(ev_user, &timeout);*/
174     return ev_user;
175 }
177 static void* user_event_proc(void* data) {
178     printf("start user event thread\n");
179     struct event *ev_user = (struct event*)data;
180     char buf[512] = {0};
182     while(1) {
183         fgets(buf, sizeof(buf), stdin);
184         printf("read %d bytes from stdio, now fire user event\n"0);
185         event_active(ev_user, EV_READ|EV_WRITE, 1);
186     }
187     return NULL;
188 }

Sunday, October 21, 2012

communication between android widget and application

Widget is a convienent feature in android that gives users quick access to frequently used application functions. A example is the power control widget. It helps us quickly toggling accessories such as WIFI, GPS power to conserve battery, without tedious operations.
android power control widget
When we plan to provide widget in our own application, an important thing to think about is the communication model between the widget and application. In a simiplified manner, the commucation model is shown below.
communication model diagram
The widget is shown on home screen(which is a AppWidgetHost), and user can interact(e.g., touch) with the widget. The result of the interaction is either showing an activity to the user to display more information, or controlling the state of a background service. Meanwhile, the background service may proactively update the widget to inform user current state. The communication model is bidirectional.

Launch activity from widget

To launch an activity from widget, we can use RemoteViews's setOnClickPendingIntent method to set a intent for the target button. Once the button is clicked, the intent will be sent to start desired activity. The snippet below shows how to do this in AppWidgetProvider.
 1 import android.app.PendingIntent;
 3 import android.appwidget.AppWidgetManager;
 4 import android.appwidget.AppWidgetProvider;
 6 import android.content.Context;
 7 import android.content.Intent;
 9 import android.widget.RemoteViews;
11 public class GestureAppWidgetProvider extends AppWidgetProvider {
13         public void onUpdate(Context context, AppWidgetManager appWidgetManager, int[] appWidgetIds) {
14             final int N = appWidgetIds.length;
16             // Perform this loop procedure for each App Widget that belongs to this provider
17             for (int i=0; i<N; i++) {
18                 int appWidgetId = appWidgetIds[i];
20                 // Create an Intent to launch Activity
21                 Intent intent = new Intent(context, MainActivity.class);
22                 PendingIntent pendingIntent = PendingIntent.getActivity(context, 0, intent, 0);
24                 // Get the layout for the App Widget and attach an on-click listener
25                 // to the button
26                 RemoteViews views = new RemoteViews(context.getPackageName(), R.layout.appwidget);
27                 views.setTextColor(R.id.startRecord, Color.GREEN);
28                 views.setOnClickPendingIntent(R.id.startRecord, pendingIntent);
30                 // Tell the AppWidgetManager to perform an update on the current app widget
31                 appWidgetManager.updateAppWidget(appWidgetId, views);
32             }
33         }
34 }

Send message to service from widget

The skeleton of code to send message to a service is pretty much the same as the snippet above. The change we need to make is substitute PendingIntent.getActivity with PendingIntent.getBroadCast. The result is once we clicked the button, a broadcast Intent will be sent, and our AppWidgetProvider(which is a subclass of BroadcastReceiver) will get this intent. Thie AppWidgetProvider runs in our application's process, so it can send the message to our service with StartService.
 1 public class WeatherWidgetProvider extends AppWidgetProvider {
 2     public static String REFRESH_ACTION = "com.example.android.weatherlistwidget.REFRESH";
 4     @Override
 5     public void onReceive(Context ctx, Intent intent) {
 6         final String action = intent.getAction();
 7         if (action.equals(REFRESH_ACTION)) {
 8             // send message to background service via startService here
 9             // ..............
10         }
11         super.onReceive(ctx, intent);
12     }
14     @Override
15     public void onUpdate(Context context, AppWidgetManager appWidgetManager, int[] appWidgetIds) {
16         for (int i = 0; i < appWidgetIds.length; ++i) {
17             final RemoteViews rv = new RemoteViews(context.getPackageName(), R.layout.widget_layout);
18             rv.setRemoteAdapter(appWidgetIds[i], R.id.weather_list, intent);
20             // Set the empty view to be displayed if the collection is empty.  It must be a sibling
21             // view of the collection view.
22             rv.setEmptyView(R.id.weather_list, R.id.empty_view);
24             // Bind the click intent for the refresh button on the widget
25             final Intent refreshIntent = new Intent(context, WeatherWidgetProvider.class);
26             refreshIntent.setAction(WeatherWidgetProvider.REFRESH_ACTION);
27             final PendingIntent refreshPendingIntent = PendingIntent.getBroadcast(context, 0,
28                     refreshIntent, PendingIntent.FLAG_UPDATE_CURRENT);
29             rv.setOnClickPendingIntent(R.id.refresh, refreshPendingIntent);
31             appWidgetManager.updateAppWidget(appWidgetIds[i], rv);
32         }
33         super.onUpdate(context, appWidgetManager, appWidgetIds);
34     }
35 }

Update widget from service

To update a widget from service, we can send a broadcast message from the background service to AppWidgetProvider. Once the AppWidgetProvider receives the message, it tries to fetch current state and calls notifyAppWidgetViewDataChanged function to refresh the widget.
public void onReceive(Context ctx, Intent intent) {
    final String action = intent.getAction();
    if (action.equals(SHOW_NEW_DATA_ACTION)) {
        final AppWidgetManager mgr = AppWidgetManager.getInstance(context);
        final ComponentName cn = new ComponentName(context, WeatherWidgetProvider.class);
        mgr.notifyAppWidgetViewDataChanged(mgr.getAppWidgetIds(cn), R.id.weather_list);
    super.onReceive(ctx, intent);


android WeatherListWidget sample
Introducing home screen widgets and the AppWidget framework
android appwidget source code
android appwidget service source code

Friday, August 31, 2012

debug linux kernel with gdb

People often use gdb to debug user mode applications, which is a convenient debugging means. It's also possible to use gdb to debug linux kernel and drivers with the help of kgdb. This page is a good tutorial for how to use kgdb.

Benefits of using gdb to debug kernel

  • It helps us understanding linux internals. In linux, it's very common to see structures with a lot of function pointer members being passed around. And it's not easy to find out where and how these function pointers are actually called by reading code. By setting breakpoint on the function, we can easily find how linux call into it.
  • It saves the debugging time. If we only debug by printk, we usually have to compile and deploy linux kernel multiple times to fix a minor bug. It's more efficient to debug if we can step through the code and see all variables' value in real time.


As the precedent document illustrates, to enable kgdb for a kernel, we need to:
  • Enable kernel config options for kgdb
  • Provide a polling tty driver for the kgdboc I/O driver.
  • Set linux boot argument to instruct linux kernel use our kgdb I/O driver

How to complete a kgdb I/O driver

Linux contains a kgdb I/O driver, kgdboc, short for kgdb over console. It's acutally a thin driver that relies on low level hardware driver supporting polling operation. This low level driver must be implemented by us.
To complete the polling driver, we need to implement poll_get_char and poll_put_char callbacks in the UART driver. There is a good example for us to follow in linux source code: 8250.c.

How to debug if there is only one serial port

kgdboc is designed to work when there is only one serial port on our board. The serial port can be used as primary console as well as the communication channel with gdb. In this case, we should first connect our serial port client (e.g., kermit) to the console and input 'echo g > /proc/sysrq-trigger' command to break into linux kernel. Now linux should halt and wait for a gdb client to connect. Then we exit the serial port client process and start a gdb client to connect to the linux on the same serial port. It's time-division multiplexing on the serial port.
The agent-proxy make the process even easier. agent-proxy is a tty to tcp connection mux that allow us connect more than one client application to a tty. By using it, we can run the serial port client and gdb process simultaneously. aa

How to debug linux initialization code

If we specify kgdbwait parameter in kernel boot args, the kernel will halt automatically during the initialization process and wait for a gdb client to connect. There are several things to note:
  • The kgdb core tries to break the execution as soon as a kgdb io driver is registered, which is done while the kgdboc module is initialized. As a result, it's necessary to set kgdboc module as built-in, rather than a module.
  • Our UART driver must be initialized before the kgdboc driver. Or the kgdboc driver will fail to initialize. Becuase there isn't a reliable way to specify loading order for built-in modules at the same level, it's better to specify our UART driver at a precedent level than kgdboc, for instance, fs_initcall.
  • The module initialization is called through this call stack: start_kernel -> rest_init -> kernel_init -> do_basic_setup -> do_initcalls. So, we can't debug code earlier than do_initcalls.

How to debug loadable module

When we need to debug a loadable module, we should add the ko file with symbol information to gdb with add-symbol-file command. We must provide the module's load address explicitly. How can we find out where the module is loaded? After we've insmod the module, we can find out the load address of the module by either read the /proc/modules pseudo file or use info shared command in gdb. But what if we need to debug the module_init function? It will be too late to set breakpoint after we've alreay loaded the module to find out its load address. We can solve this dilemma by setting a breakpoint in sys_init_module after the load_module function returns. And we can find out the module's load address with p mod->module_core command in gdb. We can add symbol file at this point and set a breakpoint in the actual module_init function. Or we can set a breakpoint in do_one_initcall.

Saturday, July 7, 2012

improve c++ autocomplete in vim with clang-complete plugin

clang-complete is a powerful vim autocomplete plugin for c/c++ developers. Unlike the famous OmniCppComplete plugin, which makes use of ctag database to implement completion, the clang-complete plugin take advantage of the clang compiler. With the help of compiler, far more knowledge can be gained than the tag matching method. So the plugin can achieve a very precise completion, just like how visual studio does.

clang-complete mode

1. executable mode

In this mode, each time we trigger a completion (Ctrl_X Ctrl_U) in vim, the plugin will invoke the clang executable on the specified position in source code, then read and parse the executable's output to use as the candidates list.

2. library mode

In this mode, the plugin will run a python script to invoke the libclang library to get the candidates list. As the author indicates, the libclang library employs cache mechanism and runs much faster than the executable mode. I also observed another difference. On windows, the clang.exe may fail to compile our source code and returns a non-0 exit code. In this case, the plugin only returns an empty list, even though it may be able to produce a correct list. But the library mode doesn't have this limitation. So, it's the recommended way to use.

how to use it


By following instructions in this wiki page, the plugin works very well on ubuntu. The only thing wasn't mentioned in the documenataion is that in order to use library mode, we must install the libclang-dev package.


The experience of using the plugin of windows is much more difficult.

1. Get a windows version clang

Since new version (v3.1) of clang can be compiled with visual studio, it's not difficult to compile the clang.exe and libclang.dll myself. Just note that though the clang can run on windows and can compile our c++ code, it can't performing linking. That's fair enough to simply use clang for our purpose.
And you can get the binaries I compiled here, for free :).

2. Get right output in executable mode

The clang.exe on windows outputs a lot of message to stderr, which are not interested by the plugin at all. Because the plugin uses system() function to invoke clang.exe, and the function will automatically redirect stderr to stdout by default. The author of the plugin suggests we can use let g:clang_user_options = '2> NUL || exit 0"' to get rid of stderr output. But it doesn't work for me. And I finally come up with this patch to fix the problem.
diff --git a/plugin/clang_complete.vim b/plugin/clang_complete.vim
old mode 100644
new mode 100755
index 7cb0fe0..6db164d
--- a/plugin/clang_complete.vim
+++ b/plugin/clang_complete.vim
@@ -421,6 +421,8 @@ function! s:ClangCompleteBinary(base)
     return {}
   let l:escaped_tempfile = shellescape(l:tempfile)
+  let l:shellredir_orig = &shellredir
+  let &shellredir ='>%s 2>NUL'

   let l:command = g:clang_exec . ' -cc1 -fsyntax-only'
         \ . ' -fno-caret-diagnostics -fdiagnostics-print-source-range-info'
@@ -429,6 +431,8 @@ function! s:ClangCompleteBinary(base)
         \ . ' ' . b:clang_parameters . ' ' . b:clang_user_options . ' ' . g:clang_user_options
   let l:clang_output = split(system(l:command), "\n")
   call delete(l:tempfile)
+  " restore original shellredir
+  let &shellredir = l:shellredir_orig

   call s:ClangQuickFix(l:clang_output, l:tempfile)
   if v:shell_error

3. Make python ctypes module to work in library mode

The plugin uses ctypes module to invoke the libclang.dll. Due to a mysterious reason, the ctypes module can't be loaded successfully when run from embedded python in vim. I got the the "ImportError: No module named _ctypes" error and the plugin failed to work. But when I tested from a standalone python instance, the ctypes module worked well. After some debugging, it seems the embedded python doesn't search {python_root}/dlls directory to load _ctypes.pyd file, but the standalone python does. So I take a nasty method to solve the problem by copying the _ctypes.pyd to the clang_complete's plugin directory, right besides libclang.py file.

Saturday, June 30, 2012

got multiple singleton instances

We meet a subtle bug while adopting singleton design pattern in a project. The singleton class creates multiple instances.
The image below depicts the dependency relationship between different modules.

The executable depends on two dynamic libraries, dynamic_lib1 and dynamic_lib2. And both dynamic_lib1 and dynamic_lib2 depends on static_lib. There is a singleton class in static_lib. dynamic_lib1 and dynamic_lib2 use get_instance method to retrieve the instance of the singleton class.
The basic skeleton of the singleton class is shown below:

 1 // singleton.h
 2 #pragma once
 4 class __declspec(dllexport)  singleton
 5 {
 6 public:
 7     singleton(void);
 8     ~singleton(void);
10     static singleton* instance;
11     static singleton* get_instance();
12 };
// singleton.cpp
16 #include "singleton.h"
17 #include <iostream>
19 singleton* singleton::instance;
21 singleton::singleton(void)
22 {
23 }
25 singleton::~singleton(void)
26 {
27 }
30 singleton* singleton::get_instance()
31 {
32     // lock here
33     if(!instance)
34     {
35         instance = new singleton();
36     }
37     // unlock here
38     return instance;
39 }

But when we run the executable, we're surprised to find that the the use of get_instance method in dynamic_lib1 and dynamic_lib2 doesn't share the same instance.

After thinking about it carefully, it's clear that the bug is caused by we use static_lib3 as a static library. When we compile dynamic_lib1 and dynamic_lib2, they both link with static_lib, and each get a separate copy of the singleton::instance in data section.

But be aware that the behavior is compiler specific. The singleton is still singleton when I tested with gcc. And the singleton got created multiple instances when I tested microsoft's C++ compiler and apple's developer tools.