I need to backup my Ubuntu Jaunty

Recently purchased laptop was now somewhat out of order. I said somewhat because it is still working, but the fan is not running even when my CPU is very hot. Anyway, I suddenly needed to backup my entire installation of Ubuntu Jaunty for later recovery. So I came up with two options.

a. Copy all files and directories
b. Copy the image of a disk or logical partition

The first option is cool, if someone want to practice old style procedure using UNIX like commands in the open terminal. I could find instructions in “Linux Complete Backup and Recovery HOWTO” and “Howto: Backup and restore your system”.

However what I wanted was simple and easy tool to backup and restore my OS image in a snap! as with Ghost. Furthermore, I wanted to backup other OS or disk images such as the disk I assigned for Windows Vista and shared files between OS’s. That was the second option. I found three commonly used tools in Linux.

a. G4L (Ghost for Linux)
b. Partimage (Partition Image)
c. CloneZilla

Any one of these three tools looked promising to me. G4L has many options from copying only portion of file to network backup. CloneZilla looked perfect choice if I needed to clone Jaunty from central management server to many company-wide desktop as in Enterprise Desktop Management System. However, I chose Partimage because it seemed one of the most popular(G4L seemed also popular) and simplest in terms of menu and usage. Most of all, I install directly from Synaptic Package Manager without any adjustment.

After I selected my tool, I needed to make bootable version of Linux to be used at restore time. Most of the cloning tools have its own CD image for this purpose with small and light version of Linux. But I didn’t want to create CD because it’s just not as portable as my USB thumb drive which I carry most of the time. Even some Laptop does not comes with CD/DVD drive, right? So, I decided to created portable USB Linux for emergency. There might be several scenario to prepare emergency USB Linux.

1. Making Ghost like USB Linux from existing USB Linux.
If someone already have Debian compatible Linux on USB thumb drive, it is easy. After booting from USB Linux, invoking

sudo apt-get install partimage

command will make your thumb drive to be a Ghost like tool. (I also recommend you to install gparted.)

2. Creating new USB Linux using “USB Startup Disk Creator”
If you do not have USB Linux, and if you want to install same Ubuntu as you're using now, it is still easy. Using “USB Startup Disk Creator” in System/Administrator menu will solve your problem easily. Maybe you already used this tool, when you install Ubuntu to your Laptop without CD/DVD drive. All you need is ISO image of Ubuntu and USB drive with at most 2G. After you select source image and target USB disk, pressing “Make Startup Disk” will create USB Linux. All you need is reboot with the USB disk and install Partimage using command above.

From Better than nothing

3. Moving RescueCD to USB thumb drive
This option need several steps.
1) Download CD image
Another way to create Ghost like tool is using CD image provided by tool creator’s web site. Because I selected Partimage, I could download CD image called RescueCD from web site. (Or, just click RescueCD to download)

2) Create partition
Then I needed to make new partition to install Linux on my USB thumb drive. Because fdisk does not provide resize feature, and parted still does not accept creating ext3 type, I prefer gparted. And maybe, above all the reasons, nice GUI of gparted attracted me most. If you do not have gparted already, you can install it using following command.

sudo apt-get install gparted

There are options, though, when you create partition for USB Linux.

a.FAT32
b.ext2 or ext3

Most people choose FAT32 when creating a new partition for USB drive. Because it only use single allocation table in top of the file system rather than tree structured nodes, it usually seems more robust and safer for removable disk. However, it is quite inefficient when it comes with small sized files. So, I created ext3 type partition for my USB Linux with 250M in size. (because the size of image is around 234M). If you plan to use FAT32, you may need another 100M or more.

3) Mount CD image and USB drive
Once partition is ready, mount downloaded image and USB drive using following command.

sudo mount -t iso9660 -o loop=/dev/loop3 ./systemrescuecd-x86-1.1.7.iso ~/mnt1
sudo mount -t ext3 /dev/sdb2 ~/mnt2

4) Copy all files in CD image to USB drive
sudo cp -af ~/mnt1 ~/mnt2

5) Make USB Linux bootable
Because CD image comes with isolinux, which is boot loader for CD, more precisely ISO image, and because syslinux can only be used with FAT32 file system, I had to use Grub to make my USB Linux bootable.

grub-install --root-directory=/media/disk/ /dev/sdb2
(where /media/disk is mount folder and /dev/sdb2 is device for USB disk.)

Then, I was able to boot from USB Linux and to use Partimage.

4. Using DSL (Damn Small Linux)
Another Debian based Linux and one of the smallest is DSL. It only occupy 50M of disk space, but only comes with X. So, unless you are not going to use gparted which is partition tool of my choice, this might be another option. I believe that the preparation procedure is similar to what I described in 3. However, I love gparted and I want it to be installed in my USB Linux. So I did not tried this option.

From Better than nothing

Backup and restore using Partimage was really simple, once I booted from USB. After I started Partimage by typing "partimage" and pressed return, the above screen was shown. Then, I typed in the location where to store backup image after “image file to create/use,” and pressed F5. (Backup is default choice)

From Better than nothing

I kept the default options and pressed F5, then entered some comment. Finally, it started to backup my Jaunty. The result image was almost halt the original size. Restore from image is almost as simple as backup. After I entered image name and selected “Restore partition from image file” using TAB and arrow keys, I was able to recover Jaunty from stored image.

Finally you need to backup your partition table in safe place to prepare recovery from the lost of entire disk, not from single partition.

sfdisk -d /dev/hda > /media/disk/saved_partition_table

If you loose entire partition table, recover it from this file using following command.

sfdisk /dev/hda < /media/disk/saved_partition_table

You can find more information from "Partimage-manual Backup-partition-table."

Simple object is enough for simple Ajax

In modern web environment, Asynchronous Javascript And XML, a.k.a. Ajax, is common technique or technology to make web more dynamic. There are many Ajax frameworks in the open source community such as GWT from Google, Django, Dojo, and many more. However, in the bottom of every Ajax framework, there is single Javascript object called XMLHttpRequest. It is this browser object that make these bulky frameworks request data to sever, asynchronously.

So, if simple text-based communication or simple XML document that does not need heavy framework to manipulate every inches of documents is required, you may not need to embed bulky framework in your web program. You may only need to know how to use XMLHttpRequest.

The following is one of the example to show how to use XMLHttpRequest. It is simple constructor function that you can reuse when you need Ajax. Because of the length of the article, I’ll introduce client side first.

function AjaxClient(){
    this.getXmlHttpRequest = function(){
        if (window.XMLHttpRequest) 
            return new XMLHttpRequest();
        try {
            return new ActiveXObject("Microsoft.XMLHTTP");
        } 
        catch (e) {
            alert("The browser is not supported!");
        }
        return null;
    }
    this.sendRequest = function(url, method, data, handler){
        var request = this.getXmlHttpRequest();
        if (request != null) {
            request.onreadystatechange = function onStateChange(){
                if (request.readyState == 4) {
                    if (request.status == 200) {
                        handler(request.responseText);
                    }
                    else 
                        alert("Problem recieving data:" + request.statusText);
                }
            }
            if (method == "GET") {
                request.open("GET", url + "?" + data, true);
                request.send(null);
            }
            else if (method == "POST") {
                request.open("POST", url, true);
                request.setRequestHeader("Content-length", data.length);
                request.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
                request.setRequestHeader("Connection", "close");
                request.send(data);
            }
        }
    }
}

The first method in AjaxClient is GetXmlHttpRequest() to create XMLHttpRequest. Before version 7, Internet explorer need ActiveX component to create XMLHttpRequest object. I believe that many people still use Internet explorer 6 or even 5, and that it may still need to discriminate older version of IE's from other browsers.

Before going further, let’s look at the XmlHttpRequest.

interface XMLHttpRequest {
  // event handler
  attribute EventListener onreadystatechange;

  // state
  const unsigned short UNSENT = 0;
  const unsigned short OPENED = 1;
  const unsigned short HEADERS_RECEIVED = 2;
  const unsigned short LOADING = 3;
  const unsigned short DONE = 4;
  readonly attribute unsigned short readyState;

  // request
  void open(in DOMString method, in DOMString url);
  void open(in DOMString method, in DOMString url, in boolean async);
  void open(in DOMString method, in DOMString url, in boolean async, in DOMString user);
  void open(in DOMString method, in DOMString url, in boolean async, in DOMString user, in DOMString password);
  void setRequestHeader(in DOMString header, in DOMString value);
  void send();
  void send(in DOMString data);
  void send(in Document data);
  void abort();

  // response
  DOMString getAllResponseHeaders();
  DOMString getResponseHeader(in DOMString header);
  readonly attribute DOMString responseText;
  readonly attribute Document responseXML;
  readonly attribute unsigned short status;
  readonly attribute DOMString statusText;
};

from “The XMLHttpRequest Object”

As can be seen from definition, XMLHttpRequest has three methods, open(), send() and abort() in request side, two methods in response side, one event handler, and five read-only attributes including readyState. open() accept at least two arguments. The first argument method is one among “GET”, “POST”, “HEAD”, “PUT”, “DELETE”, “OPTIONS.” The second argument is URL of server. If you use less than 255 bytes(which I believe is almost the least length of GET URL that an old browser can permit) of data and server URL, you can use “GET” and embed necessary data in URL as key and value pairs. If you want to send more than 255 bytes of data, you use “POST” and embed data into the body of HTTP protocol through send(). More details can be found in W3C documents.

Back to my example, sendRequest() accepts URL of server, method of request which is either “GET” or “POST”, parameters to send, and handler which is your code to run after data is received from server.

Because handler will be wrapped by already prepared code, which monitors readyState to be “4 (DONE)” and status be “200 (OK),” your handler focus on what to do with data received. As in the sample below.

function myHandler(receivedData) {
 document.write(receivedData);
}

The usage is simple. After include AjaxClient in your script, you create AjaxClient() object, and invoke sendRequest() like this.

client = New AjaxClient();
client.sendRequest(“http://localhost:8080/test”, “POST”, “Give me something”, myHandler);

Once call is successfully finished, you can see what server has sent in the browser.

It's odd working with IE - Part 2. comma

2. comma makes big difference in IE.
Like most technical people out there, I used to be chased by time. So, it happened from time to time to insert unwanted character by hitting keyboard without notice when I hastily moving around numerous codes. However, IE stop all my script, once, I mistakenly put comma ‘,’ in the declaration of Javascript object as follow;

<script type=”text/javascript”>
var test = {
    abc: new Array(),
    …
    def: function(arg1, arg2) {
    …
    },
    ghl: function(arg3) {
    …
    },
}
…
</script>

Because I prefer use Aptana when editing HTML pages, I did not notice what’s happened. (By the way Aptana does not provide strict error checking when it comes to Javascript) Furthermore, because I usually develop using Firefox under Linux environment, even in the first running test it goes without noticed. (Firefox seemed generous to people like me.) However, when I loaded in IE6 (I only installed IE version 6 in Linux) using sometime later, I was surprised to see that not a single line of Javascript was running from the web page, and no error message was shown.

It is true that I still did not find an editor with perfect syntax or semantic checking, in the market. Furthermore, unfortunately, Unlike C++ or Java, it does not comes with error checking program such as compiler. So, it comfortable to check such error.

There are, however, ways to avoid this pitfall.

a. Using JSLint: just as link does for C program, JSLint can hint erroneous or even under-performing code. Online version can be accessed using http://www.jslint.com/, plug-in version can be found in http://www.rockstarapps.com.
b. Using multiple editor: Just to double check errors that may remain in source code, reviewing code with another editor(editor comes with ATF, Ajax Toolkit Framework, for example) with more rigorous error checking is also helpful.

P.S. By the way, because I’m a heavy user of Eclipse, editor or plug-in is viewed from Eclipse environment.

It's odd working with IE - Part 1. <td>

It took me a while googling to find why height in is not rendered as I expected in IE (Internet Explorer), when I use code like;
<style>
…
td.under {
…
height: 9px;
…
}
</style>

<td class="under" onclick="insertRow(this, verAttribute)" onmouseover="onSideOver(this)" onmouseout="onSideOut(this)">
<img src="img/add.gif">
</td>

But, it didn’t help much. I find the answer in the Technet of Microsoft.

Many people have already knew that IE is working differently than other browser. Well, actually, most browsers are acting somewhat differently from each other. Fortunately, many of the differences in terms of using HTML tags or DOM elements are rather perceivable, if I use informative tools like Aptana. Still, most of the tools do not describe why <td> is rendering differently in IE. So I decided to add any findings in this column starting from <td>.

1. The height of <td> does not seem to work in IE.
The size of <img> is not easily controlled unless I wrapped the image inside <td>, especially in IE. In many cases, if I use “Source Formatter” or developer’s habit to separate <td> and </td>l; with CRLF, which means separated by new line, another DOM object is added. (This can be checked easily using firebug add-on.) And this cause IE to behave differently when deciding height of <td>.

If it happens, even if I enforce the height of cell with height attribute as in above example, IE re-calculate height of the cell using another new line. Which make the cell grow to the size of text. To avoid this and if you’re really considering to use your HTML code also in IE, <td> and </td> should not be separated by new line. So, if I fix above code like this;

<td class="under" onclick="insertRow(this, verAttribute)" onmouseover="onSideOver(this)" onmouseout="onSideOut(this)"><img src="img/add.gif"></td>

Then, the height of the cell in IE is rendered the same height as in Firefox.

My laptop can access any files in my desktop

After I installed Linux on my laptop, I was able to connect and use any files in my desktop, just like old days with servers. Using SSH, Secure SHell which installed as default, I could connect to desktop and control my work from remote. However, for my laptop to accept connection, I installed openssh-server by typing

sudo apt-get install openssh-server

After that I need to alter configuration file to protect my desktop. By changing the file, /etc/ssh/sshd_config, I could block any other connection except from my laptop. For example, if I uncomment “ListenAddress” and assign only one IP address, I could block virtually all connection attempt from outside. And if I change “PermitRootLogin” to false and add only one user name in “AllowUsers,” I could permit only myself into desktop as non root account. Once I finish my configuration, I could connect my desktop by typing

ssh 192.168.2.1 -l username

After I looked for the path where my game file was located, I could copy the file into my laptop by typing

rcp username@192.168.2.1:/home/sweethome/Desktop/yofrankie_bge.zip .

from another terminal in my laptop. Then I decided to watch movie in my desktop folder from my laptop. Because I didn’t want to move movie file into my laptop, I mounted the folder in one of my local folder. To do that, I first installed Secure SHell File System, sshfs, by typing

sshfs username@192.168.2.1:/media/disk-1/ ~/mnt

Of course, I needed to create mount point ~/mnt before I mount. I also needed to synchronize some of my folder with my desktop. So I installed Unison both on my laptop and server by typing

apt-get install unison

Additionally, I installed GUI interface for my laptop by typing

apt-get install unison-gtk

After the installation, I ran Union client under “/Applications/Accessories" menu. When Unison ask me to enter root 1 and root 2 folders to sync, I entered one local folder

/home/laptop/memos

and one desktop folder name like this

ssh://username@192.168.2.1//home/username/memos

Then the GUI came up and I could synchronize memos. And I don't have to worry about sharing my folder any more.

More information can be found in following URL’s:
http://www.openssh.com/ (For Secure SHell)
http://fuse.sourceforge.net/sshfs.html (Secure SHell File System)
http://www.cis.upenn.edu/~bcpierce/unison/ (Unison)

It’s nice to have Ubuntu Jaunty for my new laptop

My new laptop was finally arrived. Because of the recession, most laptops were selling in lower price than they’re expected in usual market. So, I got my new one in one third price of what I paid for the old one three years ago. Ever since I decided to quite my job to join CMU, I’ve gradually moved to open source and free softwares. As a result, I seldom boot into Windows system in these days. And finally, I decided to use Ubuntu as my main OS rather than Windows. Ironically, I used to use Windows as main OS and Linux as secondary or working environment while I worked for Unix company.

However, it was not as easy as Windows system to make my machine running. Thanks to new Jaunty, my work was reduced a lot. I wanted to record how I did to help others and myself.

1. Installation considerations
When I was working for the company, I used to divide a cluster of drives in logical partitions to make the system fast and extensible. Dividing consideration was usually i) which portion of system would need faster access, ii) which would be increasing most, iii) which would need to be replaced, iv) which parts would be simultaneously accessed, v) what’re the nature (OS core, DB, Log, File holder, etc), and so on.

So, the first thing to consider was dividing hard disk. To do that I asked myself, “what am I doing with laptop?” “What kinds of files that I’m going to hold with my laptop?” Well, most of the time, I make some program and write documents. From time to time, I watch movies, enjoy some music and collect some data. Usually with Ubuntu but sometimes with Windows.

Then I thought about what device I’m going to use with my laptop? Since I move mostly from Windows to Ubuntu, I need to support my Treo, iPod, as well as bluetooth mouse I use to use. Of course I need to take advantage of internal device of Dell Vostro 1330.

2. Installation
Most of installation is automated in Ubuntu just like Windows (or maybe simpler than Windows). The only thing I have to seriously consider during installation was partitioning. With the answers to my questions in mind, I decided to divide my hard drive into five parts.

a. Recovery (Dell allocated),
b. Windows OS, (shrink into half to share common and casual files)
c. Ubuntu 9.04, (at most 40G to hold only OS and open source software)
d. swap for Ubuntu (3G same as my laptop's memory)
e. Data (the rest for holding most of projects files and home folders under /home)

I divided Ubuntu into two / and /home to separate data from program because I didn’t want to be disturbed by the possible re-installation of Ubuntu in the future. So, I chose manual partitioning and first resize Windows partition. (Thanks to partman, I can safely resize Windows partition by just editing the size.) And created new partitions as I planned. It manual installation creating swap partition is necessary, and usually make same size as memory. (If needed, swap can be increased during operation using swap command like mkswap, swapon. However, unless I planned to use large program like Oracle DB, increase is seldom needed)

That’s all I need to do during installation.

3. Bluetooth
a. Microsoft Notebook Mouse 5000
Because of small touch pad, I really wanted to use mouse as soon as possible. Fortunately, I could see bluetooth symbol in the panel. That means I don’t have to worry about adding bluetooth stack into kernel. So, I was ready to pair my devices.

From Better than nothing

I first clicked bluetooth symbol and select “Setup new device.” After wizard dialog appeared, I moved into “Device Search” section by pressing “Forward.” Either I choose "Automatic” or "Use Fixed PIN Code" with 0000, in this section, pairing result said successful. However, the LED in MS Mouse was still blinking. Meaning MS Mouse was not paired, yet. I found answer in a corner of forum somewhere in launchpad. After some struggling, I found correct procedure.

First, I needed to install bluez-compat package with command;

sudo apt-get install bluez-compat

Then turn off and on MS Mouse. Again make mouse into pairing mode by pressing small button in bottom. When I saw LED blinking, type

hidd –search

then the LED of mouse was gone, means that you are finally paired. The mouse keep paired even after reboot or resumed.

b. The Treo755p.
The next challenge was smart phone, Palm Treo755p. I said challenge because I suffered enough last I tried with my desktop. However, this time, I was actually surprised to see that new gnome-pilot was supporting bluetooth. I used to follow guide like “Palm Bluetooth How-to to make my Treo to sync with Evolution. It was much easier now in Jaunty, Ubuntu 9.04. My procedure was simple.

First, pair palm and laptop. As before, just followed wizard after pressing bluetooth symbol in pannel. This time I needed to use “Fixed PIN code” and my PIN number to pair with Treo.

From Better than nothing

Next, I launched gnome_pilot by choosing the menu, System\Preference\Palm OS Device, and followed the wizard. Some errors came but ignored the message and follow the instruction, except selecting bluetooth as communication device. That’s it. Now I was able to sync with Evolution through bluetooth. I selected usual conduits to sync with contact, calendar, menu and todo data in Evolution.

c. iPod.
I chose Banshee to manage my music in iPod. It looked that Songbird was still unstable somehow, and Amarok asked me to install KDE libraries. After the installation, everything works fine, except album photos. It was not a big deal to me now, so I moved on to the next.

This may help to troubleshoot Eclipse

When Eclipse came out to public as free, I was using various other IDE's to make programs. Most of them was not free, so I could not use them as I wanted, unless it is provided to me. Naturally, I fell in love with Eclipse when I first saw it, althgouth it was not as fancy as commercial IDE's.

Eclipse now-a-days is a lot stronger, faster, and flexible than before. But sometimes it does not work as it supposed to be, as least to the users, especially if you've never encountered the problem before.

1. Eclipse does not start.
It happens time to time, when you install a newer version of a new installation of Eclipse for some reasons. (I'll grab some image or copy of error message, if I make one)

The first thing to check is, whether you are using the correct version of Java? It may be easy, since Eclipse will tell why it is not working. But if you're working with lots of plug-ins and updated regularly, you may not notice what's happening inside Eclipse. For example, under Eclipse 3.4, some plug-ins, for example latest version of CDE, require Java 6 to work properly, instead of Java 5 which is required in Eclipse 3.4. But Eclipse does not say much about that on screen. Checking Java version is first thing to do.

The next thing to check is multiple Java Runtime. If Eclipse does not start, even if you checked and confirmed correct version of Java is installed on system, you may have multiple version of Java Runtime. I saw this case in Microsoft Windows environment from time to time. It is possible that you installed Java Runtime by the request of web site to run some applets or programs in the past and forgot about that. In that cases, Eclipse does not run no matter how many times you remove and install new version of Eclipse. The simple remedy is to add following line in eclipse.ini file which exist right under ./eclipse directory. (Or you may just remove unnecessary Java Runtime.)

vm
C:\Sun\SDK\jdk\bin\javaw.exe

This is just an example, and the path of "javaw.exe" is depend on where you've installed your Java. if you're using Linux, it will be like

vm
/usr/share/java/jdk1.6/bin/java

You have to remember that path must be placed in the right next line. And don't put it between the pair of arguments. So it may be better to place it at the end of file.

2. SVN is not working.
This is another disturbing and time consuming case, if you are not familiar with error. This happens mostly in Linux environment, and the main cause is JavaHL native library. Typical error message is “Unable to load default SVN client,” if you're using Subclipse. If you're using only remote repositories and you have more than on client installed, the solution is simple.

Change the default SVN client.

You can change SVN client by changing Client field, under the menu of “\Window\Preferences” and under \Team\SVN category. Or if you're using "Subversive," you can change SVN connector under "SVN connector" tab. All you have to do is select other SVN client than “JavaHL(JNI),” such as SVNkit.

But if you're using local repositories and want faster access, you may have no other choice but to use "JavaHL". The procedure is as follow.

First, you need to install libsvnjavahl library, because "Subclipse" or "Subversive" does not comes with it. You can search the package name like libsvn-java using

sudo apt-cache search libsvn

Then install package using

sudo apt-get install libsvn-java

then, you may have to check the version of libsvn you've installed, in case you do not remember. Type

sudo apt-cache showpkg libsvn-java

If you have a package manager like “Synaptic Package Manager,” you may install and check the version inside package manager.

Current version of Ubuntu comes with 1.5.4 version of libsvn-java. Thus you can only install 1.4.x version of subclipse for Eclipse 3.2, 3, or 4. If you want 1.6.x version of subclipse, you need higher than 1.5.x version of libsvn libaray.

Next step is as before, modify eclipse.ini. This time you need to pass an argument to Java VM by adding following line

-Djava.library.path=/usr/lib/jni

and this should be placed below “-vmargs” So, the tuple will be like

-vmargs
-Dosgi.requiredJavaVersion=1.5
-Djava.library.path=/usr/lib/jni

Then, you can check the client in the preference, under the menu of “\Window\Preferences” and under \Team\SVN category. If it is working, "JavaHL" and version number will be displayed in client combo box, if not, it will say “JavaHL(JNI) Not Available.”

If that happens your Eclipse is not loading eclipse.ini file you modified when it boot up. And example of this case is when you've registered Eclipse in the menu with different working directory. Then all you have to do is find the correct place to put eclipse.ini.

ThreadPool is useful when I treat a group of threads

Crafting and using threads is almost as sensitive as allocating and maneuvering memory, at least to me. So, I spent more time looking at the result of memory check tools like “Valgrind,” or allocated more time to create memory management mechanism when I program with C++. I guess these are some of the reasons why I feel more comfortable and effortless when I using Java to test the logic or algorithm of a program.

Almost all the time when I needed tasks to be executed concurrently, I needed a mechanism to manage group of threads. “ThreadPool” is simple class that take care of a pool of threads which run tasks fed through “Command” interface. This pool is very simple but can be used in most common cases. If I need prioritize the tasks, then changing data structure of queue from list to priority_queue will be sufficient. If I need canceling of a task, I can add search and remove of the task from queue to “ThreadPool” class.

“ThreadPool” initialize specified number of workers by using pthread_create(). After that I can add task using addCommand(). The safeguarding of running task which implemented through interface of “Command” is completely up to user. So, if there are any critical sections inside running tasks, use pthread_mutex to protect them.

Here is source code of “ThreadPool.” First the header,

#ifndef THREADPOOL_H_
#define THREADPOOL_H_

#include <list>
#include <pthread.h>

struct Command {
   virtual void run()=0;
};

void* worker(void* args);

class ThreadPool {
public:
   class ThreadCreationError {};

   ThreadPool(int poolSize): poolSize(poolSize) {
      inErrorStatus = false;
      pRunners = new pthread_t[poolSize];
      bootUp(poolSize);
   }

   virtual ~ThreadPool() {
      shutDown();
      delete[] pRunners;
   }

   void addCommand(Command& com);
private:
   int poolSize;
   bool inErrorStatus;
   bool workerLoop;
   pthread_t* pRunners;
   pthread_attr_t attr;
   pthread_mutex_t queueLock;
   pthread_cond_t signal;
   std::list<Command*> queue;

   void bootUp(int size);
   void shutDown();
   friend   void* worker(void* args);
};

#endif /* THREADPOOL_H_ */

Next, cpp code,

#include "ThreadPool.h"

void ThreadPool::bootUp(int size) {
   workerLoop = true;
   pthread_mutex_init(&queueLock, NULL);
   pthread_cond_init(&signal, NULL);
   pthread_attr_init(&attr);
   pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
   pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
   pthread_setcanceltype(PTHREAD_CANCEL_DEFERRED, NULL);
   for (int i = 0; i < size; ++i) {
      if (pthread_create(&pRunners[i], &attr, worker, this) != 0) {
         if (i != 0) shutDown();
         inErrorStatus = true;
         break;
      }
   }
}

void ThreadPool::shutDown() {
   workerLoop = false;
   pthread_cond_broadcast(&signal);
   for (int i = 0; i < poolSize; ++i) pthread_join(pRunners[i], NULL);
   pthread_attr_destroy(&attr);
   pthread_cond_destroy(&signal);
   pthread_mutex_destroy(&queueLock);
}

void ThreadPool::addCommand(Command& com) {
   pthread_mutex_lock(&queueLock);
   queue.push_back(&com);
   pthread_cond_signal(&signal);
   pthread_mutex_unlock(&queueLock);
}

void* worker(void* args) {
   ThreadPool& pool = *static_cast<ThreadPool*>(args);
   std::list<Command*>& queue = pool.queue;
   pthread_mutex_t& queueLock = pool.queueLock;
   pthread_cond_t& signal = pool.signal;
   bool& loop = pool.workerLoop;
   Command* pWorker;
   while (loop || queue.size() > 0) {
      pWorker = NULL;
      pthread_mutex_lock(&queueLock);
      if (queue.size() == 0)
         pthread_cond_wait(&signal, &queueLock);
      if (queue.size() > 0) {
         pWorker = queue.front();
         queue.pop_front();
         if (queue.size() > 0)
            pthread_cond_signal(&signal);
      }
      pthread_mutex_unlock(&queueLock);
      if (pWorker) {
         pWorker->run();
      }
   }
   pthread_exit(NULL);
}

As before, usage is simple. First, I need to instantiate “ThreadPool” with pool size.

ThreadPool pool(5);

And prepare my task.

struct TestCommand: public Command {
   int val;

   TestCommand(int val): val(val) {}

   void run() {
      cout << "Worker[" << val << "," << pthread_self() << "] is running" << endl;
      sleep(1);
   }
};

Then, run it putting task using addCommand().

TestCommand com(100);
pool.addCommand(com);

Unit test is easy with this small program

Whenever I have time, I try and test new technology with my program. Naturally, tools to revert and test what I developed is essential. I move around various version of programs through SVN and history feature of Eclipse, which is definitely my favorite IDE. In terms of feature test, I often use small program I created. It is very small and as much expandable as I needed. And it is really simple, but also covers hierarchical structure of test cases. The test program is mixed up version of C and C++ to really simplify test. Building such program may not take much time if someone want to create class like this. As such, it is very easy to modify or expand as needed.

Here is UnitTest class.

typedef bool (*_TEST_FUNC_)();

#define BEGIN_TEST   bool _test_result_ = true;
#define END_TEST  return _test_result_;

#define ASSERT(value)\
result = result && (value == ture);
#define ASSERT_EQUAL(value1, value2)\
_test_result_ = _test_result_ && (value1 == value2);
#define ASSERT_NOT_EQUAL(value1, value2)\
_test_result_ = _test_result_ && (value1 != value2);
#define ASSERT_GREATER(value1, value2)\
_test_result_ = _test_result_ && (value1 > value2);


class TestCase {
public:
TestCase(string name): caseName(name) {}
virtual ~TestCase() {}

inline void setTest(_TEST_FUNC_ test) { this->test = test; }
inline _TEST_FUNC_ getTest() { return test; }
inline string getCaseName() { return caseName; }

protected:
string caseName;
_TEST_FUNC_ test;
};

class UnitTest: public TestCase {
public:
UnitTest(string name):TestCase(name) {}
virtual ~UnitTest() {
   for (vector<TestCase*>::iterator itr = cases.begin(); itr != cases.end(); ++itr)
      delete *itr;
}

inline UnitTest& TEST_CASE(string caseName, _TEST_FUNC_ test) {
   TestCase *pTest = new TestCase(caseName);
   pTest->setTest(test);
   cases.push_back(pTest);
   return *this;
}

inline void TEST_BLOCK(UnitTest* pBlock) {
   cases.push_back(pBlock);
}

inline bool test() {
   clog << "<< Start running unit test '" << caseName << "'" << endl;
   bool result = true;
   for(vector<TestCase*>::iterator itr = cases.begin(); itr != cases.end(); ++itr) {
      TestCase* pTest = *itr;
      if (typeid(*pTest) == typeid(UnitTest)) {
         result = result && ((UnitTest*)pTest)->test();
      } else {
         bool caseResult = pTest->getTest()();
         result = result && caseResult;
         if (caseResult)
            clog << ".Test case '" << pTest->getCaseName() << "' is successful." << endl;
         else
            clog << ".Test case '" << pTest->getCaseName() << "' is failed." << endl;
      }
   }
   clog << ">> Finished unit test '" << caseName << "' with " << ((result) ? "success." : "failure.") << endl;
   return result;
}
private:
vector<TestCase*> cases;
};

The usage is simple. First make function to test something like this

#include “UnitTest.h” // let’s say that header file is “UnitTest.h”

bool testprogramname() {
BEGIN_TEST
ASSERT_EQUAL(a(), 100)
END_TEST
}

And create main application

int main(int argc, char **argv) {
UnitTest test("Main");
test.TEST_CASE("TestName1", testprogramname);
test.test();
}

That’s it. If test case is divided into group then create group test module like

UnitTest* index_test() {
UnitTest *pTest = new UnitTest("SubGroup1");
pTest->TEST_CASE("TestName2", anothertestprogram);
return pTest;
}

then modify main like this.

int main(int argc, char **argv) {
UnitTest test("Main");
test.TEST_CASE("TestName1", testprogramname);
test.TEST_BLOCK(pTest);
test.test();
}

Then, the result of test will be

<< Start running unit test 'Main'
.Test case 'TestName1' is success.
<< Start running unit test 'SubGroup1'
.Test case 'TestName2' is failed.
>> Finished unit test 'SubGroup1' with failure.
>> Finished unit test 'Main' with failure.

Verifying an array is easy with dump program

Sometimes, I need to see or dump data inside the array into screen. It is simple job, and often required in cases when I'm dealing with raw data moving from here to there. Such cases can be transferring data through Internet using TCP/IP socket or using more higher protocol like HTTP or moving data to and from raw file structure or even disk itself. It may be necessary to create well defined monitoring program for such cases. However, in many cases dumping bunch of data inside the array into screen without running into debugger, like GDB, is quite useful.

Here is a simple data dump program that can dump array of simple data type like integer or char into screen. Although it is defined as template, the use of type T in the program is limited to primitive data type like integer, float, or char. Although limiting the usage of T through, for example, specialization is possible, I only focused on what is intended, dumping simple data. Using this template with more sophisticated structure like class is limited to hex dump and non-pointer like members only. As implied before, it does not provide any mechanism to check error arose from wrongful usage of type.

template<typename T>
void dump(T* from, T* to,
      const _Ios_Fmtflags fmt=ios_base::hex,
      const int width=0,
      const int maxline=20,
      const char* header = "<======== start dump =========>",
      const char* footer = "<========= end dump ==========>") {
   _Ios_Fmtflags org = cout.flags();
   cout.flags(fmt);
   int size = width;
   if (!width) {
      size = sizeof(T) << 1;
      switch(fmt) {
      case ios_base::oct:
         size <<= 1;
      case ios_base::hex:
         cout << setfill('0');
         break;
      default:
         if (sizeof(T) == 1)  size = 3;
         else {
            ostringstream str;
            str << numeric_limits<T>::max();
            size = str.str().length();
         }
         break;
      }
   }
   cout << header << endl;
   int itr = 1;
   if (sizeof(T) > 1) {
      for(itr = 1; from != to; ++from, ++itr) {
         cout << setw(size) << *from << ',';
         if (!(itr % maxline))
            cout << endl;
      }
      if ( ((itr - 1) % maxline) )
         cout << endl;
   } else {
      unsigned char* f = (unsigned char*)from;
      unsigned char* t = (unsigned char*)to;
      for(itr = 1; f != t; ++f, ++itr) {
         cout << setw(size) << (short)*f << ',';
         if (!(itr % maxline))
            cout << endl;
      }
      if ( ((itr - 1) % maxline) )
         cout << endl;
   }
   cout << footer << endl;
   cout.flags(org);
   if (fmt == ios_base::hex || fmt == ios_base::oct)
      cout << setfill(' ');
}

As you can see from the source code, dump() accepts up to 7 arguments to format the output. First two out of seven are necessary, because they define what to dump. But the rest of arguments can be left as default. Third one in argument, “fmt”, control how to display numbers in screen, and the default value is hexadecimal format, which I prefer. Fourth “width” is for minimum width of each data, and fifth “maxline” is for the number of data in a line. Sixth and seventh argument are header and footer to write, which I used to distinguish from one dump to another.

Usage is simple. For example, if I want to dump an array of characters in buffer into screen, which occurs mostly to me, type like this

char buffer[1024];
...
dump(buffer, buffer+1024);

then the result will be

<======== start dump =========>
0a,00,72,e1,09,00,61,19,04,9e,3f,44,8b,6c,e7,fb,a9,0a,40,00,
00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,
...