Tuesday, April 28, 2009

ThreadPool is useful when I treat a group of threads

Crafting and using threads is almost as sensitive as allocating and maneuvering memory, at least to me. So, I spent more time looking at the result of memory check tools like “Valgrind,” or allocated more time to create memory management mechanism when I program with C++. I guess these are some of the reasons why I feel more comfortable and effortless when I using Java to test the logic or algorithm of a program.

Almost all the time when I needed tasks to be executed concurrently, I needed a mechanism to manage group of threads. “ThreadPool” is simple class that take care of a pool of threads which run tasks fed through “Command” interface. This pool is very simple but can be used in most common cases. If I need prioritize the tasks, then changing data structure of queue from list to priority_queue will be sufficient. If I need canceling of a task, I can add search and remove of the task from queue to “ThreadPool” class.

“ThreadPool” initialize specified number of workers by using pthread_create(). After that I can add task using addCommand(). The safeguarding of running task which implemented through interface of “Command” is completely up to user. So, if there are any critical sections inside running tasks, use pthread_mutex to protect them.

Here is source code of “ThreadPool.” First the header,

#ifndef THREADPOOL_H_
#define THREADPOOL_H_

#include <list>
#include <pthread.h>

struct Command {
virtual void run()=0;
};

void* worker(void* args);

class ThreadPool {
public:
class ThreadCreationError {};

ThreadPool(int poolSize): poolSize(poolSize) {
inErrorStatus = false;
pRunners = new pthread_t[poolSize];
bootUp(poolSize);
}

virtual ~ThreadPool() {
shutDown();
delete[] pRunners;
}

void addCommand(Command& com);
private:
int poolSize;
bool inErrorStatus;
bool workerLoop;
pthread_t* pRunners;
pthread_attr_t attr;
pthread_mutex_t queueLock;
pthread_cond_t signal;
std::list<Command*> queue;

void bootUp(int size);
void shutDown();
friend void* worker(void* args);
};

#endif /* THREADPOOL_H_ */

Next, cpp code,

#include "ThreadPool.h"

void ThreadPool::bootUp(int size) {
workerLoop = true;
pthread_mutex_init(&queueLock, NULL);
pthread_cond_init(&signal, NULL);
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
pthread_setcanceltype(PTHREAD_CANCEL_DEFERRED, NULL);
for (int i = 0; i < size; ++i) {
if (pthread_create(&pRunners[i], &attr, worker, this) != 0) {
if (i != 0) shutDown();
inErrorStatus = true;
break;
}
}
}

void ThreadPool::shutDown() {
workerLoop = false;
pthread_cond_broadcast(&signal);
for (int i = 0; i < poolSize; ++i) pthread_join(pRunners[i], NULL);
pthread_attr_destroy(&attr);
pthread_cond_destroy(&signal);
pthread_mutex_destroy(&queueLock);
}

void ThreadPool::addCommand(Command& com) {
pthread_mutex_lock(&queueLock);
queue.push_back(&com);
pthread_cond_signal(&signal);
pthread_mutex_unlock(&queueLock);
}

void* worker(void* args) {
ThreadPool& pool = *static_cast<ThreadPool*>(args);
std::list<Command*>& queue = pool.queue;
pthread_mutex_t& queueLock = pool.queueLock;
pthread_cond_t& signal = pool.signal;
bool& loop = pool.workerLoop;
Command* pWorker;
while (loop || queue.size() > 0) {
pWorker = NULL;
pthread_mutex_lock(&queueLock);
if (queue.size() == 0)
pthread_cond_wait(&signal, &queueLock);
if (queue.size() > 0) {
pWorker = queue.front();
queue.pop_front();
if (queue.size() > 0)
pthread_cond_signal(&signal);
}
pthread_mutex_unlock(&queueLock);
if (pWorker) {
pWorker->run();
}
}
pthread_exit(NULL);
}

As before, usage is simple. First, I need to instantiate “ThreadPool” with pool size.

ThreadPool pool(5);

And prepare my task.

struct TestCommand: public Command {
int val;

TestCommand(int val): val(val) {}

void run() {
cout << "Worker[" << val << "," << pthread_self() << "] is running" << endl;
sleep(1);
}
};

Then, run it putting task using addCommand().

TestCommand com(100);
pool.addCommand(com);

Tuesday, April 21, 2009

Unit test is easy with this small program

Whenever I have time, I try and test new technology with my program. Naturally, tools to revert and test what I developed is essential. I move around various version of programs through SVN and history feature of Eclipse, which is definitely my favorite IDE. In terms of feature test, I often use small program I created. It is very small and as much expandable as I needed. And it is really simple, but also covers hierarchical structure of test cases. The test program is mixed up version of C and C++ to really simplify test. Building such program may not take much time if someone want to create class like this. As such, it is very easy to modify or expand as needed.

Here is UnitTest class.

typedef bool (*_TEST_FUNC_)();

#define BEGIN_TEST bool _test_result_ = true;
#define END_TEST return _test_result_;

#define ASSERT(value)\
result = result && (value == ture);
#define ASSERT_EQUAL(value1, value2)\
_test_result_ = _test_result_ && (value1 == value2);
#define ASSERT_NOT_EQUAL(value1, value2)\
_test_result_ = _test_result_ && (value1 != value2);
#define ASSERT_GREATER(value1, value2)\
_test_result_ = _test_result_ && (value1 > value2);


class TestCase {
public:
TestCase(string name): caseName(name) {}
virtual ~TestCase() {}

inline void setTest(_TEST_FUNC_ test) { this->test = test; }
inline _TEST_FUNC_ getTest() { return test; }
inline string getCaseName() { return caseName; }

protected:
string caseName;
_TEST_FUNC_ test;
};

class UnitTest: public TestCase {
public:
UnitTest(string name):TestCase(name) {}
virtual ~UnitTest() {
for (vector<TestCase*>::iterator itr = cases.begin(); itr != cases.end(); ++itr)
delete *itr;
}

inline UnitTest& TEST_CASE(string caseName, _TEST_FUNC_ test) {
TestCase *pTest = new TestCase(caseName);
pTest->setTest(test);
cases.push_back(pTest);
return *this;
}

inline void TEST_BLOCK(UnitTest* pBlock) {
cases.push_back(pBlock);
}

inline bool test() {
clog << "<< Start running unit test '" << caseName << "'" << endl;
bool result = true;
for(vector<TestCase*>::iterator itr = cases.begin(); itr != cases.end(); ++itr) {
TestCase* pTest = *itr;
if (typeid(*pTest) == typeid(UnitTest)) {
result = result && ((UnitTest*)pTest)->test();
} else {
bool caseResult = pTest->getTest()();
result = result && caseResult;
if (caseResult)
clog << ".Test case '" << pTest->getCaseName() << "' is successful." << endl;
else
clog << ".Test case '" << pTest->getCaseName() << "' is failed." << endl;
}
}
clog << ">> Finished unit test '" << caseName << "' with " << ((result) ? "success." : "failure.") << endl;
return result;
}
private:
vector<TestCase*> cases;
};

The usage is simple. First make function to test something like this

#include “UnitTest.h” // let’s say that header file is “UnitTest.h”

bool testprogramname() {
BEGIN_TEST
ASSERT_EQUAL(a(), 100)
END_TEST
}

And create main application

int main(int argc, char **argv) {
UnitTest test("Main");
test.TEST_CASE("TestName1", testprogramname);
test.test();
}

That’s it. If test case is divided into group then create group test module like

UnitTest* index_test() {
UnitTest *pTest = new UnitTest("SubGroup1");
pTest->TEST_CASE("TestName2", anothertestprogram);
return pTest;
}

then modify main like this.

int main(int argc, char **argv) {
UnitTest test("Main");
test.TEST_CASE("TestName1", testprogramname);
test.TEST_BLOCK(pTest);
test.test();
}

Then, the result of test will be

<< Start running unit test 'Main'
.Test case 'TestName1' is success.
<< Start running unit test 'SubGroup1'
.Test case 'TestName2' is failed.
>> Finished unit test 'SubGroup1' with failure.
>> Finished unit test 'Main' with failure.

Saturday, April 18, 2009

Verifying an array is easy with dump program

Sometimes, I need to see or dump data inside the array into screen. It is simple job, and often required in cases when I'm dealing with raw data moving from here to there. Such cases can be transferring data through Internet using TCP/IP socket or using more higher protocol like HTTP or moving data to and from raw file structure or even disk itself. It may be necessary to create well defined monitoring program for such cases. However, in many cases dumping bunch of data inside the array into screen without running into debugger, like GDB, is quite useful.

Here is a simple data dump program that can dump array of simple data type like integer or char into screen. Although it is defined as template, the use of type T in the program is limited to primitive data type like integer, float, or char. Although limiting the usage of T through, for example, specialization is possible, I only focused on what is intended, dumping simple data. Using this template with more sophisticated structure like class is limited to hex dump and non-pointer like members only. As implied before, it does not provide any mechanism to check error arose from wrongful usage of type.

template<typename T>
void dump(T* from, T* to,
const _Ios_Fmtflags fmt=ios_base::hex,
const int width=0,
const int maxline=20,
const char* header = "<======== start dump =========>",
const char* footer = "<========= end dump ==========>") {
_Ios_Fmtflags org = cout.flags();
cout.flags(fmt);
int size = width;
if (!width) {
size = sizeof(T) << 1;
switch(fmt) {
case ios_base::oct:
size <<= 1;
case ios_base::hex:
cout << setfill('0');
break;
default:
if (sizeof(T) == 1) size = 3;
else {
ostringstream str;
str << numeric_limits<T>::max();
size = str.str().length();
}
break;
}
}
cout << header << endl;
int itr = 1;
if (sizeof(T) > 1) {
for(itr = 1; from != to; ++from, ++itr) {
cout << setw(size) << *from << ',';
if (!(itr % maxline))
cout << endl;
}
if ( ((itr - 1) % maxline) )
cout << endl;
} else {
unsigned char* f = (unsigned char*)from;
unsigned char* t = (unsigned char*)to;
for(itr = 1; f != t; ++f, ++itr) {
cout << setw(size) << (short)*f << ',';
if (!(itr % maxline))
cout << endl;
}
if ( ((itr - 1) % maxline) )
cout << endl;
}
cout << footer << endl;
cout.flags(org);
if (fmt == ios_base::hex || fmt == ios_base::oct)
cout << setfill(' ');
}

As you can see from the source code, dump() accepts up to 7 arguments to format the output. First two out of seven are necessary, because they define what to dump. But the rest of arguments can be left as default. Third one in argument, “fmt”, control how to display numbers in screen, and the default value is hexadecimal format, which I prefer. Fourth “width” is for minimum width of each data, and fifth “maxline” is for the number of data in a line. Sixth and seventh argument are header and footer to write, which I used to distinguish from one dump to another.

Usage is simple. For example, if I want to dump an array of characters in buffer into screen, which occurs mostly to me, type like this


char buffer[1024];
...
dump(buffer, buffer+1024);


then the result will be


<======== start dump =========>
0a,00,72,e1,09,00,61,19,04,9e,3f,44,8b,6c,e7,fb,a9,0a,40,00,
00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,
...

For my dear father

Recently, my father’s passed away.

I didn’t support him much lately, in the excuse of living abroad. It seems like that I still am hearing his voice calling my name, and that I could see him through my tears.

This Blog is for my father.