C# .NET Programming Blog

A multi-core processor combines two or more independent cores on a single chip. This multi-core architecture is adopted by most modern processors enabling execution of multiple threads simultaneously.

This article concentrates on how to optimize multithreading on multi-core CPUs.

Understanding Thread Behavior

Multithreading does not actually mean that a CPU core processes multiple threads at a time. No matter how many cores the CPU has, each core can process only one thread at a time (Note that in Hyper threading enabled CPUs the operating system treats the single CPU as 2 cores).

Consider the example of a shopping mall billing counter. Let us assume that there are 2 billing counters which represent CPU cores and each billing counter can bill for only 10 items at a time. If you wanted to buy 100 items you would have to go in and out of the billing queue 10 times.

To get the billing done faster you a take a friend along, who stands in the other queue. This time each one of you will have to stand in the queue 5 times each, and your work will be done in half the time. But however this does not mean that if there were 10 of you your work will be done 10 times faster. Since there are only 2 billing counters only 20 items will be processed at a time. If there were 10 billing counters you will definitely get your work done 10 times faster.

In the above example you and your friends represented one thread each, which will get work done only if it gets CPU time. So it is quite clear that the number of threads should ideally be the same as the number of CPUs or CPU cores right?

Well not just yet. We forgot one minor detail. You are not the only customer in the mall there are other customers too. Similarly we do have other processes and their threads running on the computer that require processing time. So if there were 10 of you the probability of you or one of your friends getting in the line is higher.

Thread execution time is managed by the operating system and is a highly unpredictable activity. CPU execution time is known as a time slice or a quantum. If we create more and more threads the operating system spends a fraction of this time slice in deciding which thread goes first, thus reducing the actual execution time each thread gets. In other words each thread will do lesser work if there were a large number of threads queued up.

Calculating the Optimal Thread Concurrency

To confirm the theory above and to arrive at a conclusion as to how many threads should be declared for a doing the same task, a small console application which records the CPU time for each thread is shown below. We will try to determine if we get higher performance when we declare threads equal to the processor count or greater than the processor count.

We record the time for multiple threading scenarios. Processor count is used to determine how many concurrent threads should be tested. If processor count is ‘N’ we determine result for threads in range from N-1 to N+2 concurrent threads.

The processing done by each thread is quite simple. Each thread has to run for 1 second. This means that each thread should get a total execution time of one second. The thread might not get continuous execution cycles causing the thread to wait for CPU time before it can start executing again. So one second of execution time might actually take more than one second if the thread does not get dedicated CPU time.

Once the thread has finished execution it signals the WaitHandle. After all the threads have finished execution the total time per thread is calculated. Time per thread is the total time required from start of first thread to end of last thread divided by number of threads. This gives the average time required for each thread to process.

Show Code

class Program
{
    static void Main(string[] args)
    {
        TextWriter original = Console.Out;
        Console.WriteLine(@"Recording data to file C:\Report.txt");
        using (StreamWriter sw = new StreamWriter(@"C:\Report.txt"))
        {
            Console.SetOut(sw);

            for (int i = Environment.ProcessorCount - 1;
                 0 < i && i < Environment.ProcessorCount + 3;
                 i++)
                 Analyze(i);
        }

        Console.SetOut(original);
        Console.WriteLine("Data recorded successfully.");
        Console.ReadKey();
    }

    static void Analyze(int threadcount)
    {
        Console.WriteLine("Generating data for " + threadcount + " threads");
        WorkProcessor[] processors = new WorkProcessor[threadcount];
        WaitHandle[] waitHandles = new EventWaitHandle[threadcount];

        for (int i = 0; i < threadcount; i++)
        {
            processors[i] = new WorkProcessor();
            waitHandles[i] = processors[i].WaitHandle;
        }

        // Wait for all threads to finish
        WaitHandle.WaitAll(waitHandles);

        // Calculate Time/Thread
        long min = long.MaxValue;
        long max = 0;

        foreach (WorkProcessor processor in processors)
        {
            if (processor.Log[0] < min)
                min = processor.Log[0];
            if (processor.Log[processor.Log.Count - 1] > max)
                max = processor.Log[processor.Log.Count - 1];

            processor.Dispose();
        }

        TimeSpan time = new TimeSpan(max - min);
        Console.WriteLine("Time/Thread: " +
            time.TotalMilliseconds / threadcount + " ms\r\n");
    }
}

Show Code

class WorkProcessor : IDisposable
{
    Thread worker;
    EventWaitHandle _waitHandle;

    public WorkProcessor()
    {
        _log = new List<long>();
        worker = new Thread(new ThreadStart(Record));
        _waitHandle = new ManualResetEvent(false);
        worker.Start();
    }

    public EventWaitHandle WaitHandle
    {
        get { return _waitHandle; }
    }

    List<long> _log;
    public List<long> Log
    {
        get { return _log; }
    }

    public void Record()
    {
        while (Log.Count != TimeSpan.TicksPerSecond)  // Record log for 1 sec
        _log.Add(DateTime.Now.TimeOfDay.Ticks);
        _log = _log.Distinct().ToList<long>();
        _waitHandle.Set();
    }

    public void Dispose()
    {
        _log.Clear();
        _waitHandle.Close();
    }
}

The Result

Data was recorded for the following CPUs

1. AMD X2 64 3800+ (Dual Core)
2. Intel Dual Core 3.0Ghz (Dual Core)
3. Intel Xeon E5450 3.0 GHz (Dual Core) *
4. Intel Xeon E5310 1.6 GHz (Quad Core)

* I could not find a dual core version of Xeon E5450 on Intel’s web site however the server I tested it on was actually running a dual core version of this processor and not a quad core one.

The testing results are shown in the graphs below. Figures to the right of the bars indicate the time/thread in milliseconds. Lesser time/thread indicates faster execution.

Conclusions

From the result it is quite clear that dual core CPUs give the best performance for 2 concurrent threads and the single quad core that was tested gives best performance at 4 concurrent threads. More detailed analysis can be done by recording data for longer time periods to give a more accurate reading.

When a dual core CPU runs just one thread it takes almost twice as much time as it would if 2 concurrent threads were running, which is quite expected. As the number of concurrent threads increases to 3 there is a slight increase in time required. This is probably due to the operating system switching between the threads.

If this data was to be recorded at high CPU load then the results might have been different.

Also setting the thread priority as high or highest the thread can be given more dedicated CPU time for execution.

Download Sample
MultithreadingSample.zip