Some benchmarks of SOM vs C++ based objects
Written by Gordon Zeglinski
Introduction
So are we sick of SOM yet? I hope not! This issue last will be the last column on SOM for a while. We will look at some benchmarks of SOM vs C++ based objects. Hopefully you are a curious as I am about the overhead built into SOM.
On with the Benchmark
What we will try to do here is write a series of tests to determine the overhead involved with calling a member function in SOM. Because we are interested only in the function call overhead, the function will simply increment a counter variable. This counter variable will be stored in the object's instance data. For comparison, a C++ version of the SOM object will be used to establish a base mark. The interface definition of the SOM object follows.
interface SOMFoo : SOMObject
{
attribute long Count; // Stores the Count Data
void IncrementCount(); //procedure
void OffIncrementCount(); //offset
void LookIncrementCount(); //lookup
#ifdef __SOMIDL__
implementation
{
releaseorder: _get_Count,_set_Count,IncrementCount,\
OffIncrementCount,LookIncrementCount;
IncrementCount: procedure;
LookIncrementCount: namelookup;
callstyle=oidl;
filestem = sombench;
somInit: override; // initialize instance variables to 0
somUninit: override; // just for fun
};
#endif /* __SOMIDL__ */
};
Three functions are defined to test the various resolution methods. IncrementCount is used to test the "procedure" resolution method. A procedure doesn't use a resolution method per se; a procedure is similar to a non-virtual C++ function because it is called by its address. OffIncrementCount is used to test the offset resolution method. This is the default method used by SOM to resolve a function call. This method is similar to virtual functions in C++. LookIncrementCount is used to test the name lookup resolution method.
A function can be labelled a procedure as follows:
IncrementCount: procedure;
Because the default "type" for a function is method, one only has to list functions that are to be procedures.
Note: only methods can be overridden.
Similarly, one can specify that a method is to lookup offset resolution by using the following:
LookIncrementCount: namelookup;
Again, one only lists functions which deviate from the normal resolution method. The normal method is offset.
Note: there seems to be a bug in the C++ emitters. This bug causes incorrect code to be generated for the lookup resolution method.
For the C++ object, we use the following class definition:
class CPPfoo{
int Count; // Stores the Count Data
public:
CPPfoo();
~CPPfoo();
void IncrementCount(); //similar to SOM procedure
void InlineIncrementCount(){Count++;} //no equivalence in SOM procedure
virtual void OffIncrementCount(); //similar to SOM offset
void SetCount(int C){Count=C;}
int GetCount(){return Count;}
};
The C++ object duplicates the SOM object wherever possible. The two major differences between tests on the C++ and SOM objects are that the C++ object can't do "name lookup resolution" and the SOM object can't do inline code.
Now that we have seen the objects, we can look at how they will be used. Both the C++ and SOM objects are structured similarly, thus, it follows that both of the tests will be implemented similarly.
#define INCL_DOS
#include <os2.h>
#include "sombench.xh"
#include <stdio.h>
void main(){
ULONG StartTime, EndTime;
int i;
SOMFoo *sfoo=new SOMFoo;
DosQuerySysInfo(QSV_MS_COUNT,QSV_MS_COUNT,&StartTime,4); //Get Start time using the system timer
for(i=0;i<1000000;i++)
SOMFoo::IncrementCount(sfoo); //lets call the fnc a million times
DosQuerySysInfo(QSV_MS_COUNT,QSV_MS_COUNT,&EndTime,4); //Get End time we
printf("Time for Procedure calls %i ms count= %i\r\n",EndTime-StartTime,sfoo->_get_Count());
sfoo->_set_Count(0);
DosQuerySysInfo(QSV_MS_COUNT,QSV_MS_COUNT,&StartTime,4);
for(i=0;i<1000000;i++)
sfoo->OffIncrementCount();
DosQuerySysInfo(QSV_MS_COUNT,QSV_MS_COUNT,&EndTime,4);
printf("Time for Offset lookup calls %i ms count= %i\r\n",EndTime-StartTime,sfoo->_get_Count());
sfoo->_set_Count(0);
DosQuerySysInfo(QSV_MS_COUNT,QSV_MS_COUNT,&StartTime,4);
for(i=0;i<1000000;i++)
lookup_LookIncrementCount(sfoo);
// sfoo->LookIncrementCount();
DosQuerySysInfo(QSV_MS_COUNT,QSV_MS_COUNT,&EndTime,4);
printf("Time for Name lookup calls #1 %i ms count= %i\r\n",EndTime-StartTime,sfoo->_get_Count());
sfoo->_set_Count(0);
DosQuerySysInfo(QSV_MS_COUNT,QSV_MS_COUNT,&StartTime,4);
for(i=0;i<1000000;i++)
(somResolveByName(sfoo,"LookIncrementCount") )(sfoo);
DosQuerySysInfo(QSV_MS_COUNT,QSV_MS_COUNT,&EndTime,4);
printf("Time for Name lookup calls #2 %i ms count= %i\r\n",EndTime-StartTime,sfoo->_get_Count());
}
To determine the overhead in making the function call, we call the function 1 million times. The time it takes to call the function 1 million times along with the value of the incremented variable is displayed after each iteration. The results of the SOM test follow.
Time for Procedure calls 656 ms count= 1000000 Time for Offset lookup calls 719 ms count= 1000000 Time for Name lookup calls #1 22031 ms count= 1000000 Time for Name lookup calls #2 175031 ms count= 1000000
On a side note, The procedure IncrementCount is emitted as a static member function in C++. This is strange because you still have to pass it a pointer to the instance of the C++ object that wraps instance SOM object. This of course is exactly what a C++ member function does (in an implicit manner).
Running the tests on the C++ object yields the following.
Time for Procedure calls 219 ms count= 1000000 Time for Offset lookup calls 281 ms count= 1000000 Time for inline calls 0 ms count= 1000000
Examining the Results
As usual to give a benchmark meaning, the system it is run on has to be listed. The following list shows the syslevels of the various components involved in the tests.
IBM OS/2 Base Operating System
Version 2.11 Component ID 562107701
Type 0
Current CSD level: XR06200
Prior CSD level: XR02110
IBM C/C++ Tools (compiler)
Version 2.00 Component ID 562201703
Current CSD level: CT00010
Prior CSD level: CT00009
SOMobjects Developer Toolkit
Version 2.00 Component ID 96F8647TK
Current CSD level: SM20004
Prior CSD level: SM00000
All of this was running on a 486 DX2-66 PC clone machine with 20 megs of RAM.
Let's start by looking at the C++ test. For 1 million calls to a member function, only 219 milliseconds were used. Calling a virtual function took just slightly more time. The inline case took no measurable amount of time. What happened most probably in the inline case is that the optimizer completely removed the loop and did a direct assignment instead. (I haven't checked the compiled code to verify if this is the case or not. If you are curious, feel free to check it yourself.)
One of the arguments you see tossed around when arguing the merits of C++ versus C is that the additional amount of time it takes to call a virtual function compared to a normal "C function" is negligible. This test confirms those arguments. The overall amount of time used to call one a virtual function for "real cases" is even less important.
The offset resolution case in the SOM test took about 2.5 times longer to execute than did the virtual C++ function test. Recall that the offset resolution case in SOM is equivalent to the virtual function case in C++. There are several ways to call a function by its name. In the first name lookup test, we use the macro created by the SOM compiler called lookup_LookIncrementCount. For the second test, we use somResolveByName. Note that the difference in execution time between the two lookup tests.
Even though the second name lookup case took much longer than the other cases, one should remember that this test does not necessarily represent a real world use of the somResolveByName function. This function was called a million times. If one were to write this in an efficient manner, only 1 call would be made to determine the address of the function. Once this address has been determined, it can be reused inside the loop. The time used by the revamped lookup test should then approach that of the procedure test.
One of the reasons the SOM version is slower than the C++ version is because, in SOM, the object's interface doesn't have to be fully disclosed. The base pointer to the data is not the same as the pointer to the objects instance. Thus, in SOM, one has to query the pointer for the instance data in order to access it. Consider the following SOM function:
SOM_Scope void SOMLINK OffIncrementCount(SOMFoo *somSelf)
{
SOMFooData *somThis = SOMFooGetData(somSelf);
// SOMFooMethodDebug("SOMFoo","OffIncrementCount");
(somThis->Count)++;
}
Note: I commented out the debug call to increase performance.
Now compare the SOM function to the following C++ member function:
void CPPfoo::IncrementCount(){
Count++;
}
Notice that in the SOM function, we have to determine the value of somThis.
Finally a word of caution. OS/2's time slicing is far too coarse for high degrees of accuracy with numbers this small. A variation of 32ms (or so) may occur when running these tests, since that is the resolution of the clock used to pre-empt tasks running within the system.
Included Files
- CPPTEST.CPP C++ test
- SOMBENCH.IDL SOM idl file
- SOMBENCH.CPP C++ implementation for SOM object
- SOMBENCH.XH SOM header
- SOMBENCH.XIH SOM implementation header
- SOMBTEST.CPP main routine for SOM tests
Wrapping Things Up
That's it for another fun-filled look at SOM. It should be noted that there probably will not be an instalment of this column next month while I wrap up the development of INTERcomm, a commercial communications package that I have developed.