C++ Compiler Review - Part 3
Written by Gordon Zeglinski
Part 1 | Part 2 | Part 3 |
---|
Introduction
There were a few typos in the C/C++ compiler review benchmark tables presented in the last issue. Revised tables including the results of the Watcom 10.5 benchmarks will be included in this issue. Watcom 10.5 benchmarks you ask? Yes due to the number of questions I have gotten about Watcom 10.5, in this issue, we will look at what's new in Watcom version 10.5.
What's new in Watcom C/C++ 10.5?
According to the registration card, anyone registering version 10.5 will be mailed a copy of Watcom 10.6. 10.6 will offer more Win95 support (yuck :>). Features that are different or are not found in version 10 are considered to be new.
For the OS/2 programmer, 10.5 features better warp support. The debugger and sampler (used in profiling code) run better under Warp than their 10a counterparts. Although they still have problems dealing with multi-threaded PM programs. Remote debugging of OS/2, Win NT/95 through TCP/IP is now possible. For those doing 16 bit OS/2 programming the OS/2 1.x toolkit is now included as well. This compiler can now be used to generate TCP/IP applications.
For the Windows programmer, Win95 support has been added. Also, the NT 3.5 toolkit is included. MFC 3.0 support and a new visual programmer have been added. The visual programmer is used to build MFC applications. The editor now includes source revision control hooks.
Improvements of a general nature include a new install program, native C++ exception handling, the ability to browse C code, and better compatibility with the Microsoft compiler.
Although the sampler and profiler are not new. It is new that they run under Warp. So we'll take a peek at what the sampler/profiler have to offer.
The profiler displays info about time usage in various levels of detail. The highest level displays the usage based upon compilation unit. The next level displays the time usage for functions in a particular module. Finally, the third level displays the time usage for the lines of a particular function. The lack of thread specific support severely limits the use of the profiler in any multi-threaded OS.
Benchmarks
To allow easy comparison, the complete set of benchmark tables will be included here with Watcom 10.5 added. Note: the Watcom sieve time reported in the last issue was incorrect.
The following tables show the results of the C benchmarks:
Compiler | bsort | sieve | dhry21 | whetstone(S) | whetstone(D) |
---|---|---|---|---|---|
Metaware | 2.468 | 4.218 | 62500.0 | 142857 | 111111 |
VAC++ | 2.219 | 3.750 | 58139.5 | 34482 | 34482 |
Watcom 10a | 1.970 | 4.090 | 65789.5 | 111111 | 76923 |
Watcom 10.5 | 2.190 | 4.25 | 54347.8 | 90909 | 125000 |
Borland | 2.406 | 6.062 | 32467.5 | 17241 | 17241 |
Compiler | Rolled Linpack(S) | Rolled Linpack(D) | Unrolled Linpack(D) | Unrolled Linpack(D |
---|---|---|---|---|
Metaware | 3135 | 2194 | 3135 | 2312 |
VAC++ | 2928 | 1996 | 2928 | 2090 |
Watcom 10a | 2922 | 1990 | 3121 | 2328 |
Watcom10.5 | 2591 | 2081 | 3121 | 2215 |
Borland | 2444 | 1758 | 2586 | 1829 |
The units for the C benchmarks are as follows. Bsort and Sieve are in seconds (the smaller the number the better). Dryh21 is in Dhrystones/sec. The two whetstone tests are in KWhets/sec. All the Linpack tests are in KFlops. Except for the first two benchmarks, larger numbers are better.
Watcom 10.5 performed both better and worse than Watcom 10a in the C benchmark suit. Overall it performed somewhat worse than version 10a.
The following table show the results of the C++ benchmark.
Compiler | Test 1 | Test 2 | Test 3 | Test 4 | Test 5 |
---|---|---|---|---|---|
Metaware | 3.187 | 8.531 | 2.938 | 33.406 | 10.344 |
VAC++ | 2.187 | NA | 2.282 | 460.156 | NA |
Watcom | 3.090 | 18.720 | 4.910 | 93.400 | 20.970 |
Wat 10.5 | 2.84 | 17.63 | 4.93 | 95.79 | 19.34 |
Borland | 3.406 | NA | 3.032 | 825.344 | NA |
Test 1 simply tests the construction and destruction of an object including a call to it's virtual function. Test 2 is like test 1 except a more complex object is used. Test 3 is similar to test 1 except that the object is created/destroyed from within a try/catch block. Test 4 is like test 3 except that an exception is actually thrown and caught. Test 5 assigns the complex object to another object. The same complex object that was used in test 2 is used here.
Test 3 uses 1 less function call than test 1. In theory, it should use up less time than test 1 unless the amount of time used to setup the try/catch block exceeds that of a function call. In both the Metaware and Borland test results, test 3 was faster than test 1.
In the C++ test suit, Watcom 10.5 performed better than version 10a in every test except those involving C++ exceptions. It seems the new C++ exception handling code in 10.5 performs slightly worse than the code in version 10a.
The following table shows the compilation time and executable size for the C++ bench mark program. Notice the compile time and executable size decrease comparing version 10a and 10.5.
Compiler | Compile+Link Time (s) | EXE size (bytes) |
---|---|---|
Metaware | 59.19 | 80400 |
VAC++ | 35.35 | 39936 |
Watcom | 84.63 | 65898 |
Wat 10.5 | 57.22 | 59269 |
Borland | 14.22 | 41489 |
The unzip test compares the compilation time, executable speed, and executable size for Info unzip version 5.12. The following table shows the results of the unzip test.
Compiler | Compile+Link (s) | 'unzip -t unzip512.zip >NUL' (s) | EXE size (bytes) |
---|---|---|---|
Metaware | 629.47 | 2.37 | 128579 |
VAC++ | 189.10 | 2.50 | 136755 |
Watcom | 232.22 | 2.07 | 96027 |
Wat 10.5 | 231.91 | 2.13 | 96245 |
Borland | 168.53 | 2.40 | 140339 |
Note: although VAC++ compiled unzip faster than Watcom, this is may or may not be the case. VAC++ takes twice as long to compile the AdeptXBBS source code. Automatic function inlining had to be turned off in VAC++ because the resulting EXE falsely reported error in a good zip file. Metaware had inlining turned on.
Summary
The upgrade to Watcom 10.5 is worthwhile. Compared to version 10a, one gains remote OS/2 debugging, better performance under warp (although not perfect yet), and faster C++ compilation times. For those doing TCP/IP programs, there's finally an alternative to C Set and EMX (a freeware C/C++ compiler). The cross platform support is great.
Watcom 10.5 works well enough to be the choice for a general purpose OS/2 compiler. It's not a perfect package however, it's extremely versatile and reasonably priced. Personally, I still prefer using C Set++ 2.1 for debugging and analysing code performance. The debugger and profiler in C Set++ 2.1 work well and do so in a reasonable amount of memory.