C++ Compiler Review - Part 3

From EDM2
Revision as of 15:19, 8 December 2017 by Ak120 (Talk | contribs)

Jump to: navigation, search

Written by Gordon Zeglinski

Part 1

Part 2

Part 3

Introduction

There were a few typos in the C/C++ compiler review benchmark tables presented in the last issue. Revised tables including the results of the Watcom 10.5 benchmarks will be included in this issue. Watcom 10.5 benchmarks you ask? Yes due to the number of questions I have gotten about Watcom 10.5, in this issue, we will look at what's new in Watcom version 10.5.

What's new in Watcom C/C++ 10.5?

According to the registration card, anyone registering version 10.5 will be mailed a copy of Watcom 10.6. 10.6 will offer more Win95 support (yuck :>). Features that are different or are not found in version 10 are considered to be new.

For the OS/2 programmer, 10.5 features better warp support. The debugger and sampler (used in profiling code) run better under Warp than their 10a counterparts. Although they still have problems dealing with multi-threaded PM programs. Remote debugging of OS/2, Win NT/95 through tcp/ip is now possible. For those doing 16 bit OS/2 programming the OS/2 1.x toolkit is now included as well. This compiler can now be used to generate tcp/ip applications.

For the Windows programmer, Win95 support has been added. Also, the NT 3.5 toolkit is included. MFC 3.0 support and a new visual programmer have been added. The visual programmer is used to build MFC applications. The editor now includes source revision control hooks.

Improvements of a general nature include a new install program, native C++ exception handling, the ability to browse C code, and better compatibility with the Microsoft compiler.

Although the sampler and profiler are not new. It is new that they run under Warp. So we'll take a peek at what the sampler/profiler have to offer.

profiler modulenames

profiler routinenames

profiler sourceline

The profiler displays info about time usage in various levels of detail. The highest level displays the usage based upon compilation unit. The next level displays the time usage for functions in a particular module. Finally, the third level displays the time usage for the lines of a particular function. The lack of thread specific support severely limits the use of the profiler in any multi-threaded OS.

Benchmarks

To allow easy comparison, the complete set of benchmark tables will be included here with Watcom 10.5 added. Note: the Watcom seive time reported in the last issue was incorrect.

The following tables show the results of the C benchmarks:

Compiler bsort sieve dhry21 whetstone(S) whetstone(D)
Metaware 2.468 4.218 62500.0 142857 111111
VAC++ 2.219 3.750 58139.5 34482 34482
Watcom 10a 1.970 4.090 65789.5 111111 76923
Watcom 10.5 2.190 4.25 54347.8 90909 125000
Borland 2.406 6.062 32467.5 17241 17241
Compiler Rolled Linpack(S) Rolled Linpack(D) Unrolled Linpack(D) Unrolled Linpack(D
Metaware 3135 2194 3135 2312
VAC++ 2928 1996 2928 2090
Watcom 10a 2922 1990 3121 2328
Watcom10.5 2591 2081 3121 2215
Borland 2444 1758 2586 1829

The units for the C benchmarks are as follows. Bsort and Sieve are in seconds (the smaller the number the better). Dryh21 is in Dhrystones/sec. The two whetstone tests are in KWhets/sec. All the Linpack tests are in KFlops. Except for the first two benchmarks, larger numbers are better.

Watcom 10.5 performed both better and worse than Watcom 10a in the C benchmark suit. Overall it performed somewhat worse than version 10a.

The following table show the results of the C++ benchmark.

Compiler Test 1 Test 2 Test 3 Test 4 Test 5
Metaware 3.187 8.531 2.938 33.406 10.344
VAC++ 2.187 NA 2.282 460.156 NA
Watcom 3.090 18.720 4.910 93.400 20.970
Wat 10.5 2.84 17.63 4.93 95.79 19.34
Borland 3.406 NA 3.032 825.344 NA

Test 1 simply tests the construction and destruction of an object including a call to it's virtual function. Test 2 is like test 1 except a more complex object is used. Test 3 is similar to test 1 except that the object is created/destroyed from within a try/catch block. Test 4 is like test 3 except that an exception is actually thrown and caught. Test 5 assigns the complex object to another object. The same complex object that was used in test 2 is used here.

Test 3 uses 1 less function call than test 1. In theory, it should use up less time than test 1 unless the amount of time used to setup the try/catch block exceeds that of a function call. In both the Metaware and Borland test results, test 3 was faster than test 1.

In the C++ test suit, Watcom 10.5 performed better than version 10a in every test except those involving C++ exceptions. It seems the new C++ exception handling code in 10.5 performs slightly worse than the code in version 10a.

The following table shows the compilation time and executable size for the C++ bench mark program. Notice the compile time and executable size decrease comparing version 10a and 10.5.

Compiler Compile+Link Time (s) EXE size (bytes)
Metaware 59.19 80400
VAC++ 35.35 39936
Watcom 84.63 65898
Wat 10.5 57.22 59269
Borland 14.22 41489

The unzip test compares the compilation time, executable speed, and executable size for Info unzip version 5.12. The following table shows the results of the unzip test.

Compiler Compile+Link (s) 'unzip -t unzip512.zip >NUL' (s) EXE size (bytes)
Metaware 629.47 2.37 128579
VAC++ 189.10 2.50 136755
Watcom 232.22 2.07 96027
Wat 10.5 231.91 2.13 96245
Borland 168.53 2.40 140339

Note: although VAC++ compiled unzip faster than Watcom, this is may or may not be the case. VAC++ takes twice as long to compile the AdeptXBBS source code. Automatic function inlining had to be turned off in VAC++ because the resulting EXE falsely reported error in a good zip file. Metaware had inlining turned on.

Summary

The upgrade to Watcom 10.5 is worthwhile. Compared to version 10a, one gains remote OS/2 debugging, better performance under warp (although not perfect yet), and faster C++ compilation times. For those doing tcp/ip programs, there's finally a alternative to CSet and EMX (a freeware C/C++ compiler). The cross platform support is great.

Watcom 10.5 works well enough to be the choice for a general purpose OS/2 compiler. It's not a perfect package however, it's extremely versatile and reasonably priced. Personally, I still prefer using CSet++ 2.1 for debugging and analyzing code performance. The debugger and profiler in CSet++ 2.1 work well and do so in a reasonable amount of memory.