This is my first post on our C/C++ Cafe that has been long in coming.

If you are like me, then you are a new zOS programmer. The learning ride has been quite turbulent and there are ways to go yet. If you are a devoted programmer then you feel quiet excitement of your new program almost working, tempered by the chance of another large manual being 'thrown' at you. zOS is one of those products that has too much of a good thing, that is there is a LOT of documentation. This fact very quickly becomes an advantage as one gains more experience.

No matter what design patterns you use, almost as soon as you start writing code, you will need to include other files, be it either your own include files or third party libraries. For efficiency purposes, these files will in all likelihood reside in datasets. To make things more interesting, the documentation might instruct to use directory based (HFS) #include, confusing the new-comers as to where files reside.

I want to discuss two tasks that come up when dealing with include files:

• Finding the include files
• Dealing with preprocessor macros

If the compiler already found the include file (i.e. successful
compile) and you are interested where the file is located (i.e. which version of the library you are actually using), both -qlist and
-qsource produce an includes section.

The grep command might not return anything because of some zOS specific file translations, hence be careful:

• if NAME.OF.INCLUDE contains dots, the real file name might be translated to INCLUDE.OF.NAME or just NAME
• if NAME_OF_INCLUDE contains underscores, it might get translated to NAME@OF@INCLUDE

If you are using V1R11 compiler, there is a new feature, -qmakedep (on USS only) that will produce a list of all included files. It is much easier to remember then the sed command above. For example if you compiled:

For previous releases of zOS compiler, have a look at the makedepend utility. It contains some other options that might be useful for include file debugging.

-qshowinc is another particularly useful option, if you already have the include files. It shows the the contents of the included files. It is similar to what PPONLY (-E equivalent) option does, except it outputs to the listing and does not strip preprosessor directives. However, be ready to pipe the output to other programs to filter out the thousands of lines of code produced. Most of the time I use less and its search features. sed and grep sometimes are also be useful.

If you are trying to include a file, and the compiler cannot find it, there is more research involved. In a general case, the include files can be found, top to bottom in these places:

pwd

LSEARCH

DD:USERLIB

SEARCH

DD:SYSLIB

• System includes can only be found in SEARCH and DD:SYSLIB.
• DD: statements come from the JCL, hence you might need to know how the compiler was invoked.
• SEARCH and LSEARCH come from the compiler options hence are easy to modify

SEARCH and LSEARCH both contain a list of directories and partial dataset qualifiers. The topic is discussed in detail in our Compiler User Guide in 'Chapter 7: Using include files'. I personally found the flowcharts in Chapter 7 and the Examples in the LSEARCH option explanation cleared up most of the most dataset questions that I had. It is worth noting for newcomers that terms 'z/OS Unix files' and 'HFS files' are equivalent and refer to directory based files (i.e. similar to organization Linux and Windows file systems) (as opposed to DATASETs that can be sequential or PDS in this discussion)

-qlist, -qsource options and -V and -v c89 and xlc flags provide a quick way to find out what the values of LSEARCH and SEARCH are.

As Visda has discussed before in her blog post, NOSEARCH() and NOLSEARCH() reset the respective option value back to empty.

Last topic I wanted to touch was preprocessor macros and what options are available when dealing with them.

Michael Wong has posted here a way to find compiler predefined macros on AIX using the SHOWMACROS option. zOS unfortunately does not have this option till V1R11. The best equivalent is to use the makedepend utility -Wm,list option and then view depend.lst. It will contain a list of predefined compiler macros. However, makedepend utility is being deprecated since V1R11 in preference to built-in -qmakedep option.

Nevertheless most macros should be mentioned in the zOS manuals related feature sections. Here is a small list from the manual.

If you already have the macro name you wish to use, have a look at the -qEXPMAC option. This option will show you the value of the macro in the source listing. It is most useful when combined with -qshowinc.

I hope this gets you started on the right track.

In a couple of previous posts ( TOC Overflow: what is it, and why should you care?, Dealing with TOC overflow: the traditional approach ) I have presented the issue of TOC overflow. Now I will discuss some features of the XL compilers that can help bypass TOC overflow while minimizing any negative effects on runtime performance.

1. Minimal TOC: The option -qminimaltoc makes the compiler generate code that uses a single entry in the TOC for each compilation unit (in C/C++ a compilation unit is a source file). In order to do this, a separate level of indirection must be follow in order to access TOC-based variables. This means that the program will be larger and slower than if it did not have TOC overflow, but it will still be faster than using the -bbigtoc option. This is similar to the -mminimal-toc from gcc.

Furthermore, -qminimaltoc does not need to be used on all compilation units, so you can minimize the performance impact by using this flag only on compilation units that are not relevant for performance.

2. IPA: IPA is short for inter-procedural analysis, a form of compiler optimization that looks at the whole program, not just a single compilation unit. For this, the optimizer is invoked during the linking phase of your application, to perform transformations that can affect multiple compilation units.

Applying this process significantly reduces TOC pressure, and in most cases completely eliminates TOC overflow. It does so by restructuring your program to reduce the number of global symbols. The result is similar to what could be achieved through source changes, but avoiding the widespread manual source changes.

In the XL compilers, IPA is implied at optimization levels -O4 and -O5, but those also include other complex optimizations which may not be as relevant to commercial application development. One good alternative is the option -qipa=level=0, which applies a minimal level of whole-program optimization. This is often sufficient to eliminate TOC overflow, but in very large applications you may need -qipa=level=1 instead, which will perform a more aggressive reduction of the TOC requirements, at the cost of a longer compilation process.

Note that for whole-program analysis to be performed, the -qipa option needs to be specified both at the compile and link command lines. This means that the linking of the program has to be done through the compiler driver (xlc, xlC or cc) instead of directly through the system linker (ld). For maximum effect, all source files should be compiled with -qipa, but it is possible to mix-and-match objects compiled with different options and have them interoperate.

If you try these options please add comments to this post describing your results.

Here we are eleven days into a new year; and I would like to wish all of you a happy belated 2009! May this be a year of making technology less complex, more intuitive, friendlier and greener.

Is it clear why we have RENT|NORENT compiler option? Do you know that C++ always uses constructed re-enterancy? Can you imagine how applications can benefit most from this?

Recently, I developed a greater appreciation for RENT option, that is after I banged my head against the wall to REALLY understand what RENT is all about in order to fix a related bug.

Reentrancy becomes important when a rather large application have multiple users who may access it concurrently, for example Oracle. Oracle applications developed to run on z/OS will access Oracle interface which is a reentrant code.

Per z/OS Language Environment Programming Guide the following routines must be reentrant.

• Routines to be loaded into the LPA or ELPA
• Routines to be used with CICS
• Routines to be preloaded with IMS

Furthermore, a large application with many concurrent users will benefit from reentrancy, ie runs faster, because there is less paging to auxilary storage --variables are placed in writeable static area.

This is done for you, if your application is written in C++ or is a DLL. Compile with "-qlist" and you see RENT in the Compiler Option Listing. A C program, however, can be naturally reenterant, that is the user code doesn't change the static storage.

For example:

extern int x;

int main()

{

return x;

}

is naturally reentrant because the value of x is not changed by main.

A C program can be made reentrant, constructed reenterancy by specifying

a. -qRENT on the command line (in HFS) or RENT in CPARM (in BATCH)
b. #pragma variable(x,RENT) in the source

There you have it. In a nutshell: reentrancy comes for free with C++ and DLL code, for the rest you have ways to make the code reentrant; if your code is big and is called/used by multiple users at the same time you want to take advantage of this feature because it will improve the run time performance of your application.

Dual cores have become household products, yet we see little change in the performance of the application that run on these machines. Just this morning, I started Firefox and Lotus Notes at the same time on my dual core T60p laptop, and had to wait a long time before I could use either one of the applications.

To take full advantage of hardware horse power: software has to keep up, compiler has to generate better code, or both.

Recently, I have been experimenting with compiler options to see how much I can tune the size and performance of the executable. I started off by identifying the bottle neck(s), the hottest loop(s); places in application where CPU spends most of its processing time. I used hardware profiling tool to gather the data and Visual Performance Analyzer to identify the bottle necks. With VPA you can identify the bottle neck down to the hardware instruction level. I just mention this here to give you a sense of what the tool can do. But, for the purpose of my experiment, the function name was sufficient.

Given the function name, the rest was trial an error. I used a variety of performance tuning compiler options, e.g. HOT, INLINE, HGPR, all at O3; and pragma directives, #pragma option_override and #pragma unroll.

To conclude, I have to stay a lot relied on the high level code. I couldn’t find a hat that fits all. Playing with #pragma unroll proved to be interesting. Given the limited number of registers on z/OS loop unrolling opportunities were very limited.

z/OS 1.10 introduces the dbgld utility. It will significantly improve the startup time and performance of dbx. So how does it work, and how do I take advantage of it? Read on for more details.

When you specify -g option during C/C++ compilation, a debug sidefile (.dbg) is generated for each input source file. To debug an application, dbx would need to locate all of its debug sidefiles, and process debug information stored within them, this is a necessary but time consuming task. dbgld can perform this task before dbx is invoked. In addition, dbgld consolidates all the debug sidefiles into a single sidefile (.mdbg). You can now debug your module, by bringing along just a single .mdbg file.

Speedy debugging in 3 easy steps

1. Compile and bind program with -g option:
   xlc -g hello1.c hello2.c hello3.c -o hello
   This will create an executable module hello and 3 debug sidefiles hello1.dbg hello2.dbg and hello3.dbg
2. Invoke dbgld on the module:
   dbgld hello
   This will create the consolidated debug sidefile hello.mdbg
3. Finally, debug hello with dbx:
   dbx hello
   dbx will automatically use hello.mdbg if it finds it in your current directory.

For more information about dbgld, please refer to the XL C/C++ User's Guide:
XL C/C++ User's Guide: Chapter 22. DBGLD

z/OS C/C++ Performance Features

Compliers are an important tool in your development environment. A good optimizing compiler generates performance code without you worrying about the low level details of the OS, internals of the runtime environment and hardware architecture. You can concentrate on the business logic in your application. But optimization can take up a lot of resources, both in terms of compilation time and memory space. XL C/C++ provides an optimization option with 3 levels (called suboption 1, 2 and 3). Level 1 and 2 represent compromise between execution time performance and compile time. This is the appropriate setting in most cases. But there are situation where you want to let the compiler to exploit as much optimization opportunities as it could, regardless of compilation resources. This is what optimization level 3 (suboption 3) does. Experience shows that most of a program's time is usually spent in certain areas of the code (the 80-20 rule), one way of using level 3 is to apply it in those source files which contains hot spots of the application. The rest of the files, responsible for tasks like initialization, termination, error handling, and user interactions, can be compiled with lower optimization levels, getting the best of both worlds.

This leads to the general direction of putting more control into the programmer's hand in controlling actions taken by the compiler. An important technique in optimization is loop unrolling. Unrolling eliminates the loop control checking, which in turn can expose more optimization opportunities between loop iterations. But this is a two edged sword as too much unrolling can increase code size and larger memory footprint for the application. The optimizer normally makes decisions basing on it's analysis of the code. But often times the programmer knows which are the hot loops and can direct the compiler to do unrolling on specific ones. This is the purpose of the UNROLL option and the corresponding pragma directive. You can use these to control which loops to unroll, and by how many times, applying the optimization benefit to code that are most frequently executed.

The idea of execution frequency and its impact on optimization leads to the idea of Profile Directed Feedback (PDF). This is an enhancement to inter-procedural analysis (IPA), and is used together with the IPA option. IPA performances whole program analysis; it looks at code from all source files instead of just one. This leads to many more optimization opportunities than a normal optimizer usually discovers. PDF brings this a step further -- the compiler makes use of profiling information to direct its optimization. The steps to use PDF is as follows: 1) Build the application with the opion PDF1. This results in a load module with instructmentation to collect profiling information. 2) Run the instructmented module with typical input. The instructmented code will produce a data file containing the execution frequency of the code. This is called the training run. 3) Build the application again with PDF2. This is the production build where IPA makes use of the profiling data collected in step 2 to perform aggressive optimization. The result is a load module tuned to run optimally with the typical input used in the training run. In order to use this successfully, input data in the training run must be selected carefully. It is most effective when the production data profile on average doesn't vary too much.

The above are just a few of the features that can boost your program’s performance. You can find out more in the Programming Guide (http://publibz.boulder.ibm.com/epubs/pdf/cbcpg190.pdf, Part 5, chapter 36-42).