Commercial Computing with C/C++

RE: METAL C Compiler Option in z/OS

chwan — Thu, 05 Nov 2009 22:33:07 GMT

Hi,

What you plan to do should work. Provided you'll reload r12 with the initial 64-bit token returned from __cinit before you invoke malloc, free, __cterm, etc. I suppose you already knew how to do that from the example in Chapter 3 in Metal C Programming Guide and Reference. Mixing LE programs and Metal C programs requires your understanding on the difference in environmental requirements. R12 is one in particular that LE side of the program depends on.

You can build such code in JCL with the proper options, e.g. Metal, GENASM, LP64, etc. The only requirement is that you compile 31-bit and 64-bit Metal code in separate steps.

If you are looking for larger virtual storage for your data, I just want to point out that there are couple more choices in addition to use Metal C malloc. You can explore using the far pointers to address data in data spaces. You can also use MVS services directly to have full control over how and where you want to allocated your storage to avoid the need to tie down R12 when you use __cinit/__cterm.

In terms of the data set up for __cinit, unless you want to specify subpool, I think you can set all fields to 0. You may want to override the default initial heap size and increment size to tune malloc performance.

Hope you have fun with this. If you have further questions you are welcome to contact me directly using my email address which you can find in my profile.

RE: METAL C Compiler Option in z/OS

alainatogc — Wed, 04 Nov 2009 12:57:05 GMT

Hello Raym,

we want to use Metal C to create a 31-bit C routine that will be called from a 31-bit Z/OS PL/1 program in a Z/OS Batch JOB using JCL.

using an Metal C environment handle, we want to call the C routine in the following entry points:

Create a Metal C environment, returning the handle( using the __cinit function
passing the handle, allocate 64-bit very large buffer(using the A-mode switching feature provided from Z/OS v& rel 10 onwards, returning to the PL/1 program a 8 byte value containing the allocated pointer

-passing the handle and the 8 byte value containing the pointer and the address of a 31 bit adress of Buffer in the PL/1 program + its length,
a function to copy that storage into the gotten 64-bit buffer
-likewise a function to retrieve storage from the 64-bit buffer based on a key

finally a function to release the Metal C environment, freeing the 64-bit very large buffer

Looked at the Metal C doc a few things are unclear to me:
-A-Mode switching sample build + parms is under uss, I assume equivalent JCL can build a Load module under a PDSE, linking the PL/1 and Metal C routines together

what needs to be coded in the _cinit and __cterm routines.

Looking for an expert able to help out

RE: Demistifying zOS Include files

VolodymyrPaprotski — Thu, 15 Oct 2009 12:41:02 GMT

-I command line option is treated as it it were -qsearch and for most purposes are interchangeable.

You may convince yourself of this by running the xlc command with -v flag:

{quote}
$ xlc -I. -v test.c
...object(./test.o),-qnosearch,SEARCH(.),SEARCH(//'tsctest...
{qoute}

Volodymyr

RE: Demistifying zOS Include files

timhahn — Thu, 15 Oct 2009 12:16:29 GMT

Hello,

This is great information!

How do command-line options such as "-I <directoryName>" specified when the compiler is invoked (through a USS command shell or even from a Makefile) factor into the mix ?

Tim Hahn

RE: A quick one on SEARCH/NOSEARCH

VolodymyrPaprotski — Wed, 14 Oct 2009 21:28:38 GMT

As promissed, I have created a post here on zOS Include files and their options.

RE: Speedy debugging on z/OS

kelly.arrey — Fri, 14 Aug 2009 13:16:54 GMT

Hi Evandras,

Can you please send the PMR numbers to me at kjarrey@ca.ibm.com ?

Thanks,

Kelly Arrey

IBM z/OS XL C/C++ Release Manager

RE: Speedy debugging on z/OS

evandras — Fri, 14 Aug 2009 08:18:10 GMT

Hi

For month we try to debug via "dbx" with the remote debug option, large C/C++ projects from USS
Till now without sucess, we have opened several PMR's.
We are very interested about the "dbgld".
As I heard ithe "dbx" will support it from z/OS 1.11

RE: TOC Overflow: Getting help from the XL compilers

Sean_Perry — Wed, 03 Jun 2009 15:52:57 GMT

I'm not sure what you mean by component. I think you mean a component is the output of a link step. If so, yes use one registry for each component. Think of the registry as a database that keeps track of where a template has been instantiated. To avoid unresolved symbols, all of the .o files the registry knows about (i.e. .o files that contain instantiations) would need to be included in the link step.

You don't need to use the -qtemprecompile option. It is on by default.

An incremental build is a build you do after changing a few files. Make will only recompile the source files that are needed to update the out of date .o files.

The -qtemplateregistry option will help compile time more than run time at any level of optimization. The higher levels of optimization will be able to discover most of the duplication that -qtenplateregistry would find. It really comes down to either avoiding the extra code up front or letting the compiler discover it.

RE: TOC Overflow: Getting help from the XL compilers

reganjeremy — Wed, 27 May 2009 12:44:54 GMT

Yes, the build does work with neither qtempinc nor qtemplateregistry. The template instantiations are only defined in one lib but when i used tempinc it was putting the symbols to be exported for each template instantiation into the pfactimf.o file (the template class itself object file in the tempinc directory).

quote sean perry
I would recommend one registry per shared library and to not use a registry when building the static libraries. You can add the -qfuncsect option to all the compile commands to help eliminate the extra duplicate instantiations in the static libraries.
end quote

So you're saying if you use templateregistry you use it only one per component? What I'm concerned about is the examples and the way tempinc works would lead me to believe that for templateregistry you would have to do the same thing, so that the templateregistry has the symbols defined in there, and then you link your executables into that lib so that it can find them at runtime.

I am okay without funcsect at least at first and I had tried to use -qtemplateregistry -qtemprecompile or what have you the first time I did it because that seemed like what was the "recommended" combination. You said "The -qtemplateregistry option can cause recompiles during the link step, but those are rarer and only happen when doing an incremental make." I am not sure what you mean by an incremental make here; I am doing an incremental build of components and then they sometimes need other components' template instantiations inside them. In the end the executables are built and must be able to find everything, we are using the runtime-linking option so I don't know about some things until I try to boot it. I can understand some motivation for moving to static libs but the executables would be very large, I think +100MB and link times are slow for that. Also I would have to figure out all the 3rd party linking again and the last few things I figured out were not too fun.

I am going to try to create another build using templateregistry and just keep the current build as a backup of a working O5 build. Do you think though that this method of template instantiation will really have a measurable impact on performance over just using the best optimization?

RE: TOC Overflow: Getting help from the XL compilers

Sean_Perry — Tue, 26 May 2009 16:07:39 GMT

Does your build work without -qtempinc or -qtemplateregistry? The first tricky issues I see is making sure that your singletons that are template instantiations are only defined in one shared library. By defined, I am talking about at the object file level. You want to make sure the that the singletons are only defined and exported from one shared library and that the rest import that definition. Once all that is straight adding in -qtemplateregistry should be fairly simple. I would recommend one registry per shared library and to not use a registry when building the static libraries. You can add the -qfuncsect option to all the compile commands to help eliminate the extra duplicate instantiations in the static libraries.

I'd recommend using -qtemplateregistry over -qtempinc. As you have noticed, -qtempinc requires source changes to use it and can have problems when linking. The -qtemplateregistry option can cause recompiles during the link step, but those are rarer and only happen when doing an incremental make.

RE: Compilers, what they might do ...

Sean_Perry — Tue, 26 May 2009 14:14:47 GMT

Add TEMPLATEREGISTRY to that list of template options. The template registry solves many of the problems that TEMPINC had. The most notable improvement is template registry doesn't require reorganizing your source. I'd suggest using TEMPLATEREGISTRY over TEMPINC and migrating to TEMPLATEREGISTRY if you are using TEMPINC.

RE: TOC Overflow: Getting help from the XL compilers

reganjeremy — Tue, 26 May 2009 13:18:07 GMT

Sorry one additional piece of information, the current AIX build times are about:

No optimization:

3 hours

O3 optimization, no qipa:

6 hours

O5 optimization:

30 hours

RE: TOC Overflow: Getting help from the XL compilers

reganjeremy — Tue, 26 May 2009 13:15:44 GMT

Hi Sean

I think I can describe the code structure only to a point but I think enough for understanding. There is a root-level directory and then each folder in there is either a library or a server but each server can share many of the other libraries. So in the makefiles there is a generic "target" for each component and then a list of the libs it depends on. This list is used for both the includes in the compile step and the libs in the link step.

When I used tempinc what I did was I specified this one area ../tempinc from each component so that the libs and the servers both shared the same tempinc stuff. I divided up our lowest level template class as a test into the .c and .h format; this is a persistent object factory class. It looked good to me and what tipped me off is how I determined that as I went up the components in the list, it was building onto the file and what would fail to compile was due to missing symbols from #include files that could be reached from a particular template class instantiation. The tempinc/pfactimf.o it was building got to be about 50 megs by the time I gave up. Another factor is that I had to change the makefiles in new ways because a side effect of that incremental building is how it now links against an object file that has symbols from every other template class instantiation before it. This is not desirable and when I explained this to one of our senior architects he said he did not like this approach and he actually knew it would involve changing the makefiles this way before I told him about it. He recommended trying one with -qnotempinc I think also and that one did not work either for me.

What happens is this template class has static methods in it and they are using a class-local singleton to access stuff that would be instantiated with the class when a concrete implementation calls the method. In the case of -qnotempinc and with -qtemplateregistry, the servers would fail to start up giving an unresolved external for each of the template instantiations that it needed but couldn't find. I tried all sorts of permutations of changing if each component made its own templateregistry per folder, or using one big one in the source root, but they both yielded the same results. What I realized is that I had some other build where I had introduced using -qmkshrobj and I thought it was only working for libraries but not for servers. I found out though that it was working and this involves the least code changes of all, none! =) When I used tempinc I only got about 50% through the makefiles and it had already taken me days to get there. Not only that, but the link times were ridiculously getting longer and longer, up to an hour per component by the end.

It is true that shared libraries complicates the template instantiation and you can see what I mean about how that is not a ready change here. I think they originally went with the shared memory model for two main reasons:

1) link time on solaris used to be astronomical even in its current state, we know compile times would be even longer now and we can not afford a 48-hour build on modern hardware when win32 build times are so short, usually 3 hours.

2) ease of maintenance for delivering new shared libs

I think another purpose and maybe the most important at least to me is the possibility that the shared lib memory model should be faster on the AIX kernel. What I have been curious about as well is if all of the processes are sharing the libraries, then doesn't that mean there is contention there or does it generate a new copy? And if so, then isn't managing the copies of the code expensive?

Thanks for your assistance,
Jeremy

RE: TOC Overflow: Getting help from the XL compilers

Sean_Perry — Mon, 25 May 2009 18:03:00 GMT

Hi Jeremy.

Let me try to address the issues you have seen. Firstly, you noted that you need to pass the same compilation options to the link step as for compilation. You don't have to always do this. You only need to do this if the link step will, or could, result in a recompilation of a source file. It appears that was happening for you. The -qtemplateregistry is one option that can cause compilation during the linking process. What options were you using? The options that "need" to be passed to the link step will depend on the organization of your source code. If an option must be specified (like some -D or -I option) then it should be included during the link step if that link step can cause a recompilation.

You also mentioned about exploring the -qtempinc and -qtemplateregsitry options. These options are independent. I'll check out the manuals to make sure they don't say you can't use -qtemplateregistry unless the code compiles with -qtempinc and -qnotempinc. The manuals should have an example of how to organize your code so you can use -qtempinc, -qnotempinc, -qtemplateregistry or another compiler's implicit instantiation method. Most code that already compiles and links is ready to use -qtemplateregistry. I think you should be able to use -qtemplateregistry without making any source changes.

You mentioned you are building shared libraries. These complicate managed template instantiation. If you don't share object files that contain template instantiations between shared libraries then you can use the -qtemplateregistry option and use a different registry for each shared library.

Finally, You can can use -qtemplateregistry with -qipa/-O4/-O5.

RE: TOC Overflow: Getting help from the XL compilers

reganjeremy — Fri, 22 May 2009 14:27:33 GMT

I was able to successfully use the -qipa=level=0 to solve this issue. I ended up using the -O5 level for every component except two where I used this one in stead. The follow-up issues were that with O5 the build takes about 35 hours which is a bit high but not bad if you just make a performance build at the end of all other development activities (I guess maybe another UAT run or something if it is considered a high risk but there do not seem to be any ill effects).

A main issue that I ran into was the use of the -qmkshrobj flag for the link step in stead of using makeC++SharedLib. In this case, I had problems because we use a "template"-style makefile that builds the commands for each component automatically: compilation and then the link step. In this case I found out I have to pass the library and includes exactly the same between compilation and linking or you have problems with template method instantiation. It will look like it works but at runtime if you have template methods that do static methods or stuff like that then those methods will return null pointers.

Once I got the -qmkshrobj option working then I could finish the -O5 build. We are currently now running performance tests and seeing a small improvement over O3 but maybe not enough to justify the longer compile time. We are hoping to either use the Quantify tool or some of the run-time tuning stuff, or follow up with our IBM contact regarding kernel or scheduler tuning.

I looked for a long time at the -qtempinc and -qtemplaterepository link options but for one the tempinc required me to change too many makefiles and was very long compile times, I expected to add an extra week of effort to convert our app to use that style of template compilation. The templaterepository I couldn't get to work on its own either and I think it will only work as it says in the migration guide that templaterepository will only work if tempinc and notempinc BOTH already work. Also, perhaps you can respond as to whether or not it is really true that -qtemplaterepository negates the use of -qipa and how does this affect O4 and O5? I am very interested in this piece of information.

Thanks for your articles, very informative.
Jeremy

RE: Dealing with TOC overflow: the traditional approach

reganjeremy — Mon, 20 Apr 2009 16:04:19 GMT

Thank you very much for your time and valuable information. I will update you on how it goes with the suggestions.

RE: Dealing with TOC overflow: the traditional approach

Raul_Silvera — Mon, 20 Apr 2009 15:11:57 GMT

You're actually not very far away from the limit of the TOC. Are you linking with -qipa, or another option that implies -qipa? If not, I suggest you try linking with -qipa=level=0 and that should eliminate the overflow for you.

Another alternative is to use -qminimaltoc on the largest source files you have; that will likely be enough to eliminate TOC overflow, and likely the performance will be better than what you're currently getting with -bbigtoc.

About the Internal compiler error you saw before, that likely is a compiler bug. I would encourage
you to report it to IBM through your service channels.

Also, about your question regarding -O5, yes basically -O5 is equivalent to -O3 -qhot -qipa=level=2 -qarch=auto

RE: Dealing with TOC overflow: the traditional approach

reganjeremy — Mon, 20 Apr 2009 14:07:10 GMT

Some additional information... Here is the TOC warning we get currently:

ld: 0711-783 WARNING: TOC overflow. TOC size: 71748 Maximum size: 65536
Extra instructions are being generated for each reference to a TOC
symbol if the symbol is in the TOC overflow area.

This is an ERP system and the component is the reference classes for performing spending operations so this is a very critical component for spending. Due to the inter-dependencies in spending logic this is also a difficult one to break up.

RE: Dealing with TOC overflow: the traditional approach

reganjeremy — Mon, 20 Apr 2009 13:05:53 GMT

With O5 the component actually cannot compile at all, it gives

1586-494 (U) INTERNAL COMPILER ERROR: Signal 11.

What I have been doing is using O5 for all components that do not break with it, and then the few that it can't do I revert to O3. We do use -qipa but have not seen significant gain from it, I think even less than the anticipated 10-15%. I believe the main differences between O3 and O5 can also be added piecemeal with various options to the -qhot option as well is that correct? It seems the problem with bigtoc does remain though. Also, this is a very large application. The components which fail to be under the threshold have maybe 100+ objects in that one library, each .cpp file associated with that object I would say is roughly 10k but there are some that are 50-60 and some that are over 100k for one source file. We are also currently working with Ashu Gupta at IBM about our performance issues but since we have not specifically addressed bigtoc in the past with them I am still interested in your input. Ashu has also indicated he will follow up with the bigtoc issue.

RE: Dealing with TOC overflow: the traditional approach

Raul_Silvera — Mon, 20 Apr 2009 02:29:53 GMT

Thanks for your comment. If you compile and link with -O5 (or any other compiler option that implies -qipa), you should not see TOC overflow except for extremely large applications. Are you sure you're still getting TOC overflow even with -O5? The best way to check for that is to link with -O5 and not specify -bbigtoc: the linker will abort if you reach TOC overflow and did not specify -bbigtoc.

If you did not link with -qipa in the past, I would suggest starting with -qipa=level=0, and moving up from there up to ipa=level=2, which is the level of IPA implied by -O5.

RE: Dealing with TOC overflow: the traditional approach

reganjeremy — Wed, 15 Apr 2009 14:56:15 GMT

We do have an issue with this. Currently we use the -bbigtoc option however we have lurking performance issues that I believe may be substantially due to the use of this option. We are currently testing with -O3 and -O5 optimization levels and only seeing about 25% improvement, but still performance of our application vs. win32 is about only 55% where it should be over 100% with the hardware we are using. The library that has the error thrown during compliation contains about 65 objects, each is very large with many symbols so we may end up breaking this library into smaller pieces but this is definitely a difficult task to do the dependency analysis and break it up. I am very interested in learning more about other ways that the compiler has to deal with this issue if we can use those in stead of doing any breaking up of the library.

RE: Compilers, what they might do ...

Visda — Thu, 05 Feb 2009 15:40:51 GMT

Since, I last posted I have done more selective optimization. I have done it to debug functional problems, I have done it to get the performance back. I have found #pragma option_override to be very useful. So you see, I have not forgotten you. Write me how has #pragma option_override made an impact on your application compile and run times.

RE: Compilers, what they might do ...

johnpm — Wed, 04 Feb 2009 20:18:58 GMT

Hi Visda. Don't forget me! By the way, I just reread your post. Why did you "want to disable optimization at the subprogram level"?

RE: Compilers, what they might do ...

Visda — Tue, 25 Nov 2008 03:34:59 GMT

Hello, John: Interesting questions! Yes, in fact I am studying this at the moment. I will write more soon.

RE: Compilers, what they might do ...

johnpm — Mon, 24 Nov 2008 15:22:52 GMT

HI Visda,The ability to pick and choose what you optimize is interesting.

You said....."I wanted the entire compilation unit, CU, to be compiled at O3 except one, non-overloaded function. I used* #pragma option_override* to bump down the optimization level to O2 for that one function...."

Did bumping down the optimization level for that one particular function make a big performance difference? What criteria did you use to decide to downlevel the optimization for that one function?