I Use the Ib'm i

It seems that I've been forever reading blog, forums, articles, and tweets arguing one way or the other about what to call the latest incarnation of the pantheon that we all worship. It's gotten so bad that Cardinal Klement had to write an encyclical to set some ground rules as to the proper terminology. I've been watching this argument go on and on and on with no end in sight and I don't for a minute think that this blog post will put an end to the oodles of disk storage dedicated to its living spirit, but I do hope that this story might have some bearing on it.

For years I was in the AS/400 camp. Meaning I was pissed off at our lord Ib'm for continually re-naming its liturgy and the damage it was continually causing to the AS/400 brand. That is, until I had a conversation with someone outside of our rarefied community. When he asked me what platform I worked on I told him it was the "Ib'm i". When he inevitably asked "what's that?" I told him it used to be called the AS/400. To which he asked: "Wasn't that the dinosaur platform that was supposed to be dead or dying?" I realized at that moment that the AS/400 brand was dead and it was never coming back. The whole world believes that the AS/400 is an old out-dated platform that is slowly withering away, like the HP 3000, or the DEC VAX.

Yes, I know they're wrong, but in a way they're also right, because while the AS/400 platform is alive and well, it's the AS/400 brand that is a dead-end. It belongs to the world of 5250 terminals, Twin-axial cables, Token-Ring networks, SNA, and SNADS. A world dominated by libraries, physical & logical files, OPM, RPG-III, CLP, EPM-C, SEU, SDA, RLU, out-files, and Net.Data.

Today there's a bright and shiny new brand that our lord Ib'm calls the "Power Systems running i", but since that's too much of a mouthful I like to use the much simplified version "Ib'm i". It belongs to a new world of PC clients, CAT5 cables, Ethernet & WiFi networks, TCP/IP, FTP, Apache, and NetServer. A world dominated by directories, relational databases (tables, indexes, views, triggers, constraints, etc.), SQL, ILE RPG (free-format), ILE CL, ILE C, C++, Java, API's, QSH, PASE, Eclipse, Rational, WebServices, XML, and PHP.

We're on the cusp of a new dawn and if we play our cards right it will flower into a beautiful meadow. When the outside world is invited to sees how wonderful it really is they don't need to be reminded of the dusty over grown field behind them that was repeatedly sown with salt. I'm never going to say to anyone again that I work on an AS/400 and I encourage everyone who reads this to do the same. The "Power Systems running i" or "Ib'm i" is the platform I use.

Say it with me now: "I used to use the AS/400, yes that old dinosaur. It's true; it's gone, but it's been replaced by something better, something called the Ib'm i." Amen.

CL Program to Apply Technology Refresh PTF

I had a deep sense of foreboding when I found that a HIPER PTF I downloaded came with technology refresh PTF 5770999-MF99002. These things are a pain in the butt to apply as it requires one to do the following steps: 

1. Perform a full system save, like GO SAVE, option 21.
2. Apply all microcode PTFs permanently.
3. Load the TR PTF and set it to be applied temporarily.
4. IPL the system.
5. Apply the TR PTF permanently.

On our system there were two pre-requisite PTFs that were downloaded with MF99002; plus we already had a number of microcode PTFs that were set to be applied during the next IPL. I didn't want to permanently apply these new microcode PTFs, so I took the following steps:

1. Unloaded the microcode PTFs set for delayed apply, except for those that were pre-requisites to MF99002.
2. Performed a full system save.
3. IPLed the system to apply the pre-requiste microcode PTFs.
4. Permanently applied all microcode PTFs.
5. Loaded MF99002 and set it to delayed apply.
6. IPLed the system.
7. Permanently applied MF99002.
8. Loaded the rest of the microcode PTFs and set them to delayed apply.
9. IPLed the system.

Our weekly "full system save and IPL" is full automated, so I needed some way to insert a CL program into this mix that would do the job. After finishing it I realized that others might be able to use it as well. It was designed to run in the QSTRUPPGM program, before it actually starts anything. It's assumed that the very first IPL, after the full system save, would first apply any PTFs that were pre-requisites of the TR PTF. All pending microcode PTFs to be applied, including the TR, would remain in a "Save file only" un-loaded state.

After that first IPL it would find that the TR PTF wasn't loaded yet, so it would apply all microcode PTFs permanently, load the TR PTF, set it to delayed apply, and IPL. When the system came up again it would find that the TR PTF was applied. It would then permanently apply it, load the rest of the microcode PTFs sitting in *SERVICE, set them to be applied delayed, and IPL again.

I've included the code below:

---

/******************************************************************************/
/* Program....: APYTRPTF                                                      */
/* Description: Apply technology refresh PTF.                                 */
/* Author.....: Joe Code                                                      */
/* Date.......: 02/18/2011                                                    */
/*                                                                            */
/* Create                                                                     */
/* ------                                                                     */
/* CRTCLMOD DBGVIEW(*ALL)                                                     */
/* CRTPGM USRPRF(*OWNER) AUT(*EXCLUDE)                                        */
/* CHGOBJOWN OBJTYPE(*PGM) NEWOWN(QSECOFR)                                    */
/* GRTOBJAUT OBJTYPE(*PGM) USER(QPGMR) AUT(*USE)                              */
/*                                                                            */
/* License                                                                    */
/* -------                                                                    */
/* This library is free software; you can redistribute it and/or modify it    */
/* under the terms of the GNU Lesser General Public License as published by   */
/* the Free Software Foundation; either version 2.1 of the License, or (at    */
/* your option) any later version. (Unlike the normal GNU GPL, the "lesser"   */
/* GPL allows libraries to be used in commercial/proprietary software.)       */
/*                                                                            */
/* This library is distributed in the hope that it will be useful, but        */
/* WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANT-       */
/* ABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General    */
/* Public License for more details.                                           */
/*                                                                            */
/* You should have received a copy of the GNU Lesser General Public License   */
/* along with this library; if not, write to the Free Software Foundation,    */
/* Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA           */
/* (http://www.gnu.org/licenses/lgpl.html#SEC3)                               */
/******************************************************************************/

PGM

   COPYRIGHT TEXT('Copyright (c) 2011 Joe Code')

   DCL VAR(&COUNT)     TYPE(*DEC)  LEN(3 0)
   DCL VAR(&ERROR)     TYPE(*LGL)          VALUE('0')
   DCL VAR(&RCVVARLEN) TYPE(*INT)  LEN(4)  VALUE(131)

   DCL VAR(&PTFINFO)   TYPE(*CHAR) LEN(50)
   DCL VAR(&PTFID)     TYPE(*CHAR) LEN(7)  STG(*DEFINED) DEFVAR(&PTFINFO  1)
   DCL VAR(&PRODID)    TYPE(*CHAR) LEN(7)  STG(*DEFINED) DEFVAR(&PTFINFO  8)
   DCL VAR(&RLSLVL)    TYPE(*CHAR) LEN(6)  STG(*DEFINED) DEFVAR(&PTFINFO 15)
   DCL VAR(&CCSID)     TYPE(*INT)  LEN(4)  STG(*DEFINED) DEFVAR(&PTFINFO 21)
   DCL VAR(&CLSPTFF)   TYPE(*CHAR) LEN(1)  STG(*DEFINED) DEFVAR(&PTFINFO 25)
   DCL VAR(&RESERVED)  TYPE(*CHAR) LEN(25) STG(*DEFINED) DEFVAR(&PTFINFO 26)

   DCL VAR(&RCVVAR)    TYPE(*CHAR) LEN(131)
   DCL VAR(&BYTESRTN)  TYPE(*INT)  LEN(4)  STG(*DEFINED) DEFVAR(&RCVVAR   1)
   DCL VAR(&BYTESAVL)  TYPE(*INT)  LEN(4)  STG(*DEFINED) DEFVAR(&RCVVAR   5)
   DCL VAR(&LOADEDSTS) TYPE(*CHAR) LEN(1)  STG(*DEFINED) DEFVAR(&RCVVAR  41)
   DCL VAR(&IPLACTION) TYPE(*CHAR) LEN(1)  STG(*DEFINED) DEFVAR(&RCVVAR  66)

   MONMSG MSGID(CPF0000 CEE0000 MCH0000) EXEC(GOTO CMDLBL(ERROR))

   QSYS/CHKOBJ OBJ(QGPL/APYTRPTF) OBJTYPE(*DTAARA)
   MONMSG MSGID(CPF9801) EXEC(QSYS/CRTDTAARA DTAARA(QGPL/APYTRPTF) +
      TYPE(*DEC) LEN(3 0) VALUE(0) TEXT('Stop infinite loop if error +
      applying TR PTF.'))

   QSYS/RTVDTAARA DTAARA(QGPL/APYTRPTF) RTNVAR(&COUNT)
   CHGVAR VAR(&COUNT) VALUE(&COUNT + 1)
   QSYS/CHGDTAARA DTAARA(QGPL/APYTRPTF) VALUE(&COUNT)

   CHGVAR VAR(&PTFID)    VALUE('MF99002')
   CHGVAR VAR(&PRODID)   VALUE('5770999')
   CHGVAR VAR(&RLSLVL)   VALUE('V7R1M0')
   CHGVAR VAR(&CCSID)    VALUE(0)   /* Use job's CCSID. */
   CHGVAR VAR(&CLSPTFF)  VALUE('0') /* Close PTF files afterwards. */
   CHGVAR VAR(&RESERVED) VALUE(' ')

   QSYS/CALL PGM(QSYS/QPZRTVFX) PARM(&RCVVAR &RCVVARLEN &PTFINFO +
      'PTFR0100' X'00000000')

   SELECT

      WHEN COND(&BYTESRTN < &RCVVARLEN *OR &BYTESAVL < &RCVVARLEN) +
         THEN(QSYS/SNDPGMMSG MSGID(CPF9898) MSGF(QCPFMSG) MSGDTA('No data +
         returned') TOPGMQ(*PRV (*PGMBDY)) MSGTYPE(*ESCAPE))

      WHEN COND(&LOADEDSTS = '0') THEN(DO) /* Not loaded? */
         IF COND(&COUNT = 1) THEN(DO)
            QSYS/APYPTF LICPGM(5770999) APY(*PERM)
            MONMSG MSGID(CPF3660) /* No PTFs identified. */
            QSYS/LODPTF LICPGM(5770999) SELECT(MF99002)
            QSYS/APYPTF LICPGM(5770999) SELECT(MF99002) DELAYED(*YES)
            QSYS/PWRDWNSYS OPTION(*IMMED) RESTART(*YES) IPLSRC(B)
         ENDDO
      ENDDO

      WHEN COND(&LOADEDSTS = '1') THEN(DO) /* Loaded, but not applied? */
         IF COND(&COUNT = 1) THEN(DO)
            QSYS/APYPTF LICPGM(5770999) APY(*PERM)
            MONMSG MSGID(CPF3660) /* No PTFs identified. */
            IF COND(&IPLACTION = '0') THEN(QSYS/APYPTF LICPGM(5770999) +
               SELECT(MF99002) DELAYED(*YES)) /* IPL action isn't set? +
               Then set it. */
            QSYS/PWRDWNSYS OPTION(*IMMED) RESTART(*YES) IPLSRC(B)
         ENDDO
      ENDDO

      WHEN COND(&LOADEDSTS = '2') THEN(DO) /* Temporarily applied? */
         IF COND(&COUNT = 2) THEN(DO)
            QSYS/APYPTF LICPGM(5770999) SELECT(MF99002) APY(*PERM)
            QSYS/LODPTF LICPGM(5770999)
            MONMSG MSGID(CPF35A8) /* No PTFs to be loaded. */
            QSYS/APYPTF LICPGM(*ALL) DELAYED(*YES)
            MONMSG MSGID(CPF3660) /* No PTFs identified. */
            QSYS/PWRDWNSYS OPTION(*IMMED) RESTART(*YES) IPLSRC(B)
         ENDDO
      ENDDO

   ENDSELECT

   RETURN

/******************************************************************************/
       /*                                                                     */
ERROR: /* Global error handling section.                                      */
       /*                                                                     */
/******************************************************************************/

   IF COND(&ERROR) THEN(RETURN)  /* Prevents an infinite loop if an error     */
   CHGVAR VAR(&ERROR) VALUE('1') /* occurs within the error handling section. */

   QSYS/SNDPGMMSG MSGID(CPF9898) MSGF(QCPFMSG) MSGDTA('Unexpected error +
      occurred. See previous messages') TOPGMQ(*PRV (*PGMBDY)) MSGTYPE(*ESCAPE)

ENDPGM

---

On our small system this program added an extra two hours to the normal weekend backup/IPL. The only problem I ran into was that the microcode PTFs that I removed weren't loaded again. Apparently when you permanently remove a microcode PTF, it's also removed from the list of "*ALL PTFs in *SERVICE", meaning they can't be loaded when you specify LODPTF DEV(*SERVICE) SELECT(*ALL). You have to specifically name them in the SELECT parameter. Next week when I do this again I am going to load all of the microcode PTFs and leave the delayed apply flags shut off. (Except of course for the TR PTF's pre-requisites.)

Looking at the program again this morning I realized that if there was a subtle problem with applying the TR PTF then it might cause an infinite loop of IPLing, which would be very bad. I've since changed the program to include a safety valve in the form of a data area used to count the number of IPLs. If the count doesn't match up with what is supposed to happen then it doesn't do anymore IPLs. This amended version of the program compiles, but I haven't tested it yet. If you want to wait, it will be tested again next weekend when I apply the TR PTF to another system in our network. I will update this post with the results.

Update (2011-02-28): This version of the program did it's job and applied the TR PTF without error.

Update (2011-06-07): IBM has since issued PTFs to make this task easier, so it would seem that the program above is obsolete.

Parallel Lines

Had kind of a wild ride with this one liturgical conundrum, but I think it's resolved now. It was deep, but the ultimate solution was very simple.

The main program is written in ILE RPG and it does all of it's I/O using SQL. The main result set consists of multiple sub-selects that are UNIONed together into one giant pile. In the WHERE and HAVING clauses of most of these sub-selects it utilizes a user defined function (UDF) that does some data massaging.

This all ran perfectly fine on v5.4 of i5/OS, but after the upgrade to v7.1 of IBM i the job started behaving strangely. Sometimes it would work fine. Sometimes it would run inside the first FETCH statement all day long and take up huge amounts of CPU. Other times the UDF would function check because of invalid data in the strangest places, like on the RETURN statements.

To try and stem the function checking I added MONITOR/ENDMON blocks to all of the RETURN statements in the UDF. When an error occurred they would return *LOVAL instead. Then it started function checking on the statements that were calling the functions with the aforementioned function checking RETURN statements. It was very weird.

Over a two month period I was sending all kinds of traces, SERVICEDOCS, and job logs to Ib'm support angels. The last thing we did was set up a job watcher in iDoctor and the output found that when the FETCH was looping and it was doing so in one of the functions in the UDF. That's when they asked me if it the service program was specified to support multi-threading.

Doh!

As it turns out the SQL to create the functional stubs that called the RPG service program functions defaulted to PARALLEL while the program did not. So remember, if you're writing (or have written) an SQL UDF in a language other than SQL, don't forget that you might have to make your program or service program multi-thread capable. In RPG it means adding the keyword THREAD to the control specifications (aka H-specs) with either the *CONCURRENT or *SERIALIZE attribute. (THREAD(*CONCURRENT) did the trick for me.) Be sure also to thoroughly read the sections in the RPG and SQL manuals on multi-threaded processing.