****************NEWSLETTER FIFTEEN**************************************
             MXG NEWSLETTER NUMBER FIFTEEN  November 11, 1989           
Technical Newsletter for Users of MXG :  Merrill's Expanded Guide to CPE
                         TABLE OF CONTENTS                              
I.   MXG SOFTWARE VERSION 7 ANNOUNCEMENTS             pages   2 thru   7
  1. Prerelease MXG 7.2 now available upon request.             page   2
  2. Production MXG VERSION 7 planned for early 1990.           page   2
  3. Summary of new products and enhancements in MXG Version 7. page   2
  4. Summary of critical changes described in Change Log.       page   3
  5. Enhancements still under development for MXG Version 7.    page   4
II.  TECHNICAL NOTES                                  pages   7 thru   9
  1. MVS/ESA changes to SMF Accounting AND RMF capture ratio.   page   7
  2. MVS SMF Type 30 Subtype 3 TCB/SRB times after 922 ABEND.   page   9
  3. MVS SMF Type 26 Time stamps after JES2 Spool Transfer.     page   9
  4. MVS execution under VM, VM/XA, or PR/SM publication.       page   9
  5. MVS SMF Type 30 interval records missing for early ASIDs.  page   9
  6. MVS SMF Type 26 record corrupted.                          page   9
III. CHANGE LOG                                       pages  10 thru  44
     Changes through Change 7.165 to 7.036                              
IV.  "The History of SMF"                             pages  45 thru  76
      Paper presented at the 20th anniversary                           
      of the Availability of the IBM SMF Product                        
      at SHARE 73, Orlando, August, 1989.                               
I.   MXG SOFTWARE VERSION 7 ANNOUNCEMENTS                               
  1. Prerelease MXG 7.2 now available upon request.                     
 The Prerelease of MXG Version 7.2 is now available upon request from   
 Merrill Consultants (or through your "MXG Representative" outside USA).
 Request by phone (214-351-1966) or fax (214-350-3694) if you would like
 the prerelease sent to you (at no cost).  MXG is sent on 3480 cartridge
 tape media, unless you specifically requested MXG on a 3420 tape reel. 
 The Beta MXG 7.0 was installed at 130 sites, and their feedback was    
 incorporated in the Beta MXG 7.1 sent to a small number of additional  
 sites.  The MXG 7.2 is called a Prerelease now to indicate that it has 
 been validated sufficiently to replace MXG 6.6 at sites that need its  
 new product support, its enhancements or its corrections.              
   (Prerelease   MXG 7.2 was created 10/19/89 thru Change 7.161).       
   (Beta test of MXG 7.1 was created  9/14/89 thru Change 7.140).       
   (Beta test of MXG 7.0 was created  5/31/89 thru Change 7.098).       
  2. Production MXG Version 7 planned for early 1990.                   
 We still intend to ship the Production Version of MXG 7 in early 1990, 
 probably in early February.  Unlike the prerelease, which is sent only 
 if you request it, the "Production Version" of MXG is automatically    
 sent to all supported sites, accompanied by an MXG Newsletter.         
 See below for the enhancements that are not yet available in 7.2, but  
 that should be available in the Production Version 7 when shipped.     
  3. Summary of new products and enhancements in MXG Version 7.2.       
 The following major enhancements are available in MXG 7.2. Details     
 are in the actual Change Description (below) and documentation is      
 also found in comments in the members that are cited in the change     
 and in DOCVER and DOCVERnn members of the actual MXG 7.2 source.       
 New Products supported:                                                
  1. MXGTMNT, the long awaited MVS/XA or MVS/ESA MXG Tape Mount Monitor.
  2. 3490 and 9348 tape drives (cart and reel respectively) recognized. 
  3. ARBITER SMF records from Tangram product.                          
  4. NETVIEW Release 2 Hardware/Session Monitor External Log Record.    
  5. 3480/3490 Tape Compression (IDRC) recognized.                      
  6. JES2 mods to capture SYSOUT release timestamp in type 6 SMF record.
  7. FILEAID SMF record from COMPUWARE product.                         
  8. OMEGAMON Command Audit SMF record from Candle.                     
  9. AION Development System SMF record from AION.                      
 10. MDFTRACK SMF record from Amdahl MDF environment.                   
 11. TLMS (Tape Library Management System) catalog records.             
 12. TMS (Tape Management System) catalog records.                      
 13. RMF Monitor III VSAM data set records.                             
 14. Numerous additional DB2PM-like reporting in ANALDB2R.              
 15. DB2 Audit Class trace type 102 records.                            
 Enhancements to previously supported products:                         
  1. Identification of IBM RMF from Boole and Babbage RMF records.      
  2. DB2 SQL text of user inquires captured.                            
  3. ACF2 'ARE' data sets captured.                                     
  4. Additonal trending summaries and report examples.                  
  5. CPU Serials added to RMFINTRV.                                     
  6. ROSCOE 5.6 support for variable number of complexity levels.       
  7. ANALDSET enhancement to include VSAM type 64 records.              
  8. VMXGVTOC enhanced to expect up to 128 extents (for VSAM).          
  9. NETVIEW Release 2 Hardware Monitor External Log SMF record.        
 10. NETVIEW Release 2 Session Monitor External Log SMF record.         
 11. VM/XA new fields introduced by PTF VM35971.                        
 12. IMS 2.0 and IMS 2.1 response measurement corrections.              
 13. Step accounting fields added to PDB.STEPS.                         
 14. New MVS/ESA CPU timings in step records.                           
 15. Support for DOS/VSE Power 3.1.2.                                   
 16. Support for RACF 1.8                                               
 17. Validation and cleanup of the Trend Data Base and related macros.  
 18. Non-support of IMS 1.3 transaction processing. See Change 7.075.   
 19. Validation and cleanup of all reported MXG 6.6 errors.             
  4. Summary of critical changes described in  the Change Log (below).  
 Critical changes (i.e., those that correct error conditions) include   
  Change      Member        Description                                 
  7.142       VMAC7072      TYPE70 CPU under PR/SM may be wrong.        
  7.139       VMACSYNC      SYNCSORT record formats decoded wrong.      
  7.123       VMACDB2       DB2STAT0 wrong after PL29461.               
  7.115       VMACACF2      ACF2 datasets contain zero observations.    
  7.108       TYPEMONI      MONITASK "OVHD" transaction record short.   
  7.105       VMAC6         PSF FORM variable wrong.                    
  7.103       VMXG....      MWORK=28K option required for %MACROs.      
  7.098       BUILDPDB      PROTECT= option considerations.             
  7.083       RMFINTRV      RMFINTRV to count DASD I/O only.            
  7.082       VMAC110       CICSTRAN MAXTASK/SHRTSTOR.                  
  7.081       VMAC7072      TYPE72MN trashed under MVS/ESA.             
  7.078       VMAC77        TYPE77 trashed under MVS/ESA.               
  7.076       TYPEMONI      MONISYST trashed.                           
  7.065       VMAC102       STOPOVER on type 102 for several subtypes   
  7.054       VMAC28        STOPOVER on type 28 for subtype 5C.         
  7.050       VMACVMON      Concatenated VM/Monitor wrong.              
  7.047       ANALAUDT      Audit report corrections.                   
  7.044       VMACIDMS      IDMS 10.2 cleanup of retained values.       
  7.039       VMAC22        STOPOVER on type 22 for MVS/370.            
  7.038       XMAC7xxx      Circumvention for SAS Error 344.            
  7.036       BUILDPDB      JPURTIME not found in some circumstances.   
  (The following changes were previously reported in MXG Newsletter     
   FOURTEEN and their text is printed therein):                         
  7.035       WEEKBLD       Implementation considerations.              
  7.023       TYPEIDMJ      IDMS Journal correction.                    
  7.022       CMS SAS       CMS DMSSOP3633 Code 04 OPEN error.          
  7.009       VMACVMXA      Concatenated VM/XA Monitor files            
  7.008       BUILDPDB      JES2 Spool Transfer Purge Records           
  7.007       MONTHBLD      Syntax error                                
  7.006       BUILDPDB      PRENTIME inadvertently in DROP list         
    "         WEEKBLD       PRENTIME not in BY list                     
  7.005       VMAC102       DB2 Trace 102 STOPOVER on subtype 58 or 87  
  7.004       VMAC28        NPM type 28 "Unexpected subtype" message    
  7.003       JCLHSM        HSM included nonexistent XMXGHSM member     
  7.002       VMACCIMS      IMF CPUTM calculation incomplete            
  7.001       TRND...,etc.  Trend Data Base finger faults               
  5. Enhancements still under development for MXG Version 7.            
The following New (new products) and Change (corrections)   
 are anticipated to be completed in the Production Version 7 of MXG.    
New   MXG Version 7 will support SAS Version 6.06 (now in beta,
Unknown        expected to be available in the first half of 1990). The 
XXX xx, 1989   MXG Newsletter accompanying Version 7 will discuss.      
New   DB2 Version 2 Release 2 (DB2 2.2) is now available, but  
VMACDB2        documentation has not yet arrived to permit detail look  
VMAC102        at the IBM changes. It is known that new IFCIDs and new  
XXX xx, 1989   fields in existing IFCIDs are created in DB2 2.2.        
New   Landmark's TMON/MVS development is not yet started, but  
unnamed        documentation and test data has been received from the   
XXX xx, 1989   vendor.                                                  
New   EPILOG/MVS product records will be supported in the      
Unnamed        Production Version.                                      
XXX xx, 1989                                                            
New   HSM SMF record is partially decoded (Change 7.096) but   
VMACHSM        not all segments have been decoded yet.                  
XXX xx, 1989                                                            
New   RMF Type 79 records subtypes 1 and 2 processing is not   
VMAC79         completed and will not execute successfully yet.         
XXX xx, 1989                                                            
   Thanks to Tom Skasa, GE Medical Systems.                             
   Thanks to Sterling Green, GE Capital.                                
New   In a JES3 environment, the MXGTMNT Tape Mount Monitor    
Unnamed        does not create a record for the mounts issued by JES3   
XXX xx, 1989   Main Device Setup, MDS. Those mounts are recorded by JES3
               in TYPE25 observations.  Further research in TYPE25 shows
               there is only one TYPE25 record per job, and it contains 
               the number of mounts actually issued by JES3, but only a 
               single mount duration, from MNTTIME to VERFTIME of the   
               final mount completion.  This new member will combine the
               TYPETMNT (MVS Issued tape mounts) with TYPE25 extracts to
               create one observation for every actual tape mount. For  
               multiple mounts by JES3, the second and subsequent mount 
               observation will only count that a mount occurred at the 
               MNTTIME; the duration and end of mount timestamps will be
               set missing (since you have no idea of how long it really
               took). It also deletes TYPE25 observations with TAPEMNTS 
               zero (happens when UNIT=,,,DEFER is specified, no real   
               mount occurs, but a type 25 is written by JES3). This    
               program does not yet exits (its in telephone alpha!) and 
               this change will be updated when it exists.              
   Thanks to Mark Hutchinson, Atlantic States Bankcard.                 
Change   %INCLUDE SOURCLIB(IMACACCT) logic needs to be added to   
VMAC26J3       control JES3-only type 26 ACCOUNT1 variable's length.    
XXX xx, 1989                                                            
   Thanks to Dick Clarke, British Airways, Heathrow.                    
Change   %INCLUDE SOURCLIB(IMACACCT) logic needs to be added to   
VMACIDMS       IDMS TASBFLDn accounting variables (to be renamed to     
XXX xx, 1989   ACCOUNT1-ACCOUNTn, with lengths defined in IMACACCT, as  
               for all other MVS accounting fields.                     
   Thanks to Dick Clarke, British Airways, Heathrow.                    
Change   IBM incorrectly stores values in PWCOUNT (hex values are 
TYPEVM         F0F8 F0F9 F0C1 or EBCDIC 08 09 0A for actual data counts 
XXX xx, 1989   of 8, 9, and 10!)  MXG will circumvent IBMs error.       
Chnage   RMDS records have reported counts in errors for some of  
TYPERMDS       the page count variables, but have not completed the     
XXX xx, 1989   investigation.                                           
Change   EPILOG CICS CL1000 Versions 430 and 440 support is still 
TYPEEPIL       incomplete (Change 7.094). Test data and documentation   
XXACEPIL       have been received from the vendor. One site has reported
XXX xx, 1989   that the offset needed to be set to -4 to process the    
               Version 430 data, but I have not yet even validated that 
Change   VM/370 USER Class data can have erratic large negative   
VMACVMON       values because one of the two-byte IBM counters overflow 
XXX xx, 1989   (typically DRPCANUS) which causes MXG to falsely think a 
               LOGON event occurred. (Because IBM does not flag LOGON   
               event in the USER data, MXG must attempt to infer that it
               happened.)  MXG will be revised to use only four-byte IBM
               counters in its LOGON event detection (inference engine?)
 Other products which may be added to MXG in the future that have been  
  requested by users include these candidates:                          
   Likely to be in MXG Version 7:                                       
      MVS/ESA 3.1.3 (hopefully to be in MXG Version 7)                  
      MXGMENU Selection Macro Variable definitions.                     
   When product becomes generally available:                            
      CICS 3.1                                                          
      IMS 3.1                                                           
      New Devices                                                       
   After SAS Version 6.06 is generally available on the Mainframe:      
      MXGMENU Menu Architecture (needs SCL, Screen Control Language)    
   Not likely in MXG Version 7:                                         
      JES3 Jes Measurement Facility SMF type 84 record.                 
      VM/XA VMPRF product reports may be replicated in MXG.             
      NETVIEW File Transfer Program Release 1.0 "User" SMF Record.      
      FACOM PDL record processing.                                      
      TPNS activity log.                                                
      IMS Fastpath                                                      
II.  TECHNICAL NOTES                                                    
    a. CPU Time measurements have changed with MVS/ESA as this schematic
       (aligned vertically, but not scaled horizontally) drawing shows  
       exactly what is captured where:                                  
TYPE 70 (RMF-captured CPU measurement of real or logical CPUs):         
        Elapsed Interval (Duration multiplied by number of CPUs)        
     ______________________________________________________ ---------   
                          CPU                                  CPU      
                        Executing                            Waiting    
     ________ ________ _________ ________ ________ ________             
      CPU 1    CPU 2    CPU 3     CPU 4    CPU 5    CPU 6               
       Busy     Busy     Busy      Busy     Busy     Busy               
     Total Hardware CPU Busy Time (from Type 70 "non-wait")             
TYPE 72 (summing only the control performance group's data) contains:   
     _____ _____ ------------------------------------------             
      RMF   RMF                  Uncaptured                             
      TCB   SRB                  RMF CPU Busy                           
      CPU   CPU                  pre-ESA 3.1.3                          
     ___________ _____________________________ ------------             
        CPUTM       Reportedly to be            Uncaptured              
      pre 3.1.3     captured in ESA 3.1.3        RMF CPU                
                                                with 3.1.3              
TYPE 30 (if all work started and ended in the interval) contains:       
     _____ _____ _____ _____ _____ _____ _____ ------------             
      SMF   SMF   SMF   SMF   SMF   SMF   SMF   Uncaptured              
      TCB   SRB   TCB   SRB   IIP   RCT   HPT      SMF                  
      CPU   CPU   CPI   CPI   CPU   CPU   CPU      CPU                  
      CPUTM = Sum of all CPU times in TYPE30                            
       Three new CPU time durations (in addition to the existing four)  
       have been added to the SMF type 30 record, which is the source of
       job accounting (and PDB.JOBS, STEPS, SMFINTRV, TYPE30_V, etc.):  
       CPURCTTM - Region Control Task (Quiesce/Restore/Swap), caused    
                   by this TSO user's swapping.                         
       CPUIIPTM - I/O Interrupt Processing (Second Level Interrupt      
                   Handler, SLIH)                                       
       and one brand new CPU measurement:                               
       CPUHPTTM - Hiperspace processing CPU time. HPT CPU time is NOT in
                  captured in existing CPU TCB or SRB fields. When SORT,
                  for example, begins using Hiperspace processing, the  
                  TCB CPU time will be significantly reduced, because   
                  function was moved from TCB to the new HPT facility.  
                  DFSORT benchmarks showed examples of TCB reduced from 
                  100 seconds, but that reduction required 25 seconds of
                  HPT time, so the true reduction was from 100 to 75. If
                  your billing system is using the CPUTM variable in MXG
                  (which has always contained the absolute total of ALL 
                  CPU times found in the type 30 record, even though the
                  original 1984 book said it was only CPUTCBTM+CPUSRBTM)
                  you were wise and will have no immediate CPU billing  
                  problems, but if you charge only for CPUTCBTM, you are
                  unwise and extremely vulernable to incorrect billing. 
       Of great concern to capacity planning in the preceding  schematic
       is the absence of these three new CPU measures in the Performance
       Group measures in TYPE72 data.  These CPU measures will be in the
       RMF data for the newly announced MVS/ESA 3.1.3.                  
    b. Additional MVS/ESA measurement data (already in MXG 6.6) included
       TYPE 30: Step Hiperspace/Data Space Activity                     
                   HIPAGINS - Hiper Page Ins                            
                   HIPAGOUT - Hiper Page Outs                           
                   DSSIZHWM - Data Space High Water Mark, Megabytes     
                   CREADMIS - CREADS Missed in Hiperspace               
                   DIVRREAD - Address Space total Reread Count          
       TYPE 72: Performance Group paging/memory measures                
                   PGPAGEIN - Page Ins (from Auxiliary DASD)            
                   ACTFRMTM - Frame-seconds of memory occupancy         
                              while task in this performance group      
                              period were "SRM Resident"; includes      
                              both Central Store and Expanded Store.    
                   HIPPGINS - Hiperspace Page Ins (from Aux DASD)       
                   HIPRDMIS - Hiperspace Read Misses (CREADS in 30s)    
    c. MVS/ESA also creates an RMF record from RMF Monitor III screens, 
       in a new subtype of Type 72 records, but only a small part of    
       the RMF III screen data is written to SMF. MXG's member JCLZRB   
       will process the screen data in the RMF III VSAM file, but the   
       TYPE72MN dataset is still valuable for a quick look, containing: 
         User Statistics                      Frame Statistics          
         AVGUSERS - Logged On                 MONACTV - Active Frames   
         AVGACTV  - Active Users              MONIDLE - Idle Frames     
         MONDUTR  - Users Out and Ready       MONDIV  - DIV Frames      
         MONPAGE  - Users Delayed Paging      MONFIX  - Fixed Frames    
         MONSWAP  - Users Delayed Swap        MONSLOT - Slots Used      
                                              MONPGIN - Page Ins        
  2. MVS SMF Type 30 Subtype 3 TCB/SRB times after 922 ABEND            
 MVS SMF Type 30 Subtype 3 TCB/SRB times after 922 ABEND contain trash, 
 but because first bit is on, the number is seen by IBM as a negative   
 value and by MXG as a very large value (because MXG uses PIB4. instead 
 of IB4. for these fields). Now, IBM zeroes the TCB/SRB fields in the   
 record, with PTF UY24926. Thanks to Diane Epstein, SW Bell.            
  3. MVS SMF Type 26 Time stamps after JES2 Spool Transfer.             
 MVS SMF Type 26 Time stamps are wrong after reload of SYSOUT that was  
 previously offloaded with JES2 Spool Transfer program. APAR OY21221.   
  4. MVS execution under VM, VM/XA, or PR/SM publication.               
 The MVS under VM paper formerly available to your SE under PROFS that  
 was referenced in previous MXG newsletters, is now published in IBM's  
 "Operating MVS/XA in Multi-Image Environments", GG24-3274-01. This pub 
 has a number of additional topics. Must reading for PR/SM users as well
 well as MVS under any form of VM!                                      
  5. MVS SMF Type 30 interval records missing for early address spaces. 
 Type 30 interval records are not created on tasks which are created    
 prior to the completion of SMF initialization. OZ96676 discusses this  
 design "feature". You may miss records on the catalog address space, as
 it usually starts right after SMF, but before SMF has completed its own
 initialization. There appears to be no fix at present, however the PLM 
 for Master Scheduler, LY28-1200-4 page 5-567 discusses why the SMF task
 must WAIT before other address spaces! We've re-opened the problem with
 IBM to see how to relocate the SMF and its WAIT ahead of Catalog ASID. 
  6. MVS SMF Type 26 record corrupted.                                  
 Type 26 JES2 records (ONLY for HJE3311 Rel C11) are invalid beginning  
 in the programmer name field, causing trashed variables in TYPE26. The 
 IBM fix is PTF OY21719.                                                
III. CHANGE LOG                                                         
--------------------------Changes Log---------------------------------  
 You MUST read each Change description below to determine if a Change   
 will impact your installation.  You can then either make the change    
 from the Change Description text, or you may wish to call Merrill at   
 214-351-1966 to request the prerelease of MXG 7.2, which already has   
 all of these changes installed and tested. Member CHANGES of the MXG   
 SOURCLIB will always be more accurate than the printed changes in a    
 Newsletter, and always identifies the MXG Version and creation date.   
 Printed changes may actually be implemented more comprehensively in the
 actual MXG Software. Always read the comments at the beginning of each 
 source member named under the Change Number when you receive a new     
 Version of MXG. Documentation of new datasets and variables, validation
 status, notes, etc., are usually in comments in the source members.    
 If you choose to install printed changes, you could copy the affected  
 member(s) from MXG.SOURCLIB into your existing USERID.SOURCLIB and then
 make these changes therein. Alternatively, you can copy into a new PDS,
 MXG.CHANGLIB, to hold these in-between-version changes, concatenating  
 MXG.CHANGLIB between USERID.SOURCLIB and MXG.SOURCLIB until you install
 the next MXG replacement library. Then when you do install the next    
 version, you must then delete all of the members in the MXG.CHANGLIB   
 (if you used that technique), or you must individually remove only the 
 changed members from your USERID.SOURCLIB.  It is always wisest to     
 document all of your changes in a member CHANGES in the USERID.SOURCLIB
 or MXG.CHANGLIB library.                                               
 Whenever you install changes or test a new version of MXG (or even for 
 your own reports), be extra careful to look on the SAS log for any real
 error conditions. Search for all occurrences of "ERROR:"  "ERROR :"    
 "UNINITIALIZED" "NOT CATLGD", as they usually indicate a serious error.
 PROC PRINT and PROC MEANS of each new MXG-built SAS dataset will go    
 a long way in explaining their contents, and can be examined to detect 
 unusually large, negative, or suspicious values.                       
 If you are not having any problems reported herein (and many of these  
 changes only correct enhancements added in 7.0 or 7.1 or for leading   
 edge software), you can simply wait until the Production Version 7 is  
 automatically sent to your site next February.                         
Completed changes:                                                      
  Change text is in member CHANGES or CHANGEnn, and not kept here to    
  save space. The text was actually printed in the physical newsletter. 
--------Changes thru 7.165 were printed in NEWSLETTER FIFTEEN-----------
--------Changes thru 7.035 were printed in NEWSLETTER FOURTEEN----------
          and are in member CHANGES of MXG Version 7 Library, which     
          becomes member CHANGE07 in the subsequent MXG version.        
End of Changes Log, but how's this for page filler, printed verbatim    
 from an entry in IBM's INFO/SYS (SMF6CPS is COPIES in TYPE6):          
                                        E343725 (HIT LIST 000003/000013)
---------------------------------------------LINES:   1 THRU   15  OF   
H E343725 S=TOOLS C=GY4 D=JUL89 E=JUL91                          L=00015
                           F   -SUGG -OY21704--5665-27-501--IN-INCORR OU
REPORTED RELEASE    R220                                                
ERROR DESCRIPTION:                                                      
SMF6CPS IS NOT UPDATED CORRECTLY.                                       
CHANGE WILL NOT BE IMPLEMENTED.                                         
89/07/19,CHICAGO FS                                                     
                     CME 20/20:  The History of SMF                     
                              Session M710                              
                            August 21, 1989                             
                       H.  W.  Barry Merrill, PhD                       
                          Merrill Consultants                           
                             Dallas, Texas                              
                         SHARE Installation CM4                         
SMF  became available 20 years ago with OS/360 Release 18.  SHARE's 1964
SORC report was part of the input to  IBM's  1966  SMF  design  document
(authored  by  an  IBMer who had been a SHARE board member).  The design
objective was two-fold:  the stated SHARE requirements for  control  and
evaluation  of  the  installation,  and  the  unstated  need  for IBM to
understand how its OS and its program products were  being  used  (which
and  how  much).   That  1966  design  document,  when compared with the
current SMF implementation, will be shown to be remarkably  accurate  to
the  needs  of users even today!  The presentation will also discuss the
never-announced  and  never-released  IEHMAN   report   package!    This
presentation is based on recently de-classified IBM design documents and
interviews with  the  original  designers  and  developers  of  the  SMF
         A. Jun 15, 1964 SORC Report             46-47                  
         B. 1964-1967 Brockish Interview Notes   48-49                  
         C. Aug 25, 1967 Task Force Report          50                  
         D. Nov  1, 1967 IBM Memo                   51                  
         E. Nov  1, 1966 Objectives              52-57                  
         F. Mar 13, 1967 Addendum I Subset 2     58-59                  
         G. Mar 14, 1967 Addendum II                60                  
         H. Jul 27, 1967 Proposal Memo              60                  
         I. 1967-1969 Schiffman Interview Notes  61-62                  
         J. Oct 16, 1968 Subset I Final             62                  
         K. Jan 31, 1969 Subset II Final         63-66                  
         L. Jun 25, 1969 Subset II Final Final      67                  
         M. IEHMAN                                  68                  
         N. Jul, 1969 Release 18                    69                  
         O. Oct, 1969 Release 18.6                  70                  
         P. Jun, 1970 Release 19                    71                  
         Q. Feb, 1971 Release 20                    72                  
         R. Aug, 1972 Release 21.6                  72                  
         S. 1972-1974 Merrill Notes              73-74                  
         T. 1975-1977 Hankison Interview            74                  
         U. Aug  1, 1977 Objectives                 75                  
         V. Author's summary                        76                  
         W. Contributors to this paper              76                  
Copyright  (C) 1989 Merrill Consultants, Dallas, Texas.  This paper will
be published in a 1989 issue of the "Technical Newsletter for  Users  of
MXG".   Permission  is  hereby  granted  to SHARE, Inc.  to publish this
presentation in SHARE  Proceedings.   Merrill  Consultants  retains  its
right to distribute copies of this presentation to whomever it chooses. 
On April 7, 1964, IBM Announced OS/360. Were you computer literate then?
The  IBM  Design  of SMF Was The Direct Result Of The 1964 SHARE Systems
Objectives and Requirements  Committee  "SORC".   The  SORC  Report  was
produced only two months after OS/360 announcement!                     
                               APPENDIX F                               
                        Report of SHARE Systems                         
                 Objectives and Requirements Committee                  
                             June 15, 1964                              
                    G.E. Bryan, Chairman      L.   Cohn                 
                    P.A. Cramer               R.   Gillespie            
                    G.H. Mealy                C.H. Weisert              
The    advent    of   multi-programmed   and   multi-processor   machine
configurations  has  re-emphasized  the  always  present  need  for  job
accounting  and  made  even  more  important  the much neglected area of
machine and program  performance  measurement.   Operating  systems  for
System/360  must  provide  as a standard feature a job accounting system
which retains records  useful  for  both  ordinary  cost  allocation  of
machine   component  time  and  measurement  of  hardware  and  software
Accounting and statistical information should be carried in the  system 
on  a  job basis and identified by the following information supplied by
the submitter of the job:                                               
1. A job number.                                                        
2. A programmer identification number.                                  
3. An identification specific to the run.                               
4. Priority.                                                            
5. Other information as deemed necessary by                             
   the individual installation.                                         
In addition, in order to facilitate automatic  scheduling  of  jobs  for
optimum  performance, the following should be supplied either by the job
submitter or, for "normal cases," be defined  by  installation  standard
6. Expected execution time, cutoff time.                                
7. Expected output volume (lines, records, cards) by data set, with an  
   installation standard limit provided in the absence  of  specifically
   supplied data.                                                       
An  installation standard factor should be applied to these numbers for 
the definition of cut-off limits.  The programmer  should  have  dynamic
access  to the limits for examination and modification during execution.
In certain installations there is a need to specify 'due'  time  --  the
time  before which the job should be completed -- and 'drop' time -- the
time at which the job should be deleted if its execution  has  not  been
"The system should  normally  add  to  the  job  accounting  record  the
following information:                                                  
8. Date, and beginning clock time of the run.                           
9. Processor identification.                                            
It  should  also  add to the accounting record information for each task
performed in the job's execution:                                       
10. Task Type (FORTRAN,COBOL,SYSTEM,EXECUTION,MESSAGE).                 
11. Mainframe time.                                                     
12. Equipment used (tapes,drums,data lines)                             
13. Output volumes (print lines, cards)                                 
14. Core used for code.                                                 
15. Core used for data.                                                 
16. Operator intervention time (waiting for operator action).           
At job completion time, all of the accumulated  information  should  be 
summarized  and  totaled  for the user, should be added to a daily total
record, and should be output in some permanent form for later accounting
and statistical analysis.                                               
Accounting   for  message  switching  should  be  done  using  the  same
accounting mechanism with each message transmission treated as a  single
task.   The  descriptive parameters in 1-5 are supplied by the system on
the basis of the called and calling parties.  The task type (item 10) is
Flexibility  should be included for the expansion of the kind and number
of  statistics  retained  at  installation  or  IBM   option.    Special
installation  requirements  and  special  IBM  performance  measurements
should be readily accomplised  by  addition  of  code  and/or  parameter
In June, 1966, Bob Brockish joined IBM.                                 
Bob  had  been on the SHARE Executive Board 1963-64, and had taken Share
Systems Division job when George Mealy resigned to join IBM.   Had  been
at  Martin-Marietta  (1959)  and was now data center manager at Thiokol,
In 1966, OS/360 SYS1.ACCT accounting consisted of only an IEFACTRT  Exit
which was taken only at Step Initiation (=8), Step Termination (=12), or
Job Termination (=16).                                                  
When the exit was taken, there were pointers provided to:               
     Job Name              Step Name                                    
     Programmer Name       Account Fields                               
     Job Running Time      Step Running Time                            
     Step Number           Cancel Bit                                   
but the user would then have to format and write the data  to  SYS1.ACCT
using ASM code in IEFWAD.  SYS1.ACCT was supported under OS/360:        
     PCP - 18K, 44K, and 100K Scheduler                                 
     MFT - 30K and 44K Scheduler                                        
     MVT - Scheduler                                                    
Clearly, this approach to accounting was inadequate.                    
Notes from my 1989 visit with Bob Brockish:                             
IBM'S MOTIVATION FOR DESIGNING SMF:                                     
1. User need to account time and resource usage.                        
2. IBM's  need to know about how the system was being used, especially  
   about which IBM Programs were used how much/often.                   
A "SYSOUT Project" had already been started inside IBM.  Originally  the
idea  was  to  solicit  customers  to submit their SYSOUT on tape, which
would then be analyzed after  the  fact  (for  compiler  messages,  link
editor, etc.) to count program usage!                                   
Ken  Brownell  was  the  manager  with  one other when Bob joined group.
Bob's task was to look at OS to see how data could be collected  in  the
operating  system.   IBM  Programming  Systems  Division at Pok had just
formalized "Programming Objectives"  in  January.   SMF  was  the  first
project to comply!  Bob began to develop SMF objectives from SORC.      
The  name,  System  Management  Facility,  was picked in a brainstorming
session July 6, 1966.                                                   
PROPOSED PHILOSOPHY:                                                    
IBM would provide a way to collect data, the customer must  process  and
report.   IBM  could  not  see  a general way to report on the collected
data.  Based on the SORC Report, Bob began to  define  the  data  to  be
collected.   The  initial  design  was  reviewed with the non-disclosure
signers at the Aug, 1966 SHARE in Toronto (but Bob could not  go  -  Ken
Brownell wanted to see Toronto!)                                        
People at that August, 1966 meeting:                                    
 Lora Steele, IBM SDD Share Liason                                      
 Arnold Smith, IBM Overall Liason to SHARE.                             
 Omar Smith, Rocketdyne?                                                
 Phil Kramer, SDC                            Bill Thunderbirkk, ?       
 Roy Dickson, Okla,  Phillips Petroleum      Harvey Siegel, ?           
 Joseph E. Kelly, SOCONY Mobile              Irv Lester, North American 
 John Noerr, Sinclair Oil                                               
Follow up copy of the objectives was sent to:                           
   Channing Jackson (SHARE)                                             
   Earl Althoff, VP Guide                                               
   Leroy J. Rodgers, Chair Guide COBOL project                          
IBM'ers also involved along the way:                                    
 Neil Lewis - primary planner at Pok       Don Ludlow - SUPERZAP author 
 Harry Reinstein                           Ira Heiney                   
 Hank Coon                                 Bob Levy                     
SMF ultimately hit 160 OS modules!                                      
The  design  had  planned  for  "packages"  of  information, selectable,
perhaps incrementally delivered  by  IBM.   As  there  was  concern  for
potential  impact of SMF on the system, packages must allow installation
to specify only what it needed.  The first package  proposed,  presented
at   SHARE  non-disclosure  meeting  in  August,  1966,  was  the  Basic
Accounting information (Start/Stop Times).  This first package was  then
reviewed  at  Feb  1967  non-disclosure  meeting,  and Bob then gave the
presentation of the revised package.     End of interview notes.        
Throughout  much  of  1967,  a  joint  SHARE/GUIDE   System   Management
Facilities  Study Group met under non-disclosure.  At the Monday May 22,
1967, meeting in New York City at the Americana Hotel included:         
   Edward Boback (GUIDE) Logistics Research                             
   T. Todd Brown (GUIDE) Iowa State University                          
   Joseph C. Kelly (SHARE) Mobil Oil Corp.                              
   Edward G. Laski (GUIDE) Aetna Life                                   
   John M. Noerr (SHARE) Sinclair Oil                                   
   David R. Paul (GUIDE) Iowa State University                          
   Harvey Sigal (SHARE) Union Carbide                                   
IBM:  Paul Bouche, Poughkeepsie                                         
      Ken Brownnell, Boulder                                            
      Arnold P. Smith, White Plains                                     
      Don Miles                                                         
               This task force reported their conclusions               
                   in the August 25, 1967 SHARE SSD:                    
"Attached is a report for SSD describing user requirements  for  systems
management   facilities   under   OS/360.    This  report  contains  the
information that was gathered by the Systems Management Facilities  Task
Force  in conjunction with GUIDE and IBM.  The Task Force is now working
on further definition and input to help provide guidance to  IBM  as  to
the  OS/360  users  requirements  for  System  Management.   Among other
things, the group would like input from users on  what  kind  of  system
management  facilities  they  are currently supporting and what level of
degradation (core, time, DASD, space, devices) they consider  acceptable
for  each  functional  feature  required  in advanced system management.
This input should be sent to both:                                      
    C. Channing Jackson (Task Force)                                    
    R. Cleveland (IBM Poughkeepsie)                                     
User members of the task force included                                 
    J.M. Noerr (SHARE, signer of report)                                
    E.G. Laski (GUIDE)                                                  
    C. Weisert (SHARE Sys. Division)                                    
    J.J. Jones (OS/360 Proj.)                                           
    J. Kelly (MCB)                                                      
A final comment on the task  force  itself.   IBM  has  appeared  quite 
interested in this work and has done considerable leg work for us and we
feel  that  these  people  deserve  recognition  and  thanks  for  their
exceptional  devotion  to  getting  a  job done despite the task force's
sometimes apathetic reaction to their questions.  These people are  (all
   Paul Bouche                                                          
   Bob Brockish                                                         
   Ken Brownell                                                         
   Don Miles                                                            
   and others to a lesser degree."                                      
              An IBM memo from Lora Steele, IBM User Group              
             Liason Poughkeepsie November 1, 1967 requests              
             IBM to make a commitment to announce SMF, and              
             details the history of user group involvement:             
"There has been long and consistent pressure from user groups for IBM to
provide System Management Facilities.  It has become a ritual for  SHARE
to  request  an  IBM  presentation  on this topic every general meeting.
Although IBM has been unable to  comply  to  date,  we  should  plan  to
provide  such  a  presentation  for the SHARE and GUIDE general meetings
following IBM announcement.                                             
The first disclosure of IBM Confidential Information relating to SMF was
made  to  user  group  officials  in  August  1966  and  consisted  of a
preliminary version of IBM's internal objectives.  These objectives were
a  result  of  considerable  effort  by  Messrs.   Ken  Brownell and Bob
Brockish  of  Boulder  to  obtain  and  organize  the  many  and  varied
requirements of user group members.                                     
Mr.   Brownell  was assigned SDD responsibility for these objectives by 
the OS Architecture and Planning management.  Messrs.   Don  Warren  and
Ward  Lambert  of  DP were present at the first IBM Confidential meeting
held in Toronto on August 2, 1966.  A revised version of the  objectives
was mailed to user group officials on January 6, 1967.                  
In  February  and  March,  1967  there  was  little activity.  In April,
however, a new joint SHARE/GUIDE committee was formed  to  again  review
the   user  group  requirements.   IBM's  internal  specifications  were
disclosed to the committee in order to verify that their  specifications
did  in  fact meet stated user requirements.  On September 15, 1967, the
User Group Nondisclosure Agreements for committee members expired....   
If  IBM  could  make  any  kind  of  commitment  for  Systems  Mangement
Facilities  in  the near future, the user groups would be mollified.  If
dates and technical details are included in the announcement, they would
be  pleased.   If  no technical details are provided, I suspect that the
committee will be revitalized and  members  will  insist  that  they  be
allowed  to  monitor IBM's implementation phase to make certain that the
considerable user group efforts have not been wasted."                  
IBM's earliest SMF objectives were specified in:                        
               From S/360-OP-001-01-Pok dtd Nov 4, 1966:                
                         Programming Objectives                         
                             IBM System/360                             
                 System Management Facilities in OS/360                 
                        Dated:  November 1, 1966                        
"1.1 Purpose                                                            
1.1.2 The  operating  information  has  two  primary  purposes.   First,
      information  is required to determine and equitably distribute the
      costs  associated  with  each  program's  use  of  the  equipment.
      Second,  the  installation manager requires certain information to
      evaluate the performance of his installation both  in  regards  to
      his  own  standards  as  well  as in comparison with other similar
      computer operations in order to increase the effficiency of use of
      his  current  equipment,  improve  the  service  provided  by  his
      installation and plan future equipment,  programming  systems  and
      personnel requirements.                                           
1.1.3 The  dynamic  control  capability is required by computer center  
      management to monitor the work load of the  System/360  as  it  is
      being  processed  to  verify  that  work  to  be  accomplished  is
      appropriately submitted with  proper  identifying  data  and  that
      processing is carried out in accordance with accepted installation
1.1.5 For purposes of these objectives  the  term  "user"  refers  to  a
      computer installation manager rather than to individuals using the
      computer.  The latter are referred to as problem programs.        
1.2   Scope                                                             
1.2.1 The  scope  of  a  Systems  Management  Facility encompasses four 
      categories of support.                                            
       a. Systems Management Data Set                                   
       b. Data Collection Packages                                      
       c. Dynamic Control Entries                                       
       d. System Management Utilities Programs                          
1.2.2 The System Management Data Set is a permanently opened data set   
      specifically  designed  for  the  recording  of Systems Management
1.2.3 Data Collection Packages include the gathering and retention of   
      data elements required for control, evaluation and cost allocation
      of system usage.  Items such as CPU time, storage utilization, I/O
      device allocation and software components used are  indicative  of
      the type of data required.                                        
1.2.4 Dynamic Control Entries allow the user timely assurance of        
      efficient  systems  utilization  by transferring control to a user
      routine at appropriate times and places.  These  Entries  transfer
      control  to  user supplied coding which performs specific auditing
      functions unique to each user."                                   
"3.1.1 The required data elements are stratified into five packages as  
       1. Basic Job Data                                                
       2. Basic Step Data                                               
       3. Additional Job and Step Data                                  
       4. Periodic Job and Step Data                                    
       5. Sampled System Data                                           
3.1.2 Package 1. Basic Job Data.                                        
       1. Job Log Number                                                
       2. //JOB Statement after validation                              
       3. Entry Time of Day                                             
       4. Exit Time of Day                                              
       5. Effective Priority                                            
       6. Output Quantities - Sysout Lines/cards                        
       7. Job Time                                                      
       8. Job Termination Status                                        
      Two messages to be added to SYSOUT:                               
        Sign on                                                         
        Sign off                                                        
3.1.3 Package 2.  Basic Step Data.                                      
       1. //EXEC Statement after validation                             
       2. Step Time                                                     
       3. Unit Addresses by Type of Device                              
       4. Output Quantities (Lines/Cards)                               
       5. Input Quantity (Card images only)                             
       6. Step Log Number                                               
       7. Step Termination Status                                       
      One message to be added to SYSOUT:                                
       Step Termination, Program, Time, etc.                            
3.1.4 Package 3. Additional Job and Step Data                           
      Additional  data elements for use in performing more sophisticated
      costing and in managing data libraries.                           
       1. Job Log Number                                                
       2. SYSIN Reader Identification                                   
       3. Reader/Intrepreter Time                                       
       4. SYSOUT Writer Class                                           
       5. Output/Writer Time                                            
      Additional per step data                                          
       6. Kept Data Set Identification                                  
       7. Deleted Data Set Identification                               
       8. Created Private Data Volume Id (Released Private Data Volume  
          ID may be supported via Data Set Accounting)                  
       9. Actual Maximum Core Used (Not the Region Parameter)."         
"3.1.5 Package 4. Periodic Job and Step Data                            
      Includes  those  additional   items   which   are   required   for
      configuration planning, software evaluation and system performance
       1. Job Input Queue Time                                          
       2. Job Output Queue Time                                         
      Additional per step data                                          
       3. Step Start Time                                               
       4. Program Time                                                  
       5. Data Set Descriptions                                         
       6. DD Statements                                                 
       7. Number of Devices Used by type/step                           
       8. Device Use Time by Type                                       
       9. Rolled Out Time                                               
      10. Storage Rolled Out                                            
      11. Number of Roll Outs.                                          
      Controlled by Operator Command.                                   
3.1.6 Package 5. Sampled System Data                                    
      Provides the elements necessary to develop a  statistical  profile
      of an installation's system use characteristics.  After the option
      is exercises at system generation time this set of  data  elements
      is  collected  on  a  time  based  sampling  interval  (delta  t).
      Whenever delta t  is  satisfied  the  data  will  be  recorded  on
       1. Time of day                                                   
       2. Total Memory in Use                                           
       3. Number of Active Jobs                                         
       4. Number of Passive Jobs                                        
       5. Number of Devices in Use by type/chan                         
       6. Number of Readers in Use.                                     
       7. Systems Work space in Use                                     
       8. Work Input Queue Lengths                                      
       9. Work Output Queue Lengths                                     
      10. Channel Queue Lengths                                         
      11. Seek Queue Lengths                                            
      12. Number of Active Tasks                                        
      13. Timer Queue Length                                            
      14. Number of Writers in Use"                                     
"3.2 Systems Management Data Set                                        
3.2.4 Systems Management Data will be recorded sequentially on SYS1.MAN 
      as  the  data is acquired.  This method of collecting will require
      the following considerations.                                     
      1.  SYS1.MAN data may have to be sorted by  Log  Number  prior  to
          input to analysis programs.                                   
      2.  Each dumping of SYS1.MAN will be an incomplete segment of the 
          Systems Management Data Set.                                  
      3.  A complete Systems Management Data Set contains  all  segments
          collected  between  an  IPL  and the subsequent "drying up" of
          the system.                                                   
      4.  In the event of an abnormal  system  termination  contents  of
          SYS1.MAN shall be preserved. "                                
3.4 Dynamic Control Provisions.                                         
3.4.1 Dynamic Control facilitiesj shall be provided in the form of      
      entries to a user routine taken at ....                           
3.4.2 JOB, STEP, and DD CONTROL CARD VALIDATION ENTRY ....              
3.4.3 JOB INITIATION ENTRY ....                                         
3.4.4 STEP INITIATION ENTRY ....                                        
3.4.5 STEP TERMINATION ENTRY ....                                       
3.4.6 JOB TERMINATION ENTRY ....                                        
3.4.7 TIME LIMIT ENTRY ....                                             
3.4.8 OUTPUT LIMIT ENTRY .... "                                         
"3.6 Systems Management Utility Programs                                
Two  Types  of  utility  programs  required  in conjunction with Systems
Management Facilities are analysis programs and a list program.         
3.6.1 Systems Management Analysis Programs                              
      The purpose of these utility programs is to produce reports  based
      upon  data contained in the SYS1.MAN data set.  These reports will
      provide management with the information  necessary  to  understand
      system usage and plan future improvements.                        
      The  analysis  programs  will yield information that describes the
      workload and system utilization from several points of view.      
      One class of information should  pertain  to  the  overall  system
      usage  and status independent of particular programs or job types.
      Such information constitutes a "System Profile".                  
      The "Job Stream Profile" on the other hand should describe  system
      utilization   in   terms  of  the  characteristics  of  the  input
      A further classification of  information  should  provide  a  "Job
      Profile"  devoted to revealing the nature of both the workload and
      systems utilization in terms of the characteristics of  individual
      job types.                                                        
      At  the lowest level of detail, a "Job Step Profile" describes the
      charateristics of individual job steps and their  contribution  to
      systems resources utilization.                                    
3.6.2 System Management List Programs                                   
      In  addition  to the programs to analyse System Management Data, a
      utility  program  for  listing  System  Management  Data  Sets  is
      required.   The utility's function is to provide a printout of raw
      Systems Management Data in either its natural order or in Job/Step
      Log  Number  sequence.  Operating under OS/360 this utility uses a
      input either a disk or tape dump of a SYS1.MAN data set.  It shall
      have  the  ability  to concatenate multiple SYS1.MAN data sets and
      print out a composite listing.   Information  from  the  data  set
      labels will be printed at the beginning of the listing."          
"4. Performance Objectives                                              
4.1 The ultimate purpose for Systems Management Facilities is to supply 
      information  from  which  installation  management  can  know  and
      improve  computing  systems  performance.   Through   the   proper
      utilization  of  good  systems management data the user can reduce
      operating costs.  On this basis users  are  willing  to  sacrifice
      some  amount  of  performance for the ability to gather this data.
      The system performance  degradation  however,  cannot  exceed  the
      reasonable  value  of  the  facilities.   To guide in implementing
      System Management Facilities in OS/360 the  following  performance
      objectives are set forth.                                         
4.2 There is to be no performance degredation when none of the Systems  
      Management Facilities are employed."                              
4.3 The performance of the operating system should not be degraded more 
      than  3%  when the SYS1.MAN data set, Dynamic Control Entries, and
      Packages 1, 2, and 3 are activated.  In addition,  the  collection
      of  data in Package 4 should not decrease performance more than 4%
      during those periods when it is active.  Package 4 should cause no
      degradation  when it has been selected at system generation but is
      not in use.  Since the collection of Package 5  is  based  on  the
      value  of  a time interval.  measurement of its performance should
      be aimed at a time per sample basis.  Each occurrence of  sampling
      of  Package  5  data  should  not require more than 500ms on a 360
      Model 50."                                                        
4.4 These performance objectives for OS/360 are summarized below:       
    Facility Enabled              Timing Target          Additional core
                                                         Maximum Bytes  
    none                              0                      0          
    SYS1.MAN +Dynamic Control     200ms/Job                256 + 128/Job
    Package 1                     500ms/Job               1024          
    Package 2                     250ms/Step               512          
    Package 3                     250ms/Job                512          
                                 +250ms/Data Set                        
    Package 4                     200ms/Job               1024          
                                 +500ms/Data Set                        
    Package 5                     500ms/Sample            1024          
      Based on jobs averaging 3.5 min and containing an average of 3 job
      steps each having 5 data sets.                                    
      Based  on  CPU  Model  50H with 2311 disk for system, SYS1.MAN and
      SYSOUT residence."                                                
               Four months later, SMF Subset 2 was specified:           
                       Externally dated April 18, 1967                  
                                  Addendum I                            
                            Programming Objectives                      
                           IBM Operating System/360                     
                  Data Set Management & DA Space Accounting             
                      Dated Internally:  March 13, 1967                 
"This addendum extends the scope of the System Management Facilities  to
supply the user with the tools required to manage data set libraries and
to control and account for space usage on direct  access  devices.   The
facilities  to furnish this capability include the recording of data set
records on SYS1.MAN and a utility  routine  to  retrieve  the  data  set
records,  put  them  in  a  data set management file, produce a data set
inventory and maintain the data set management file.                    
New data will be recorded when Package 3 is selected:"                  
A. NEW, Non-temporary Data Sets                                         
   1. Identify as NEW                                                   
   2. Data Set Name (including GOOOVOO)                                 
   3. Creation Date                                                     
   4. Expiration Date                                                   
   5. Device Type                                                       
   6. Number of Volumes                                                 
   7. Volume Serial Numbers                                             
   8. Public or Private                                                 
   9. Shared or Exclusive                                               
  10. Number of accesses to read                                        
  11. Number of accesses to write                                       
     for direct access              for tape                            
  12. File organization       Data set sequence Nr                      
  13. Amount of space         Type of labels                            
  14. Unit of Space           Recording density                         
  15. Number of extents       7 or 9 track recording                    
B. OLD Data sets - similar to NEW                                       
C. Temporary Data Sets                                                  
   1. Identify as Temporary                                             
   2. Device Type                                                       
   3. Number of temporary data sets                                     
   4. Total number of volumes involved                                  
   5. Total number of accesses/records.                                 
   6. Total amount of space in bytes"                                   
                And then, amazingly, Subset 2 described the:            
"IV. Data Set Management Utility                                        
The Data Set Management Utility is intended to run daily or periodically
as  required  by  the  installation  to provide information to guide the
operations staff in purging the data set library  and  to  maintain  the
circulation of data volumes.                                            
A. Functions of the Routine                                             
  1. Extract data set records  from concatenated SYS1.MAN dumps and use 
     them  to  build  and  maintain  the Data Set Management File.  This
     Utility does not physically remove data set records from  the  Data
     Set Management file on this basis.  Instead it "deletes" by setting
     a flag and inserting a deletion data  in  the  appropriate  record.
     Actual  removal  of  data set entries will be made via punched card
     input after the information has been used for billing, etc., by the
  2. Produce a report by device type and volume serial number of data   
     sets whose expiration date has elapsed.  This listing includes Data
     Set Name, Programmer's  Name,  Accounting  Fields,  Creation  Date,
     Expiration Date, Date Last Written and Date Last Read."            
  3. Produce data set inventory report by device type showing activity. 
     This  listing includes Data Set Name, Programmer's Name, Accounting
     Fields, etc....                                                    
  4. Accept, as input, punched cards with deletions and changes to data 
     set records in the Data Set Management File.  These cards  will  be
     prepared  by  the  user's  data  librarian  and input into the file
     maintenance  run.   Through  these  cards,  data  set  entries  can
     actually  be  removed  from  the  file and revisions can be made to
     accounting fields, responsible programmer's name, etc.             
  5. Provide exits for user supplied coding to perform other functions  
     desired by the installation."                                      
                     Addendum II was dated a day later:                 
                           Programming Objectives                       
                          IBM Operating System/360                      
                             Job and Step Timing                        
                              Space Accounting                          
                      Internally Dated:  March 14, 1967                 
"This addendum replaces "Job Time" in the  original  specification  with
"Job CPU Time", and adds a new element:                                 
      7b. Job Unoverlapped I/O Time                                     
Job  Unoverlapped I/O Time is the accumulation of time during which both
of the following conditions are satisfied:                              
  a) input and/or output activities are being carried on by or for a job
  b) and that job is not using the central processing unit.             
Job CPU Time plus Job Unoverlapped Time equals  the  time  a  job  would
require if it was the sole inhabitant of the computer system.  It is the
summation of these two time measurements that ...   should  be  compared
with when determining if the job is exceeding its maximum running time."
               Dated 7/27/67 is a Proposal for Extensions               
                               to OS/360                                
   - Maintenance of the DA Volume Inventory                             
   - A New Volume Reorganization Utility                                
   - A New PDS reorg-needed status message                              
   - A New PDS reorganization utility!                                  
It appears this document was not accepted, but it contains a tantalizing
bit of data:                                                            
"3. Market Requirements                                                 
 3.1  The problem of DA space deterioration potentially affects each of 
      the   estimated  550  installations  using  OS  as  their  primary
      operating system.  Also to be considered are the  additional  1596
      forecasted future users of the system."                           
       550+1596 = 2146 Future OS Customers!!!                           
Initial specifications were completed in Summer, 1967 and SMF  moved  to
Poughkeepsie  (Pok)  for  implementation  under Saul S.  Schiffman, with
Lynn Weisenreider in his group.  Notes from 89 Saul Schiffman interview:
Biggest implementation difficulties:                                    
  to convince Supervisor in IOS to put in lines  of  code  in  Operating
  System - they were very conservative on instructions in path.         
  Testing together to see that it worked was new, OS was the first large
  scale software project.                                               
  Had to firmly make rules how to access SMF.  Some developers  did  not
  want user records - could garbage up the collection, but they lost.   
  MANX/MANY had to be handled by the system                             
  Very hard at the beginning, everybody had their own ideas.            
  "Hundreds" of pieces of information.                                  
  Rule:  Unless you can show me how you will use it, I won't add it.    
  That reduced hundreds to 30-40 items.                                 
  Complaint  of expected SMF overhead of 5% was answered by jury rigging
  SYS1.ACCT for all of the same data.  Discovered that it too  took  5%,
  and showed in-line capture no worse than exits.                       
  This  convinced  IBM that measurement could be an integral part of the
  operating system, and also showed that SMF data was not optional  -  a
  site simply could not run without this data.                          
  Was the customer Measurement or Accounting?                           
   Accountants - look at this data                                      
   Performance Measurement - look at that data                          
   Picked common ground between the two.                                
  Made  attempt to convince users to use only the repeatable fields, and
  make data more repeatable - MFT easier than MVT.                      
  Wanted one project for billing and  for  measurement  and  evaluation.
  When  conflicts  arose,  measurement  &  evaluation  guys  gave  up on
  additional data fields.                                               
  Accounting - some things are outside of control (eg  paging)  -  could
  not  predict  branching  impact,  thus cannot charge for it.  Began to
  look at counting from application.                                    
  IBM could not answer "How do you account?"                            
    "We were not going to write accounting and                          
     billing routines at IBM.                                           
     Instead, we will document and provide data"                        
SMF Implementation began at the start of 1968.  PCP, Fortran, MFT I, MFT
II  development  were  at Kingston, while MVT, IOS, SVS, MVM were all at
Poughkeepsie.  MVM (Multiple Virtual Memories) became MVS.              
SMF Implementation Options now considered:                              
  1. Have a program in the system dynamically sample and adjust the     
     system based on the actual measurements.                           
    - heuristic approach                                                
    - radical (remember, this WAS 1968!)                                
    - major proponent of this option was                                
      Bart Page -"Brain behind the SRM.                                 
                  Mind beyond others.                                   
                  Ahead of his time, but                                
                  Not a good salesman."                                 
  or, what we actually ended up with:                                   
  2. Extension of traditional SMF design                                
       Not much on analysis                                             
       Minimal reporting                                                
  End of 1989 Interview notes.                                          
On October 16, 1968, Saul's group distributed their Final Specifications
on Subset 1:                                                            
                System Management Facilities (Subset 1)                 
              Final Functional Programming Specifications               
Unfortunately,  I  have  not  see a copy of that document.  Based on the
final implementation,  however,  Subset  1  created  the  following  SMF
  Type  0 - IPL Record                                                  
  Type  1 - Wait Time Record                                            
  Type  2 - Dump Header Record                                          
  Type  3 - Dump Trailer Record                                         
  Type  4 - Step Termination Record                                     
  Type  5 - Job Termination Record                                      
  Type  6 - Output Writer Record                                        
  Type  7 - Data Lost                                                   
  Type  8 - I/O Configuration                                           
  Type  9 - VARY ONLINE Record                                          
  Type 10 - Allocation Recovery Record                                  
  Type 11 - VARY OFFLINE Record                                         
  Type 12 - End-of-Day Record                                           
              Earlier, on February 7, 1968 there had been:              
                 Data Set Management and D.  A.  Space                  
                       Accounting (SMF Subset 2)                        
                     Initial Programming Functional                     
which I have not seen either.  However, in January 31, 1969, the "Final"
for Subset 2 was published:                                             
              Final Programming Functional Specifications               
                               IBM OS/360                               
                  Data Set Management and D.A.  Space                   
                         Accounting (Subset 2)                          
                        Dated January 31, 1969.                         
This identified the new SMF records that would be added by (Subset 2):  
" Record Types                                                   
  Type 14 & 15 - Non-Temporary User Data Set                            
  Type 16      - Temporary User Data Set                                
  Type 17 & 18 - Data Set Status SCRATCH/RENAME                         
  Type 19      - Direct Access Volume Dismount                          
  Type 20      - Job Commencement"                                      
This  "Final"  Document  also  described  three  new  specifications  in
significant detail:                                                     
  Direct Access Inventory                                               
  Tape Inventory                                                        
  Resource Management Utility                                           
Highlights from this IBM analysis package description then go on:       
"2.1.5 The Resource Management Utility                                  
The Resource Management Utility collects volume and data set information
from  the  SMF  data set.  This consolidated data is available in report
form.  The main functions of the utility are as follows:                
A.  It uses volume and data set information from the  SMF  data  set  to
create  and  update  records  in  the inventory data sets.  There is one
inventory for direct access resources and one for tape.                 
B.  From volume information in the  inventories  it  produces  a  volume
inventory report and SCRATCH volume reports.                            
C.   From data set information in the inventory it produces a variety of
data set reports.  The data  set  inventory  records  are  selected  and
arranged   according  to  such  parameters  as  data  set  name,  volume
indentification, account number, expiration date and last usage date.   
D.  It creates control cards for IEHPROGM to  SCRATCH  data  sets  which
have expired.                                                           
E.   It  creates  control  cards  for the Resource Management Utility to
REMOVE data set inventory records from the  inventories  for  data  sets
which have been SCRATCHed or DELETEd."                                  
"F.   It accepts control cards to remove volume and data set information
from the inventories.                                                   
G.  It can compress the inventories.                                    
Each inventory is a partitioned data set (PDS).  Each directory entry in
the PDS reflects  a  particular  volume.   The  member  itself  contains
information about data sets on that volume.                             
This was followed  by  twelve  pages  identifying  each  field  in  each
inventory record and its source SMF record (5,14,15,17,18,19, or 20)."  
and then the JCL:                                                       
" Job Control Language and Control Statements Used by the Utility
The Resource Management Utility can be  invoked  by  the  following  job
control language                                                        
//jobname  JOB   positional parameters and                              
//stepname EXEC  PGM=utility name                                       
//SYSPRINT DD    parameters describing                                  
                  SYSOUT device                                         
//SYSDATIN DD    parameters describing direct                           
                  access inventory PDS                                  
//SYSTAINV DD    parameters describing tape                             
                  inventory PDS                                         
//SYSMAN   DD    parameters describing the                              
                  SMF data set                                          
//SYSREPRT DD    parameters describing report                           
//SYSPUNCH DD    parameters describing punch                            
                  data set                                              
//SYSIN    DD    parameters describing control                          
                  statement data set "                                  
"The  control  statement  data set (described on the SYSIN DD statement)
contains one or more of the following operations:                       
  A. UPDATE                                                             
  B. REPORT                                                             
  C. REMOVE                                                             
  D. COMPRESS"                                                          
Then followed twelve pages  detail  the  syntax  and  use  of  the  many
operands for each operation.                                            
The  sixteen  error messages produced by the Resource Managment Utility,
IEH901E through IEH916I are then documented, and give the first clue  as
to  the planned name for the Resource Management Utility program name of
" Resource Management Utility Requirements                       
The utility program is designed to be in  a  planned  overlay  structure
with  a  minimum  of  15K  required for executable code at one time.  In
addition, core storage is required for buffers.   The  number  of  bytes
needed for buffers is device dependent.  Maximum buffer sizes (in bytes)
are shown in the table below:                                           
   Device     Type       Maximum buffer size                            
    2301      Drum              21,000                                  
    2302      Drum               5,400                                  
    2303      Drum               5,300                                  
    2311      Disk               4,000                                  
    2314      Disk               7,900                                  
    2321      Data Cell          2,400"                                 
And even a performance evaluation of the utility:                       
"The direct access inventory has entries for ten volumes, each with nine
extensions, giving a total of 100 members in the directory.  Each member
is 3500 bytes long and contains 15 data set inventory records, giving  a
total of 1500 data set entries.                                         
An UPDATE is estimated to take 9 seconds.                               
A  REPORT VOLID operation, which produces a list of direct access volume
inventory information is estimated to take 1 second.                    
A REPORT DS operation, which produces an unordered list of all data sets
on  the printer, is estimated to take 82 seconds.  Further time would be
required to produce a sorted list.                                      
A prototype of the utility was coded and  tested  in  several  different
versions.   MVTTRACE  was  used  to  obtain timings of the final version
described below.                                                        
SYS1.MAN data set resided on a 9-track 2400 tape drive, model 3, and was
recorded  at  800  bpi.   The  direct access inventory resided on a 2311
device with a member blocksize of 3200 bytes.  The printer  was  a  1403
device  with  120  print  positions.  The CPU was a model 50, the system
MVT, Release 14."                                                       
"The first UPDATE run  required  12.2  minutes.   This  initialized  the
inventory data sets.                                                    
A.   The  input  stream  from  SYS1.MAN  contained the following records
arranged as if coming from an MVT environment:                          
    60 job commencement              (type 20)                          
    60 job termination               (type 5)                           
  1500 old non-temporary             (type 14)                          
  1500 new non-temporary             (type 15)                          
    60 temporary                     (type 16)                          
    60 SCRATCH                       (type 17)                          
    20 RENAME                        (type 18)                          
    80 direct access volume dismount (type 19)                          
The resulting direct access inventory contained information  about  1862
data  sets,  of  which  39 had been SCRATCHed.  There were 197 directory
names in the directory, of which 148  were  base  members  and  49  were
extension members.  The average number of data sets per volume was 12.6.
The average length of data set inventory records was 208 bytes.         
B.  The second UPDATE required 19.7 minutes."                           
In a separate section of the Final Specification the error  expectations
of SMF were predicted:                                                  
" APAR records for IEBPRTPCH, a utility program estimated to be  
         of a  similar size (13,000 bytes) were used to project expected
         errors and severity.                                           
                                                                All Severity 1 errors should be resolved before integration.    
        Only two OS utility  programs  have  ever  received  Severity  1
        APAR's after release.                                           
 and the accompanying table showed                                      
                         After Customer Ship                            
 Expected:                6 months   18 months                          
 Severity 2, 3, 4 errors     7          15                              
 Man hours to correct       40          40                              
 Machine hours to correct    5           5"                             
                     And then along came the Final "Final":             
                  Final Programming Functional Specifications           
                            IBM Operating System/360                    
                      Data Set Management and D.A.  Space               
                             Accounting (Subset 2)                      
                        Internally Dated June 25, 1969.                 
This  document  was much thinner than its predecessor, and states on its
cover letter:                                                           
"This revision of the referenced specification contains modifications in
the following major areas.                                              
1. The Resource Management Utility program description has been deleted.
          THUS DIED SMF Reporting.                                      
This  final  specification  also  eliminated the type 16 (temporary data
set) by redesigning the type 14 and 15 records to what we now have.     
In addition, design specifications that have departed from  the  initial
specifications were identified:                                         
"1.3.2  From  Initial  Specifications:  These specifications depart from
the Objectives and Initial Specifications in the following respects:    
 a. No I/O device timing will be performed.                             
 c. The TCT of Subset 1 will be expanded to include information         
    previously  intended  for a new control block (the IOCT).  This will
    eliminate the need to modify the DEB."                              
The final error table was now modified                                  
"                             After Customer Ship                       
 Expected:                  6 months       18 months                    
Severity 2, 3, 4 errors     5 (was 7)      20 (was 15)                  
Man hours required          4 (was 40)     10 (was 40)                  
Machine hours required      4 (was 5)      10 (was 5)"                  
                 "Systems Management Utility - IEHMAN"                  
The  IEHMAN  Planning  Guide  describes  the  same  features  that  were
specified  in  the  SMF  design  for  Subset  2,  called  the  "Resource
Management Utility"!                                                    
Not  only did the IEHMAN Planning Guide exist within IBM, but through an
error,  a  small  number  of   copies   were   actually   shipped   from
Mechanicsburg, Pa., to some IBM customer sites!                         
Once the error was discovered, IBM immediately recalled all copies!     
Ken Plambeck was the author of the IEHMAN package.                      
The project had 10-12 people 12 hrs/day, but the product was killed when
the manager could not get sponsors.                                     
Even though IEHMAN was never announced nor ever released, it showed that
IBM  designers,  even  in  1966-1967,  knew that analysis tools would be
needed to exploit the SMF data.                                         
                In May, 1969, C28-6712-0, Planning for System           
                 Mangement Facilities (62 pages) described              
                the version of SMF planned for Release 18.              
               but the real announcement of availability was:           
                      IBM System 360/Operating System                   
                              Release 18 Guide                          
                                 July, 1969                             
"System Management Facilities                                           
    SMF is a new feature of the operating system, selectable  at  system
    generation  in  conjunction with the MVT configuration.  SMF may not
    be specified in conjunction with Model 65 Multiprocessing.          
SMF provides two basic functions:                                       
  Collecting and recording system-, job-, and step-related data for each
  job processed by the system.                                          
  Monitoring  job  processing  via  exits  from  the  control program to
  installation-provided  routines  at   specific   points   during   the
  processing of each job."                                              
Data  Collection:  SMF writes 13 types of records to a data set resident
on either tape or direct access.  The records contain  such  information
as:   system  configuration,  job  and job step identification, CPU wait
time, CPU usage by each job and job step, and I/O device usage and  main
storage  usage  by  each  job  step.   The writing of SMF records may be
supressed at IPL.  An  installation-provided  routine  can  periodically
analyze  the SMF dataset to evaluate system efficiency, performance, and
Job Monitoring:  SMF provides five exits within the control  program  to
installation routines:                                                  
  From  the  reader/interpreter  of the job just before each job control
  statement is interpreted.                                             
  From the initiator/terminator when a job is selected for initiation.  
  From the initiator/terminator when a step is selected for  initiation.
  From the intiator/terminator when a step and/or job is terminated.    
  From the timer second level interruption handler (SLIH) if a specified
  CPU or wait time limit for a job or step is exceeded."                
  If no wait time limit is specified, the default value provided by  IBM
  is 15 minutes;  in previous releases the default value was 30 minutes.
A new macro instruction (SMFWTM) may be used in exit routines  to  write
records to the SMF data set.  A test procedure (TESTEXIT) is provided in
SYS1.SAMPLIB to facilitate routine testing.                             
You must also provide analysis and report routines to  process  the  SMF
data set."                                                              
                    IBM System/360 Operating System                     
                         Consolidated Document                          
                               Release 18                               
                     Third Edition (October, 1969)                      
"Updated Version of Release 18, designated as Release 18.6, may  now  be
ordered from PID.                                                       
Approximately 40 PTFs have been applied to the  distribution  libraries,
correcting more than 80 APARs.                                          
(Release 18.0 text:)                                                    
A second release of SMF will support MFT, M65 Multiprocessing and Remote
Job Entry.                                                              
Graphics Job Processing (originally planned for the  second  release  of
SMF) is supported in this initial release of SMF.                       
Primary Control Program (PCP) and Conversational Remote Job Entry (CRJE)
are not supported.                                                      
If SMF is chosen at System Generation, performance will be reduced.     
The performance reduction will be dependent upon the installation's  use
of the following:                                                       
   . SMF buffer size                                                    
   . Device used for the SMF data set                                   
   . SMF data set allocation size                                       
   . Number of jobs run per day                                         
   . Execution time of the installation exits                           
The  performance  reduction  to  the  system  when all System Management
Facilities are functioning can be less than 5%.                         
Storage Requirements                                                    
A fixed amount of main storage is required when SMF is chosen at  System
Generation.   In  MVT  a  maximum  of  1500  bytes  is added to the main
resident storage.  Supervisor Queue Space is used  for  data  collection
tables,  new  job  queue  entries, and the user defined SMF buffer.  The
variable storage depends on the number of active jobs in the system  and
the SMF options chosen."                                                
                   IBM System/360 Operating System:                     
                            Release 19 Guide                            
                               June 1970                                
Announced  Subset  2 of SMF and support for MFT and M65 Multiprogramming
and RJE with all  twenty-one  SMF  records  (0-15,17-20)  with  one  new
  Type 13 - MFT Partitions                                              
and one new Exit                                                        
 (IEFUSO) for SYSOUT data set limit exceeded                            
                  System Programming Guide Release 19                   
"Example of SYS1.MANX Space Requirements                                
                                  ID    bytes                           
 1  IPL per day                    0                                    
20  Devices online              8,19                                    
 2  Vary Onlines per Hour          9                                    
 2  Vary Offlines per Hour        11                                    
 1  Device Allocation per Hour    10                                    
 1  Scratch Dataset per 4 Hours   17                                    
 1  Rename Dataset per 4 Hours    18                                    
24  Accumulated Wait Time          1                                    
        Total for these records           2,025                         
Job Processing                                                          
 1  Start of Job                  20                                    
 1  End of Job                     5                                    
 1  Dismount 2 Volumes per job    19                                    
 3  Steps per job                  4                                    
Step Processing                                                         
 3  DD statements per step         4                                    
 2  Input datasets per step       14                                    
 2  Output datasets per step      15                                    
 2  Output writers per step        6                                    
 Total for one job                        6,303                         
 Total for 48 jobs in 4 hours           302,688                         
SMF headers                                                             
  Record Descriptor Words                 5,044                         
  Block Descriptor Words                  1,684                         
TOTAL SMF DATA FOR 4 HOURS:             311,411"                        
                   IBM System/360 Operating System:                     
                            Release 20 Guide                            
                              January 1971                              
Announced 20.0 with support for S/360 and new S/370 155 processor (S/370
165 in April) and indicates to expect 20.6 in 6-8 months.               
                    IBM System/360 Operating System                     
                      System Management Facilities                      
                    (Fourth Edition, February 1971)                     
Applied to Release 20.1  and  identifies  the  eleven  new  SMF  records
created with Release 20 (now totaling 31 record types):                 
 Type 21 - ESV Record                                                   
 Type 30 - Start TSO Record                                             
 Type 31 - TIOC Initialization Record                                   
 Type 32 - Driver Record                                                
 Type 33 - Driver Modify Record                                         
 Type 34 - TSO Step Termination                                         
 Type 35 - TSO Logoff Record                                            
 Type 38 - Initial TSO Configuration Record                             
 Type 40 - Dynamic Allocation Record (not documented!)                  
 Type 41 - Modify TSO Record                                            
 Type 42 - Stop TSO Record                                              
                   IBM System/360 Operating System:                     
                            Release 21 Guide                            
                       March, 1972 (Release 21.0)                       
Announced  the  REC parameter and minor change in format of SMF records,
and indicated that 21.6 would be along in 4-6 months.                   
                   IBM System/360 Operating System:                     
                            Release 21 Guide                            
                      August, 1972 (Release 21.6)                       
was announced, with no SMF enhancements.                                
In October, 1972, I first used the Statistical Analysis System, SAS,  to
read  SMF  records from OS/360 Release 20.6 at State Farm Insurance.  At
that time, SAS cost $100!  By early  1973,  I  reported  our  successful
results  to  user groups (SAM, TESDATA) and ACM (SIGSIM, Chicago).  John
Chapman demanded that I present at SHARE.                               
In March, 1974 (SHARE XLII, Houston), in the CME  project,  I  presented
SMF/SAS;  CME scheduled an open session for the Chicago SHARE.          
That summer, IBM announced  SGP,  their  Statistics  Gathering  Package,
written  by  Bill Tetzloff.  The open session at the August, 1974, SHARE
meeting in Chicago began with Bill's SGP product description followed by
State  Farm  Auto's presentation of their use of SAS and SMF data.  Over
750 people (half of SHARE attendance) were present!  While SGP was truly
a valiant IBM effort to provide reporting from an extract of a fixed set
of SMF fields, it was not easily tailored, was  cast  in  concrete,  and
appeared  inflexible.   This  audience then saw the SAS language for the
first time, and saw SAS actually used for productive SMF  analysis.   At
the  end, one attendee pointedly asked IBM, "Now that you have seen SAS,
is there any reason why we should still consider SGP?"                  
This discovery that the SAS language was highly suited for  analysis  of
SMF  and  RMF data led to many SHARE sessions that demonstrated that SMF
data  analysis  really  did  save  money  and   answered   data   center
management's questions (response, capacity, service objectives, etc.).  
SAS was the tool that made SMF analysis practical, in 1974.             
The early releases of MVS became available:                             
 VS/2   Announced August 1972                                           
 ?/73?  MVS 2.2 MF-1 Type 70-77,79                                      
 ?/75?  MVS 3.0                                                         
 5/76   MVS 3.7                                                         
 8/76   SU7 Supervisor 2                                                
 8/76   SU20 RMF                                                        
 9/76   SU11 TSO CMD. Package                                           
 9/76   SU13 TSO/VTAM                                                   
 3/78   SU58 TSO/VTAM Level 2                                           
 3/78   SU50 MVS SE1 (for 3.7)                                          
 7/78   SU78 TSO Session Manager Rel. 2                                 
 3/79   SU65 MVS SE1.1 (for 3.7)                                        
 3/79   MVS 3.8 (SCP2)                                                  
 8/79   SU74 MVS SE2 (for 3.8)                                          
         Type 23, 30, 32, 90                                            
 1/85   NPDA V3 R2 (Type 37)                                            
 2/85   DFSORT R7 (Type 16)                                             
 3/85   NLDM R3 370 (Type 39)                                           
 7/86   DB2 (Type 100,101,102)                                          
By the mid 1970s, large TSO shops (several hundred concurrent logged  on
users) began noting degredation due to the BSAM SMF writer:             
 - ENQ/DEQ was used by all SMF records, a very slow, serialized server  
           for every logical record written.                            
 - TCLOSE  to update VTOC after every block was written moved the disk  
           drive arm - the drive shook like a washing machine!          
 - WRITE VERIFY doubled the I/O time                                    
In  addition,  the 1975 SHARE-IBM meeting in New York was called because
users were  not  migrating  from  MVT  to  MVS,  and  were  blaming  SMF
accounting  issues, technical issues about actual numbers in the records
(eg., more absolute CPU time, the loss of  "storage  measurement"  a  la
"K-Core  Hours"  from  MVT,  the  inclusion  of PCI counts into SMF EXCP
buckets) caused, Ron Hankison,  then IBM  Representative  to  SHARE  MVS
Group, to become the local SMF expert within IBM.                       
From Ron Hankison 1989 interview:                                       
Accounting  requirements  from SHARE and GUIDE had built up, and nothing
had changed since the original OS implementation (VS 2.2 and VS 1.6  had
only added a few fields regarding paging and Virtual storage).          
TCLOSE  was  out of control, the constraint was apparent and there was a
backlog of user requirements.                                           
Project goals were to:                                                  
   Fix the performance constraint                                       
   Add functionality                                                    
   Restructure after 5-10 years experience                              
Estimated project by doubling the then-known size of SMF  (Source  Lines
of  Code)  and  used  that  (and  IBM internal estimates) for the actual
estimated man-hours.                                                    
The final implementation took  four  times  as  much  as  the  estimate,
requiring 12 people 1.5 to 2 years for SE2 SMF Writer redesign.         
Because  no  one  else in VS 2 cared much about SMF, the development was
independent, which allowed many things to be considered and many went in
and out of the final SMF enhancements.                                  
Primary developers were:                                                
   Barb Butler                                                          
   Bill McTiernan                                                       
   Steve Rosengarn                                                      
   Joe Winterton                                                        
                       Programming Objectives and                       
              Initial Programming Functional Specifications             
                        MVS Accounting Facility                         
                             August 1, 1977                             
"1.4 Summary of Specifications                                          
The  rewrite  of SMF will resolve the numerous, known problems with SMF.
The  performance  of  SMF  will  be  much improved by the elimination of
bottlenecks during the collection and writing of the  data.   Additional
data  will  be  available  for  recording  that  fills  in several known
exposures for resource utilization and system  activities.   Flexibility
will  be built in that allows the installation improved control over the
recording and make tradeoffs between the integrity of the data collected
and   the  performance  impact.   Complexity  will  be  reduced  by  the
capability of establishing a common file to contain  all  the  pertinent
information  needed  to  manage the installation.  Additional useability
items will make this package very appealing to DP installations."       
1.5 Summary of RAS Characteristics                                      
Due to the critical nature of the data  being  handled,  minimizing  the
opssibility  of  loss  of  data  will  be stressed in the design of this
package.  The improvements described in this document will be covered by
either  an FRR or ESTAE routine to ensure that loss of data is held to a
minimum in the event of  a  failure  in  the  SMF  component.   Optional
facilities  will  be  provided  to  minimize  loss of data due to system
failures.  Programming techniques known to  have  demonstrated  improved
quality will be used during implementation and test."                   
2. User Requirements Addressed                                          
       Started Task Reporting                                           
       Interval Reporting                                               
       Performance/Overhead of Data Collection                          
       Recording Selectivity                                            
       Proliferation of Data and Records                                
       Installation Tracking Completeness                               
       Accounting Direction                                             
       Proliferation of Tools                                           
7. Performance                                                          
The  overall  reduction of SMF overhead and its impact on system thruput
will be significant in those environments that have  a  high  volume  of
data  that  is  recorded  to  SMF.   The  performance improvement of the
collection routine and its branch entry capability will provide  a  much
improved  interface for those components that should be recording to SMF
but were concerned about the SMF bottlenecks.                           
The path length for a Write to the SMF data set is approximately  60,000
instructions.  The frequency of that event is installation dependent but
probably averages about once every 13 logical  records.   The  size  and
frequency  of  record  receipts  will be increasing rapidly as processor
speeds increase.  The revised path length by using the VSAM ICIP in  SRB
mode  is  approximately  2500  instructions, resulting in a reduction of
57,500 instructions per Write."                                         
       Summary of the author's opinions:                                
1. SHARE was instrumental in the creation of SMF.                       
2. Users had a fair idea of what data was needed.                       
3.  IBM designers in 1966 knew more than users of the range of data that
we would ultimately need.                                               
4.   The  iterative process between IBM designers and IBM users provided
realistic validation of the design before implementation.               
5.  Designer's specifications and wants exceeded the  funding  and  time
for implementation.                                                     
6.   The  1966-1969  IBM  design  and  implementation process was better
structured, managed, and documented  than  your  company's  most  recent
managed software project in 1989.                                       
7.   Based  on this example of IBM design practices of twenty years ago,
imagine the detail we would find in today's IBM design documents!       
8.  SMF made IBM pre-eminent in the business of real data processing  by
giving  DP  management  actual  measures  of  the resource usage and the
service (response) for each user.  DP could then "manage by  objectives"
and  prove  to  the  president  the  value  and costs of the services DP
provided the company.  No other vendor  of  hardware  and  software  has
provided the accurate measurements that IBM has given its customers.    
9. As good as it is, it still isn't perfect:                            
   MVS/ESA added three important CPU measures to                        
   the type 30 (task level) record,                                     
       RCT  - Region Control Task CPU                                   
       SLIH - Second Level Interrupt Handler CPU                        
       HPT  - Hiperspace CPU                                            
   but we do not have these same CPU measures                           
   in the type 72 (performance group) record.                           
   SHARE REQUIREMENT, ANY ONE?                                          
With  sincere thanks for the dedicated developers designers and users of
SMF data, I especially salute these contributors to this research:      
 Bob Brockish, (retired) IBM         Tom Bell, Rivendel Consultants     
 Lynn Weissenreider, IBM Boulder     Brian Currah, BDC Computer Services
 Saul Schiffman, IBM Japan           Aron Eisenpress, CUNY              
 Ron Hankison, IBM Kingston          Mario Morino, Morino Associates    
 Dick Armstrong, IBM Gaithersburg                                       
 Jim Doyle, IBM Poughkeepsie                                            
 Jack Higgins, IBM Publications                                         
especially  for   their   treasure-trove   of   original   IBM   project
documentation   as  well  as  their  time  in  reminiscing  on  personal
This section added March 4, 2017, after the article was posted to the   
IBM-MAIN ListServer, and Joe Witherton (cited above) posted this note:  
Thanks Barry for the walk down memory lane.                             
I had a few years working on SMF in my early IBM days. First I was the  
receiving programmer to enhance SMF for VS2 rel 1 in Poughkeepsie when  
it moved from Boulder to Pok. I did some minor SMF enhancements for VS2 
rel 1.                                                                  
Then later in the late 70's I also was team leader for this project to  
enhance SMF and will add to this text below a little. We listened to    
Barry M and Share as they helped us understand the customer needs.      
Actually we knew the sizing effort had to be carefully handled at IBM   
and knew how much people months we could recommend to spend on the      
project and if sized correctly the project may not be approved. We low  
balled the sizing. Once we got it approved we felt we could get the     
needed people added to finish the project and we did. I later took the  
hits for our poor sizing but we met the customer needs as the team felt 
the passion to make these enhancements for Share customers. I also      
presented for IBM at Share in San Francisco to a packed ballroom the    
announcement for MVS SE2 and described how the release met many of the  
Share requirements for SMF.                                             
"Estimated project by doubling the then-known size of SMF (Source Lines 
of Code) and used that (and IBM internal estimates) for the actual      
estimated man-hours.                                                    
The final implementation took four times as much as the estimate,       
requiring 12 people 1.5 to 2 years for SE2 SMF Writer redesign.         
Because no one else in VS 2 cared much about SMF, the development was   
independent, which allowed many things to be considered and many went in
and out of the final SMF enhancements."                                 
Joe Winterton