August 25, 2009
 |
- Welcome to Cheryl's List!
If you obtained this from someone else and would like your own FREE copy
of Cheryl's List in
the future, see the information at the end of this email.
- Session 2109 — Cheryl's Hot Flashes #22, Friday, 9:30 am
- Session 2149 — Meet the Experts: WLM, Performance & Capacity Planning Subjects, Thursday, 4:30 pm
- Session 2235 - z/OS Mean Time to Recovery Improvements, Wednesday, 9:30 am
- OA22414 (z/OS 1.8-1.9, 25Jun2008) — SMF23 Statistics. This APAR produces an SMF type 23 record that contains new workload characteristics in addition to I/O density (TCB & SRB dispatches, DASD I/O, and RSM services such as getmains, page faults, fixed storage, and free storage) that may prove useful in the future for matching production to LSPR workloads.
- OA27161 (z/OS 1.8-1.10, 4Feb2009) — New Function - Additional SMF Type23 Statistics. Adds "delta" counters for the previously added new fields.
- 1. About Cheryl Watson’s Tuning
Letter 2009 No. 4
2. SHARE in Denver
3. SMF Status Correction
4. RMF Interval Recommendation
5. Tuning Letter 2009 No. 4 Update
6. IBM Red Alert
The forty-six page 2009 No. 4 Tuning Letter was emailed to paid subscribers on August 20th. You may visit our Web site at www.watsonwalker.com to obtain subscription information. The following is our Management Summary page from that issue, talking about some of the contents of this latest Tuning Letter:
SMF Management
SMF data is the basis for all tuning,
capacity planning, chargeback, and performance work. Our SMF Series
— Part 6 shows how to best manage the huge amount of data produced
by DB2, CICS, and Workload Manager. Did you know that there's a WLM type
99 record that you should always keep? I also cover the security records
that are so very important to auditors. And I discuss controlling DB2 data,
by far the most important concern for most installations. I've seen sites
where 90% of SMF records are produced by DB2.
Offloading Work
to Specialty Processors
A recent press release by a software
vendor has led to a lot of speculation about how much work can be offloaded
from the general purpose processors to specialty processors, such as the
zIIPs and zAAPs. The vendor says that you can offload 50% of your work
and reduce your software costs by 20%. But because they haven't used the
IBM-approved interfaces, nobody can say whether IBM will accept the offloading.
And according to a letter from IBM, there is a risk in attempting to use
this technique to reduce costs. See my full discussion of this situation
in our What's New? section.
Elsewhere in
this Issue
Our User Experiences section
provides a warning to be careful when migrating to z/OS 1.10 because the
documentation for WLM needs updating. There are several references specifically
for managers relating to use of the System z machines for Linux in our
Publications
section. We've also noticed recently that many of IBM's
New Function
APARs are containing many APARs that look like they should be HIPER.
Because New Function APARs are often overlooked, we've highlighted some
of the more critical ones for you.
2. SHARE at Denver
This week, I'm at the SHARE conference
in Denver, and I hope to see many of you there. I especially like the "Summer"
conference because IBM provides the specifics of the release that becomes
available in September — in this case, z/OS 1.11. And if you can't make
it to SHARE, you can look at most of the presentations online. Go to www.share.org
for the proceedings. (Some presentations are not out there until the end
of the week.) I think that SHARE is some of the best education you can
find. Here are three sessions I'll be at:
In Tuning Letter 2009 No. 3, I included a recommendation to shorten the frequency of the type 23 records (SMF buffers and workload characterization) by using the SMFPRMxx parameter of STATUS. The default is STATUS(010000) - one hour, but I recommended 10 or 15 minutes. You can also use STATUS(SMF,SYNC), which indicates that type 23 records are written at the SMF interval (normally 30 minutes) and synched to the RMF synchronization (normally 0). I mistakenly recommended that you use STATUS(001000,SYNC), which is an invalid combination. Neal Keller of Reed Elsevier Technology Services pointed out that you can use SYNC only when you’re using the SMF global interval. Because the SMF global interval is usually set at thirty minutes, you have to choose between synchronizing the records with (SMF,SYNC) or using smaller intervals and not synchronizing them with (001000). Thanks, Neal! So my recommendation has changed to STATUS(SMF,SYNC).
One of the reasons that the type 23 records are more important today is that IBM is now using them to collect new data on workload characterization. I discussed this in our Tuning Letter 2009 No. 2. Here is an extract from that newsletter regarding a SHARE presentation given by John Burg from IBM's WSC:
SMF Type 23 - The second item [that John presented] was a description of the SMF type 23 record. While this record used to simply contain SMF buffer statistics, it was expanded last year to contain workload characteristics that might help IBM classify the type of workloads that run on your machine. This would allow you to better use the published LSPR findings. I discussed this record in our Tuning Letter 2008 No. 2 (page 24), and asked that customers contact Gary King and provide him data from your site. John's now taken on that work and has asked that customers contact him to provide the SMF type 23 data. APAR OA22414 provides the data for workload analysis and a description of the new fields. New APAR OA27161 adds "delta" counters for the new fields. Once you've applied these two APARs, please contact John in order to send him your data. They need many more customer samples for their analysis. As John says, it's an "opportunity to ensure that your data is used to influence clustering analysis" to see if workload characterization can be used for zPCR or other tools. John’s email is jburg@us.ibm.com, and he’s looking for volunteers who will provide him with up to 24 hours of SMF 23s, 70s, 72s, and 113s per LPAR.
An SMF type 113 record can be produced from HIS, although there are currently no programs to produce reports from this record or the UNIX files that contain the samples. We don't know if these will come later, or be incorporated into a product. ISVs will probably use this facility a lot. I expect to provide a longer article on the CPU MF after more people have tried it. If you do use it, please tell me what you find.
If you’re going to jump in and use John’s handout to run HIS, then I’ve got some suggestions:
- 1. Study the documentation
carefully before you begin. In addition to John’s presentation, you can
also look at:
- Ed Jaffe’s SHARE session
2839
(slides 16-21). This has some good examples of output.
Greg Dayne’s SHARE session 2848 (pages 54-55). This has a good set of steps for implementation.
IBM Research article - IBM System z10 performance improvements with software and hardware synergy at http://www.research.ibm.com/journal/rd/531/jackson.pdf.
SA22-7627-19 - MVS Commands, “Setting up hardware event data collection.”
SA23-2260-00 - The Set-Program-Parameter and CPU-Measurement Facilities.
SA23-2261-00 - IBM The CPU-Measurement Facility Extended Counters Definition for z10.
[Editor's Note - The hyperlinks
refer to SHARE sessions from the Austin SHARE.]
3. You can probably safely use COUNTERS mode, which has low overhead (less than 1/100th of a second for the HIS address space during a 15-minute interval). But be careful of using SAMPLING mode because it can produce HUGE volumes of I/O unless you change the sampling frequency. The default of SAMPFREQ=800000 and DURATION=10 produces 8 million samples in 10 minutes. Start with a small value (e.g. SAMPFREQ=320), and never go higher than SAMPFREQ=130000 for a smaller z10 BC.
I'm very excited about the possibilities
that the CPU MF can provide, and I hope that I'll have an opportunity to
either use it or hear from a customer who has used it. This was one of
my favorite things from SHARE.
In our Tuning Letter 2008 No. 2 (pages
8 & 9), I made the following recommendation regarding the length of
the RMF interval:
- Recommendation:
My new recommendation is that you record RMF to match the length and synchronization
of INTVAL (30 minutes) and SYNCVAL (0). You will need to realize that everything
will tend to look better with 30-minute intervals, as compared to 15 minutes.
For example, if you are doing 10 minute intervals and get the following
response times for a volume (in ms): 7, 25, and 5, then a program
looking for device times over 20ms will identify this device. But a 30-minute
interval would produce an average re-sponse time of 12ms, which wouldn’t
be noticed.
In our Tuning Letter 2008 No. 4 (page 33), I included a chart and explanation from Chuck Hopf showing why he disagreed with my recommendation. I especially like his conclusion: "The larger the interval the more the peaks and valleys get washed out. I know IBM would like us to plan based on averages but if I stick my head in a hot oven and my feet in the freezer, my average temperature is still fine."
5. Tuning Letter 2009 No. 4 Update
On page 40 of Tuning Letter 2009
No. 4, I used the wrong link for the MXG site. Here is the corrected paragraph:
- A Serendipitous Life. The IBM
Systems Magazine May/June 2009 issue contains a wonderful article about
a wonderful man, Dr. Barry Merrill, President and founder of Merrill Consultants
(http://www.mxg.com).
Here are two APARs that were updated
since I included them in the newsletter:
- PK64852
(z/OS 1.9–1.11, 24Jul2009, Closed UR1) — New Function. The APAR
was taken to avoid a hang condition between two processes that use pipes
or sockets for communication. The problem could not be reproduced, but
the documentation is updated for C/C++ programmers to show how to prevent
such problems.
[Page 34, New Function APARs]
PTFs were issued on August 13, 2009. The note about the problem not being reproducible was removed.
OA18461 (z/OS 1.9–1.10, 29Jul2009, Closed UR1) and PK75626 (DB2 V9, 8Aug2009, OPEN) — Updates to WLM Dynamic Buffer Pool Management. WLM support for DB2 Buffer Pool Management was supposed to be available starting with z/OS 1.9. Unfortunately, the WLM support for this does not work as expected. This is closed as UNREPRODUCIBLE IN NEXT RELEASE. [Page 34, Hiper & Performance APARs]
PTFs were issued for OA18461 (for
z/OS 1.9-1.11) on August 12, 2009. The note about the problem not being
reproducible was removed.
IBM issues Red Alerts for especially
important APARs, and the most recent one was in early July. You may subscribe
to their service at http://www14.software.ibm.com/webapp/set2/sas/f/redAlerts/home.html.
In case you don’t subscribe to these alerts, you should at least be aware
of them.
- 2009.08.24
— Possible data loss for z/OS 1,8, 1.9 and 1.10 with fix and HSM PATCH
enabled for OA22507
[z/OS 1.8-1.9, 8Nov2007 — GDG Base Held for Too Long by HSM Migration].
The patch allowed changing GDG scratch processing during migration. When
enabled, HSM may erroneously invalidate data sets. The data may be lost
if no backup exists. For more information, see APAR OA30149
(HIPER, OPEN) — The TTOC Entry for a GDS Can Be Erroneously Marked Invalid
When
HSM Space Management Migrates the GDS to ML2.
Cheryl Watson
====================================================
Thanks for subscribing to Cheryl's List!
Please note that this email service is not intended to match or even summarize the large scope and volume of information we provide in Cheryl Watson’s Tuning Letter. The Tuning Letter is published six times a year and costs $950 (2009) for the electronic version: a yearly CD-ROM of all back issues since 1991 and six PDF issues emailed during the year. See www.watsonwalker.com/TUNING.html for details. Past tables of contents are at www.watsonwalker.com/TofC.html, and some FULL SAMPLE ISSUES can be found at www.watsonwalker.com/sampleissues.html.
If you obtained this from someone else and would like your own copy in the future, just go to our Web page at http://www.watsonwalker.com and fill out the form under "Cheryl's List." That signs you up, and it's free! It's a one-way list, from us to you. If you make a "reply," it will come just to us, not to the other members of the list. We never share your address or information with anyone else. To unsubscribe or change your email address, use the Web page and password mentioned in your welcome message. Or, you can send an email request to admin (at) watsonwalker.com. Past Cheryl's List issues, starting with issue #1 from December of 1996 can be obtained at www.watsonwalker.com/archives.html.
In Cheryl's List, we provide updates to our full newsletter, Cheryl Watson's Tuning Letter. We’ll also let you know if anything of importance has been added to our Web pages, and tell you about our other products and services, including BoxScore and GoalTender, our two software offerings.
So - we hope you'll find this service
valuable. Be sure to send email to technical (at) watson-walker.com
if you have any questions or comments.