|
HP OpenVMS Systemsask the wizard |
|
The Question is: This is a question about tuning RMS file performance. I've got several large "sequential, stream_LF" data files created/accessed by C code. The files are either 32688 or 58608 blocks, and they are organised into 144 "pseudo-records" of 227 or 407 blocks length. The files are on a RZ26L disk. One of my applications (a user interface) needs to read all 144 "pseudo-records" and this takes a fair bit of time (approx 9 or 16 seconds). "Pseudo-records" are read by doing an fseek followed by an fread. How do I cut this time down by tuning... 1) OpenVMS 2) Process/account limits/quotas 3) File/access "parameters" ------------------------------------------ I've already tried several things but seem to be getting inconsistent results (details for smaller files only)... 1) Initial read - approx 9 seconds. 2) Subsequent read - approx 3 seconds. I presume this is using the "Virtual I/O Cache". 3) Use "read ahead" and/or increase "multi-buffer" (=10) - no significant difference. 4) Use "multiblock" (=127) - approx 7 seconds but subsequent reads don't get any faster (almost as if the "Virtual I/O Cache" is not being used). This change seems to occur with the "multiblock" somewhere between 30 & 40. Can the cache be "tuned" ? 5) Use "multiblock" & "multibuffer" - approx 6 seconds. 6) Make files "contiguous" (couldn't use convert/fdl - record length, used a "copy" C program) - back upto 7 seconds !!! 7) One other anomaly - two of the larger files are nominally identical (except for the data values), yet one of them takes 1.5 times longer to read (regardless of order). This occured both before and after the files were made "contiguous". Is the physical position on the disk significant ? ------------------------------------------ Allthough I've got some idea of the issues involved, the VMS manuals aren't really helping fill the "gaps". I'd like to know what's going on ! Thanks. The Answer is :
The C RTL can choose to use RMS RECORD IO or BLOCK IO for IO requests.
It uses RMS IO when the user tells it to (ctx=rec) or when other user
requests make this a requirement (sharing!). It uses BLOCK IO for
'simple' access in order to save the RMS processing time (switches
to inner mode and back). For many applications this BLOCK IO mode
is more efficient, for some RMS does a better job. One way to verify
which on is used is SET FILE /STAT ... and MONITOR RMS /FILE ...
For RMS you should see best performance with SET RMS/BUF=mbf/BLOCK=mbc
where mbf is at least 4 (diminishing returns) and mbc as least 60.
If the file is re-read, then you may want to choose mbc less than 35 to
allow it to be cached by VIOC as you speculated.
Recent C RTL versions 'listen' to those RMS controls as well, while
older versions do not.
The buffer settings will matter less, as you get more CPU bound.
The Wizard uses LIB$SHOW_TIMER in his test programs to learn more.
If you are CPU bound then you may want to look at less sophisticated IO.
It would appear to me that this application could be really well suited
to issue SYS$QIOW calls for a 'logical' record in one go. Normally I
would propose SYS$READ but its limit is 127 blocks.
Now at 6 seconds for 144 records of 407 blocks you'd be reading
(and do some RTL record processing) at 5Mb/sec. This approaches
the limit of some DISK IO subsystems. To get more than that, you'll
need to take a close look at the DISKs and CHANNELS in question.
You may need FWD or UltraSCSI (KZPBA) or striping (HSZ, HSJ, KZPSC)
to get faster. You may need software striping to get much faster.
|