HP OpenVMS Systems

ask the wizard
Content starts here

Seeking Fast File I/O?

» close window

The Question is:

I am currently trying to optimize the throughput of binary file I/O using the C
 RTL and have been running some tests to see what values affect the
 performance.  I built a program that creates a 160000 block file, writing 8k
 at a time.  I tested using STRE
AM, STREAM_LF, UNKNOWN, and STREAM_CR formats.  I found that STREAM_LF,
 UNKNOWN, and STREAM_CR were all about the same in performance, but STREAM was
 about 2x faster.  I did run into problems with STREAM changing 0A to 0A0D but
 then I found the CTX=BIN op
tion and that seemed to solve the problem.  I also found that using the option
 MBC=64 on the create gave another 2x improvement. Also all sharing was turned
 off as that seemed to degrade performance 2x.
I checked the file attributes after the tests and noticed that the STREAM files
 had a maximum record size of 8k; whereas, the STREAM_LF files had a maximum
 record size of ~32k.  I noticed that 32k is the maximum record size for
 STREAM_LF files so I set th
e maximum record size option on the create to 8k but didn't see any noticable
 difference.  Also I did find that STREAM files insert a 0A0D at the end of
 each of the records in the file.  So the STREAM file was actually 40 blocks
 bigger than the STREAM_LF
file.  Why does STREAM require the 0A0D at the end of each record since we are
 writing binary data?  Why is STREAM format faster than the other formats?  Is
 there a way to make either STREAM_LF or UNKNOWN format as fast a STREAM, or is
 there a way to make
 STREAM not put the 0A0D at the end of each record and still get the same
Michael Downey

The Answer is :

  If you want the fastest file I/O available for binary file data, use
  the sys$qio interface to the XQP as described in the OpenVMS I/O User's
  Reference Manual.  Given that this is an OpenVMS Alpha system, also
  consider using the available Fast I/O system services.
  You will find that applications using C and RMS will generally have
  lower performance than applications that are using sys$qio or the
  (faster) sys$io_perform (Fast I/O) services.
  If you choose to use sequential files and RMS, consider the SQO option.
  Stream files are not normally used for binary data files, being a
  format intended for text data and thus requiring record delimiters.
  RMS itself has assumptions on the maximum size permitted of a record.

answer written or last revised on ( 8-FEB-2001 )

» close window