HP OpenVMS Systems

Performance of Large Directories?

» close window

The Question is:

 
We have an application that uses a large number of directories under it's
 "root" login default directory. There are 1,749 sub-directories under it's
 root [trademgraabp]. And under one of the sub-directories [.interface] there
 are 64 sets of additional dir
ectories which have 4 to 6 additional directories each. Several of these
 directories have as many as 50,000 files created into them on a daily basis.
Directory TRADEMGRAABP.INTERFACE.UIABN]
DONE.DIR;1      371  29-JUL-2003 23:16:18.15
KEEP.DIR;1        1  11-AUG-2003 14:07:55.21
RESULT.DIR;1    406  29-JUL-2003 23:33:35.96
TMPFILE.DIR;1     1  30-JUL-2003 00:18:49.00
TODO.DIR;1      135  30-JUL-2003 00:18:49.01
 
The Problem: As the daily processing goes on and more and more files are
 written to these directories, it takes MUCH longer to create and subsequently
 read these files. It almost quaduples! Is there anything you know of that
 could help speed up this proce
ss?
 
Also, it takes hours to either rename or delete the files. I am aware of a
 trick to reverse sort a list of filenames to delete, but I need something to
 make the Creation and Reading/Processing of the file faster.
 
Thanks,
Rich
609-497-2041
 
 

The Answer is :

 
  It would initially appear that your disk volumes are fragmented -- the
  directory files must be contiguous, and extending the files can result
  in delays locating sufficient contiguous storage.
 
  Please use DFU (Freeware) or DFO to verify the volume fragmentation,
  and consider off-loading and/or defragmenting the volume(s).  Also look
  at the application and process and volume file extent size defaults,
  and determine if the application data files involved here are being
  appropriately extended -- insufficient file extent settings tend to
  result in fragmented files and fragemented volumes.  The proper value
  is often somewhere up to around 500 blocks, depending on the file
  size and the frequency of access.  If you know the file size that will
  be required, consider preallocating it when the file is created.
 
  The choice of ODS-2 or ODS-5 will not particularly affect the operation
  or the performance of the directory file extension operations.  Further,
  you presently have the performance optimizations implemented in OpenVMS
  V7.2 and later, around the performance of larger-sized directory files.
 

  

     

     
answer written or last revised on ( 24-NOV-2003 )
     
» close window