[an error occurred while processing this directive]

HP OpenVMS Systems

ask the wizard
Content starts here

Hypersort on larger files?

» close window

The Question is:

 
Hello.  We are trying to use the hypersort utility on an Alpha running
OpenVMS v 7.2-1.  It seems to work on relatively small files (<1m blocks)
but when we try to sort a 25m block file with 1 key, it seems that hypersort
never kicks in and the sortmerge.exe utility just chugs along and takes
forever.  We define the sortshr logical to point to sys$library:hypersort,
we also turn on image accounting...but there is no sign of it being called.
Any info or hints you can give on Hypersort would be greatly appreciated.
Thanks! - Clint McCusker
 


The Answer is :

 
:OpenVMS v 7.2-1.  It seems to work on relatively small files (<1m
:blocks) but when we try to sort a 25m block file with 1 key, it
:seems that hypersort never kicks in and the sortmerge.exe utility
:just chugs along and takes forever.  We define the sortshr logical
:to point to sys$library:hypersort,
 
  This appears to be a known problem with Hypersort V04-001 and
  earlier.  This has been fixed with Hypersort V04-002 and later.
  If a Hypersort ECO kit is not presently available for your OpenVMS
  version, please contact the Compaq Customer Support Center and
  request an ECO kit.
 
  The Hypersort V04-002 release notes are attached.
 
  With Hypersort V04-001 and earlier, you can often avoid this problem
  by using a smaller working set size.  To start, make sure your page
  file quota is at least 3x your working set extent and measure the
  actual WSEXTENT and PGFLQUOTA with
 
    $ write sys$output "WSEXTENT =''F$GETJPI("","WSEXTENT")'"
    $ write sys$output "PGFLQUOTA=''F$GETJPI("","PGFLQUOTA")'"
 
  Also, there is a known Hypersort problem that is not fixed with
  V04-002.  That problem shows up if Hypersort runs out of free space
  for all sort work files.  In this case, Hypersort may hang.  The
  best way to avoid that problem is to first use SORT/STAT with SORT32
  to make sure there is sufficient work file space for SORT32.  If
  SORT32 can sort the file, this usually means Hypersort will have
  sufficient work file space.  The amount of virtual memory and work
  file space needed is very dependent on reasonable settings for
  longest record length and maximum record size.  Do DIR/FULL for the
  input file to make sure these are set reasonably for your file.
 
  The reason to use SORT/STAT with both SORT32 and Hypersort is so you
  can compare which utility performs better for your input file.  In
  some cases, Hypersort will perform better.  In some cases, SORT32
  will perform better.
 
 
  Release notes:
 
  V04-002 Hypersort and large file processing
 
  Hypersort provides support for high performance sorting of large files.
  Two bugs were found in this support that resulted in incorrect offset
  computations, and an eventual hang or ACCVIO.  These problems have
  been corrected, but you must set page file quota to at least three
  times working set extent as documented in the OpenVMS V7.3
  release notes.
 
  V04-001 Hypersort support for user_compare and/or user_equal
 
  Hypersort does not support the SORT32 API feature of allowing an
  application caller to specify an address for the user_compare and/or
  user_equal parameters, and now correctly returns NYI in those cases in
  which there's an attempt to do so.
 
  V04-001 Hypersort internal work file and user output file extensions
 
  Hypersort extends on-demand any work file or output file that it is
  writing.  A bug in the extend logic that caused the sort to terminate
  prematurely has been identified and corrected.
 
  V04-001 Hypersort compliance with the SORT32 file naming convention
 
  Hypersort should comply with the SORT32 file naming convention of
  cascading input file path attributes for use in forming the output file
  name.  A bug in the cascading logic has been found and corrected,
  resulting in improved compliance with SORT32.
 
  V04-001 Hypersort and output file attributes
 
  Hypersort should detect and set accurate output file attributes based on
  the input data being processed.  A bug was found that resulted in an
  inaccurate MRS attribute.  This has been corrected.
 
  V04-001 Hypersort use of AST quota
 
  Hypersort uses asynchronous QIO to process basic RMS file types.  A bug
  was found that resulted in an AST quota depletion, and an eventual hang.
  This has been corrected.
 
  V04-001 Hypersort and NULL duplicate records
 
  Hypersort allows for duplicate record processing.  A bug was found that
  resulted in an unintentional filtering of NULL duplicate records.  This
  has been corrected.
 
  V04-001 Hypersort and RMS-F-SYN
 
  Hypersort now properly issues the secondary diagnostic
  -RMS-F-SYN, file specification syntax error.
 
  V04-000 Hypersort and searchlists - known problem
 
  Hypersort does not fully support searchlists for input files and
  work files.
 
  V04-000 Hypersort and SORT32 performance
 
  Hypersort and SORT32 use different sorting and work file algorithms.
  Either sort utility may be faster depending on the input file
  and the memory/disk/CPU configuration.
 

answer written or last revised on ( 19-SEP-2001 )

» close window