HP OpenVMS Systems Documentation

OpenVMS User's Manual

A pipeline is a sequence of pipeline-segment commands connected by pipes, represented by the vertical-bar (|) separator. A pipeline-segment command is a DCL command that appears in a pipeline. The pipe connects the SYS$OUTPUT of one pipeline-segment command to the SYS$INPUT of the next command. The format of a pipeline is as follows:

Each pipeline-segment command runs in a separate subprocess with its SYS$OUTPUT connected to the SYS$INPUT of the next pipeline-segment command. These subprocesses execute in parallel; however, they are synchronized to the extent that each pipeline-segment command, except the first, reads the standard output of its predecessor as its standard input. A pipeline completes execution when the last pipeline-segment command is finished.

It is very common to use "filter applications" in a pipeline. A filter application is a program that takes data from SYS$INPUT, transforms it in a specific way, and writes it to SYS$OUTPUT.

Some aspects of DCL function differently in the context of a pipeline. For example:

• Using SYS$COMMAND
  The SYS$COMMAND of a subprocess is normally the same as its SYS$INPUT (if no command procedures are involved). In a pipeline, however, the SYS$COMMAND of a subprocess is set to the SYS$COMMAND of the parent process rather than to the preceding pipe (which is the SYS$INPUT of the pipeline-segment command).
• File access methods
  A pipeline segment command can only use the RMS sequential file access method to read and write to the pipes. Certain OpenVMS utilities may access their input and output files using methods other than sequential access. These operations are not supported in a pipeline, and will fail, as in the following example:

  $ PIPE CC/NOOBJ/NOLIS TEST.C | SEARCH SYS$INPUT/WIND=(1,1) "%cc-w-" %SEARCH-F-RFAERR, RMS error using RFA access -RMS-F-RAC, invalid record access mode

  
  In this example, the /WINDOW qualifier for the SEARCH command requires the relative file access method.
• Symbol substitution
  Be aware of the order in which DCL translates symbols. Symbol substitution takes place during phase 1 of command processing. If you define a symbol as part of a PIPE command, DCL attempts to translate the symbol before performing the command in which the symbol is actually defined. Use the ampersand (&) to defer symbol substitution. For more information, see Section 12.12.2.
• Using SYS$PIPE
  In most cases, input from the pipe can be obtained by reading the data from SYS$INPUT. However, when a command procedure is invoked as a pipeline segment command, SYS$INPUT is redirected to the command procedure file. To obtain data from the pipe inside a command procedure, the logical SYS$PIPE can be used.
  The following is an example of a pipeline DCL application TEE.COM:

  $ ! TEE.COM - command procedure to display/log data flowing through $ ! a pipeline $ ! Usage: @TEE log-file $ $ OPEN/WRITE tee_file 'P1' $ LOOP: $ READ/END_OF_FILE=EXIT SYS$PIPE LINE $ WRITE SYS$OUTPUT LINE ! Send it out to the next stage of the pipeline $ WRITE tee_file LINE ! Log output to the log file $ GOTO LOOP $ EXIT: $ CLOSE tee_file $ EXIT

  
  To use TEE.COM, enter the following PIPE command:

  $ PIPE SHOW SYSTEM | @TEE showsys.log | SEARCH SYS$INPUT LEF

  
  The command procedure TEE.COM is used to log the data flowing through the pipeline. It reads in the data from SYS$PIPE instead of SYS$INPUT.
• Image verification
  In a pipeline, image verification is turned off by default, even when the command SET VERIFY=IMAGE is executed before the PIPE command is entered. This prevents duplication of data records going through the pipeline.
  To turn on image verification in a pipeline, an explicit SET VERIFY=IMAGE command must precede the pipeline segment command. You can use a subshell to do this, as follows:

  $ PIPE ... | (SET VERIFY=IMAGE ; ...) | ...

A subshell is one or more command sequences separated by separators and enclosed in parentheses. The format of a subshell is as follows:

The command sequences in a subshell are executed in a subprocess environment. DCL waits for the subshell to complete before executing the next command sequence. The ( ) separator is similar to the SPAWN/WAIT command.

When using the PIPE command in this format, handle symbol substitution carefully. After defining a symbol, precede subsequent references to that symbol with an ampersand (&) to delay symbol substitution. Otherwise symbol substitution takes place during phase 1 of command processing, at which time the symbol definition is unreliable.

14.20.4 Using the PIPE Command for Background Execution

Command sequences can be executed in a subprocess environment by using the following form:

DCL does not wait for the command sequences to finish. Control passes back to DCL once the background subprocess is created.

14.20.5 Using the PIPE Command for Input/Output Redirection

A command sequence can redirect its SYS$INPUT, SYS$OUTPUT, or SYS$ERROR to a file during execution of the command as follows:

• To redirect SYS$INPUT:

  PIPE command-sequence < redirected-input-file

• To redirect SYS$OUTPUT:

  PIPE command-sequence > redirected-output-file

• To redirect SYS$ERROR:

  PIPE command-sequence 2> redirected-error-file

A pipeline-segment command can also redirect its SYS$INPUT, SYS$OUTPUT or SYS$ERROR. However, SYS$OUTPUT redirection is allowed only for the last pipeline-segment command, and SYS$INPUT redirection is allowed only for the first pipeline-segment command.

Note that a PIPE command redirection is different from one created using the DEFINE or ASSIGN command. The differences are as follows:

• Redirections are created in supervisor mode. This means that both user-mode applications and DCL commands are affected by the redirections.
• The redirected environment only applies to the command sequence or the pipeline-segment command that specifies the redirection syntax. After the execution of the command sequence or pipeline-segment command, the original process input/output environment (for example, SYS$INPUT, SYS$OUTPUT and SYS$ERROR) is restored before command execution continues.

When SYS$OUTPUT is redirected, the redirected output file is always created, whether or not the command sequence actually writes to SYS$OUTPUT. If a version of a file with the same name as the redirected output file already exists, a new version of that file is created. (This behavior is the same as using the DEFINE or ASSIGN command to redefine SYS$OUTPUT in supervisor mode.) Note that the redirected file is created before the command sequence is executed. If the redirected file is also used in the command sequence, the operation may fail, as in the following example:

$ PIPE SEARCH TRANS.LOG "alpha" > TRANS.LOG
%SEARCH-W-OPENIN, error opening TRANS.LOG;2 as input
-RMS-E-FLK, file currently locked by another user

When SYS$ERROR is redirected, the redirected error file is only created when the command sequence actually writes to the SYS$ERROR during execution, and there is no existing file with the same name as the redirected error file. If a file with the same name as the redirected error file already exists, that file is opened as the redirected error file. The error output generated by this command sequence is then appended to the end of the redirected error file. (This behavior is the same as using the DEFINE or ASSIGN command to redefine SYS$ERROR in supervisor mode.)

14.20.6 Interrupting a PIPE Command

You can interrupt a PIPE command by pressing Ctrl/Y. If the PIPE command is executing in a pipeline or a subshell command sequence, the command sequence and the PIPE command are deleted. In this case, a CONTINUE command entered immediately after the interrupt will not resume the execution of the PIPE command.

If the PIPE command is executing a command sequence other than a subshell or a pipeline command sequence, DCL behaves as if the command sequence were entered as a DCL command without the PIPE command verb and interrupted by Ctrl/Y. See Section 13.11 for more information about the Ctrl/Y interrupt.

14.20.7 Improving Subprocess Performance

A PIPE command can generate a number of subprocesses during execution. Often, the applications invoked by command sequences do not depend on the process logical names and symbol names. In this case, the spawning of subprocesses can be accelerated by using the /NOLOGICAL_NAMES and /NOSYMBOLS qualifiers, which suppress the passing of process logical names and symbols to the subprocesses created by the PIPE command.

The following examples use the PIPE command:

• The following example shows two simple uses of multiple commands with symbol definitions to build useful tools in command procedures:

  $ CD_WORK :== PIPE SAVE_DIR=F$DIRECTORY() ; SET DEFAULT FOO:[WORK] $ BACK :== SET DEF 'SAVE_DIR' $ $ CD_WORK ! Switch to working directory $ : $ : $ BACK ! Switch back to home directory $ GET_RECORD :== PIPE READ/END_OF_FILE=CLEANUP IN RECORD ; - F$EDIT(RECORD, "COMPRESS, TRIM") $ $ OPEN IN EMPLOYEE.DAT $ LOOP: $ GET_RECORD $ : $ : $ GOTO LOOP $ $ CLEAN_UP: $ :

• The following example shows a compile and link operation. Note that if the compilation does not generate any error, the object file is linked to produce an executable image. If the program compilation generates an error, the linking step is skipped.

  $ PIPE cc foo.c && link foo, sys$library:vaxcrtl.olb/lib

• The following example shows how you can use a conditional command execution to easily set up simple error handling control flow in a command procedure. Note that if the image COLLECT_DATA fails, control is directed to CLEAN_UP.

  $ PIPE RUN COLLECT_DATA.EXE || GOTO CLEAN_UP $ : $ : $ EXIT $ $ CLEAN_UP: $ : $ :

• The PIPE command in this example creates a background process to handle the copying of a large file.

  $ PIPE COPY LARGE_FILE.DAT REMOTE"user password"::[DESTINATION]*.* &

• The following example shows how a subshell command sequence is set up to be done in a subprocess. As a result, changing a process-specific characteristic (for example, the default directory) will not affect the current process after the subshell is finished. Note that the save set is restored in a subdirectory to provide the necessary data to run the program FOO.

  $ PIPE (SET DEF [.DATA_DIR] ; BACKUP DATA.SAV/SAV [...]) ; RUN FOO

• The following example uses the pipeline function to identify all hibernating processes on the system in one command:

  $ PIPE SHOW SYSTEM | SEARCH SYS$INPUT HIB

• The following example uses the pipeline function to run a test, sort the result, and compare the result to the benchmark file in a single command without generating unnecessary intermediate files:

  $ PIPE RUN TEST | SORT/SPECIFICATION=TEST.SRT SYS$INPUT SYS$OUTPUT - | DIFF SYS$INPUT TEST.BENCHMARK

• The following example shows one way a subshell can be specified as a pipe segment command in a pipeline:

  $ PIPE ( SET DEF WRK$:[WORK] ; RUN REPORT ) | MAIL SYS$INPUT SMITH

• The following example shows the use of the /PAGE qualifier within a pipeline. The /PAGE function exists in a number of other DCL commands as well and can be used similarly with the PIPE command to form other useful tools.

  $ more :== TYPE/PAGE=SAVE SYS$INPUT $ PIPE ANA/RMS PAGE.TXT | more Check RMS File Integrity 26-JAN-2002 16:12:00.06 Page 1 SYS$SYSDEVICE:[TEST]PAGE.TXT;2 FILE HEADER File Spec: SYS$SYSDEVICE:[TEST]PAGE.TXT;2 File ID: (4135,58220,0) Owner UIC: [PIPE] Protection: System: RWED, Owner: RWED, Group: RE, World: Creation Date: 26-NOV-2002 16:08:50.05 Revision Date: 26-NOV-2002 16:09:09.06, Number: 1 Expiration Date: none specified Backup Date: none posted Contiguity Options: none Performance Options: none Reliability Options: none Journaling Enabled: none RMS FILE ATTRIBUTES RETURN/SPACE=More, PREV/NEXT=Scroll, INS/REM=Pan, SELECT=80/132, Q=Quit

Lexical functions return information to a command line or command procedure. The information returned can be about your process, the system, files and devices, logical names, strings, or data types. Lexical functions are identified by the prefix F$.

You can use lexical functions in any context in which you normally use symbols or expressions. In command procedures, you can use lexical functions to translate logical names, to perform character string manipulations, and to determine the current processing mode of the procedure. Many lexical functions return information that you can also get from DCL commands.

This chapter includes information about:

• How lexical functions work
• Obtaining information about your process
• Obtaining information about the system
• Obtaining information about files and devices
• Translating logical names
• Manipulating strings
• Manipulating data types

For additional information about lexical functions, refer to online help. For more information about the commands discussed in this chapter, refer to the OpenVMS DCL Dictionary.

15.1 Why Use Lexical Functions

You can manipulate information in a command procedure more easily if you obtain it from a lexical function rather than from a command. For example, you can use either the F$ENVIRONMENT function or the SHOW DEFAULT command to obtain the name of your current default directory. The differences are as follows:

• If you use the F$ENVIRONMENT function, you can assign the result to a symbol and then use this symbol later in the procedure. For example:

  $ DIR_NAME = F$ENVIRONMENT("DEFAULT") $ SET DEFAULT DISK4:[TEST] . . . $ SET DEFAULT 'DIR_NAME'

  
  The F$ENVIRONMENT function returns the current default disk and directory and stores this value in the symbol DIR_NAME. At the end of the procedure, you use the symbol DIR_NAME to restore the default with the SET DEFAULT command.
• If you obtain the value of the current default directory by using the SHOW DEFAULT command, instead of the F$ENVIRONMENT lexical function, you cannot assign this output to a symbol directly. Instead, the procedure is as follows:

  $! Redirect the output of the SHOW DEFAULT command to a file. $ DEFINE/SUPERVISOR_MODE SYS$OUTPUT DISK4:[TEST]TEMPFILE.DAT $ SHOW DEFAULT $ DEASSIGN SYS$OUTPUT $! $ OPEN/READ DIR_FILE DISK4:[TEST]TEMPFILE.DAT ! Open the file. $ READ DIR_FILE DIR_NAME, ! Read the file. $ SET DEFAULT 'DIR_NAME' ! Reset the directory. $ CLOSE DIR_FILE ! Close the file. $ DELETE DISK4:[TEST]TEMPFILE.DAT;* ! Delete the file.

You often change process characteristics for the duration of a command procedure and then restore them. You can use the following lexical functions to obtain information about your process:

The following table shows process characteristics that are commonly changed in command procedures. It also gives the lexical functions that save these characteristics and the DCL commands that restore the original settings.

If you save process characteristics, you must ensure that an error or Ctrl/Y interruption does not cause the procedure to exit before you restore the original characteristics. (See Chapter 13 for more information on handling errors and Ctrl/Y interruptions.)

15.2.1 Changing Verification Settings

You can use the F$VERIFY lexical function to disable verification for the duration of a command procedure. This prevents users from displaying a procedure's contents while executing the procedure.

There are two types of verification:

• Procedure verification
  Displays each command line as it is being executed
• Image verification
  Displays each data line as it is being processed

By default, the SET [NO]VERIFY command and the F$VERIFY function turn both types of verification on or off. In general, the procedure and image verification settings in a procedure are the same (both on or both off). However, if you decide to change the settings, save each verification setting separately.

In the following example, the symbol TEMP is used to enable and disable verification:

$ ! Enable verification
$ !
$ TEMP = F$VERIFY(1)
$ LOOP:
$     INQUIRE FILE "File name"
$     IF FILE .EQS."" THEN EXIT
$     PRINT 'FILE'
$     GOTO LOOP
$ ! Disable verification
$ !
$ TEMP = F$VERIFY(0)
$ EXIT

In the following example, the verification settings are saved:

$ ! Save each verification state
$ ! Turn both states off
$ SAVE_VERIFY_IMAGE = F$ENVIRONMENT("VERIFY_IMAGE")
$ SAVE_VERIFY_PROCEDURE = F$VERIFY(0)
   .
   .
   .
$ ! Restore original verification states
$ SAVE_VERIFY_IMAGE = F$VERIFY(SAVE_VERIFY_PROCEDURE,-
      SAVE_VERIFY_IMAGE)

The F$ENVIRONMENT function returns the current image verification setting and assigns this value to the symbol SAVE_VERIFY_IMAGE. Next, the F$VERIFY function returns the current procedure verification setting and assigns this value to the symbol SAVE_VERIFY_PROCEDURE. The F$VERIFY function disables both image and procedure verification. You can use the F$ENVIRONMENT function to obtain the procedure verification setting before you disable verification with F$VERIFY. However, it is shorter to use F$VERIFY to accomplish both tasks in one command line, as shown in the previous example.

At the end of this procedure, the F$VERIFY function restores the original settings (specified by the symbols SAVE_VERIFY_PROCEDURE and SAVE_VERIFY_IMAGE.)