HP OpenVMS Systems Documentation

OpenVMS Programming Concepts Manual

An application that uses internal knowledge of the file system, including knowledge of the contents of a directory and how file header data is structured on a disk cannot work correctly on an ODS-5 volume.

29.5 Upgrading an Application to Support Extended File Specifications

The following sections describe the changes necessary to upgrade the level of support for extended file specifications. Note that you must first ensure that the application meets the default support level before you can upgrade it to the full support level.

To upgrade an application to provide default support for Extended File Specifications, you must ensure that it minimally supports both the ODS-5 volume structure and extended file naming as recommended in naming as recommended in Sections 29.5.1.1 and 29.5.1.2, respectively. Default support is defined in Section 29.4.2.

29.5.1.1 Providing Support for ODS-5

Applications that do not support the new ODS-5 volume structure do not operate successfully on these volumes even if they encounter only traditional file specifications.

If an application does not work properly on an ODS-5 volume, examine the application for the following:

• Does the application use physical or logical I/O to bypass the file system when accessing the volume, or does it access metadata files such as BITMAP.SYS directly? These applications are usually system programs, such as disk defragmenters, or programs that try to avoid overhead by accessing the disk directly. These applications rely on specific knowledge of the file or directory structure on the disk, which has changed with introduction of the ODS-5 structure.
  Recommendation: Applications should use documented interfaces and structures whenever possible.
• Does the application access and interpret the contents of directory files directly? If so, the application may fail when it encounters a directory that contains extended file names.
  Recommendation: Modify the application to use the search functions provided with the RMS² or QIO interface, or with LIBRTL routines such as LIB$FIND_FILE.

If an application does not handle extended names successfully, examine the application for any the following:

• Does the application attempt to parse or assume knowledge of the syntax of a file specification? For example, the application might search for a bracket ([) to locate the beginning of a directory specification, or for a space character to mark the end of a file specification.
  Recommendation: The application should rely on RMS to determine whether a file specification is legal rather than pretesting the actual name. Use the NAM$L_NODE, NAM$L_DEV, NAM$L_DIR, NAM$L_TYPE, and NAM$L_VER fields of the NAM block or SYS$FILESCAN to retrieve this information.
• Does the application attempt to determine if two file names are the same by doing a string comparison? Because file names are case-insensitive, and because there are several ways to represent some characters, a string compare may fail even though two strings represent the same file.
  Recommendation: See the example program [SYSHLP.EXAMPLES]FILENAME_COMPARE.C for a way to use the new system service $CVT_FILENAMES to compare filenames.
• Does the application depend on the NAM$V_DIR_LVLS bits in the NAM$L_FNB field to determine how many directory levels there are in the current file specification? Because there are only three bits in this field, it can only specify a maximum of eight levels. Applications seldom use these bits; they are mainly used by RMS when a NAM is specified as a related file specification.
  Recommendation: With OpenVMS Version 7.2 and greater, there is a new larger field available in both the NAM and the NAML blocks, NAM$W_LONG_DIR_LEVELS. Use this field to locate the correct number of directory levels.
• Does the application rely on the NAM$V_WILD_UFD and SFD1 - SFD7 bits to determine where there are wildcard directories? Because there are only eight of these bits, they can only report wildcards in the first eight directory levels. Applications seldom use these bits; they are mainly used by RMS when a NAM is specified as a related file specification.
  Recommendation: With OpenVMS Version 7.2 and greater, there is a new field available in both the NAM and NAML block, NAML$W_FIRST_WILD_DIR. Use this field to locate the highest directory level where a wildcard is to be found.
• Does the application use the QIO interface to the file system and specify or request a file name from QIO directly? The QIO interface requires that an application specify explicitly that it understands extended file names before it will accept or return the names. In addition, the file name format for extended file names is not identical between RMS and the QIO interface. Additionally, some file names may be specified in 2-byte Unicode (UCS-2) characters. Your application must be capable of dealing with 1 character that spans 2 bytes.
  Recommendations: Most applications that use the QIO interface also use RMS to parse file specifications and retrieve the file and directory ID for the file. They then use these ID values to access the file with the QIO interface. This method of access continues to work with extended names. Compaq recommends changing to this method to fix the problem.
  You can also obtain the name that the QIO system uses from the NAML$L_FILESYS_NAME field of a NAML block, or use the new system service (SYS$CVT_FILENAME) to convert between the RMS and the QIO file name. In this case, you will also need to provide an expanded FIB block to the QIO service to specify that your application understands extended names, expand your buffers to the maximum size, and prepare to deal with 2-byte Unicode characters.

Some OpenVMS applications, such as system or disk management utilities, may require full support for Extended File Specifications. Typically, these are utilities that must be able to view and manipulate all file specifications without DID or FID abbreviation. To upgrade an application so that it fully supports all the features of Extended File Specifications, do the following:

1. Convert all uses of the RMS NAM block to the new NAML block.
2. Expand the input and output file name buffers used by RMS. To do this, use the NAML long_expanded and long_resultant buffer pointers (NAML$L_LONG_EXPAND and NAML$L_LONG_RESULT) rather than the short buffer pointers (NAML$L_ESA and NAML$L_RSA), and increase the buffer sizes from NAM$C_MAXRSS to NAML$C_MAXRSS.
3. If long file names (greater than 255 bytes) are specified in the FAB file name buffer field (FAB$L_FNA), use the NAML long_filename buffer field (NAML$L_LONG_FILENAME) instead. If long file names are specified in the FAB default name buffer field (FAB$L_DNA), use the NAML default name buffer field (NAML$L_LONG_DEFNAME) instead.
4. If you use the LIB$FIND_FILE, LIB$RENAME or LIB$DELETE routines, set LIB$M_FIL_LONG_NAMES in the flags argument (flags is a new argument to the LIB$DELETE routine). Note that you can use the NAML block in place of the NAM block to pass information to LIB$FILE_SCAN without additional changes.
5. If you use the LIB$FID_TO_NAME routine, the descriptor for the returned file specification may need to be changed to take advantage of the increased maximum allowed of 4095 (NAML$C_MAXRSS) bytes.
6. If you use the FDL$CREATE, FDL$GENERATE, FDL$PARSE, or FDL$RELEASE routine, you must set FDL$M_LONG_NAMES in the flags argument.
7. Examine the source code for any additional assumptions made internally that a file specification is no longer than 255 8-bit bytes.

This chapter describes the programming interfaces of the Distributed Transaction Manager (DECdtm). You use these interfaces to implement distributed transactions or when you write resource managers that participate in distributed transactions. Examples of single and multiple branch applications are also presented. Additionally, this chapter describes the implementation of the X/Open Distributed Transaction Processing XA interface. This interface allows DECdtm to coordinate XA-compliant resource managers and XA-compliant transaction processing systems to coordinate resource managers compliant with DECdtm.

DECdtm system services are documented in the OpenVMS System Services Reference Manual.

This chapter contains the following sections:

Section 30.1 provides an overview of the DECdtm programming interfaces.
Section 30.2 describes single branch applications.
Section 30.3 describes multiple branch applications.
Section 30.4 describes default transactions.
Section 30.5 describes the Resource Manager interface.
Section 30.6 describes the Communication Resource Manager interface.
Section 30.7 describes the XA interface (Alpha Only).
Section 30.8 provides program examples that use DECdtm.

DECdtm provides a basic infrastructure for a distributed transaction processing system. A transaction is a collection of operations that change the system from one valid state to another. A transaction performs operations on resources. Examples of resources are databases and files.

Specifically, a transaction has the ACID properties:

A transaction often needs to use more than one resource on one or more system. This type of transaction is called a distributed transaction.

Individual OpenVMS systems within the distributed system are called nodes in this chapter.

The DECdtm model constructs a distributed transaction processing system from three types of component:

• An Application Program (AP) provides the application-specific code for the system and defines the boundaries between transactions.
  A transaction may be implemented by a single AP running in one node of the distributed system, or it may have multiple AP processes. Typically, each process runs on multiple nodes of the system.
• A Resource Manager (RM) provides ACID operations for one or more data resources on a single node of the system. Oracle Rdb and RMS Journaling are examples of resource managers.
  Typically, a distributed transaction involves two or more RMs. This might be dissimilar RMs on a single node of the system (for example, Oracle Rdb and RMS Journaling), or it might be RMs on different nodes.
• The Transaction Manager (TM) controls the interaction of APs and RMs, ensuring that they maintain a common view of the state of each transaction (in-progress, committed, or aborted).
  DECdtm is a TM. Typically, it is the sole TM in an OpenVMS system, but it also provides services that enable it to interoperate with other TMs.

DECdtm implements a two-phase commit protocol. This is a simple consensus protocol that allows a collection of participants to reach a single conclusion. The two-phase commit protocol makes sure that all of the operations can take effect before the transaction is committed. If any operation cannot take effect, for example if a network link is lost, then the transaction is aborted, and none of the operations take effect. Given a list of participants and a designated coordinator, the protocol proceeds as follows:

Note that this protocol reaches a single decision while it allows the coordinator and participants to fail. Any failure during phase 1 causes the transaction to be aborted. If the coordinator fails during phase 2, participants wait for it to recover and read the decision from disk. If a participant fails, it can ask the coordinator for the decision on recovery.

While DECdtm is not complex in itself, construction of a full-function resource manager needs knowledge of more techniques than can be given in this manual. Transaction Processing: Concepts and Techniques by Jim Gray and Andreas Reuter (Morgan Kaufman Publishers, 1993) may be helpful.

30.2 Single Branch Application

A sequence of AP operations that occurs within a single transaction is called a branch of the transaction. In the simplest use of DECdtm, a single AP invokes two or more RMs.

The AP uses just three of the DECdtm services: $START_TRANS, $END_TRANS, and $ABORT_TRANS. These services are documented in the OpenVMS System Services Reference Manual. They have not changed, but additional information is given in this manual.

$START_TRANS initiates a new transaction and returns a transaction identifier (TID) that is passed to other DECdtm services. $END_TRANS ends a transaction by attempting to commit it and returns the outcome of the transaction with either a commit or abort. $ABORT_TRANS ends the transaction by aborting it.

During the transaction, the AP passes the TID to each RM that it uses. The TID may be passed explicitly, or through the default transaction mechanism described in Section 30.4. Internally, each RM calls the DECdtm RM services. It also uses the branch services if parts of the transaction can be executed by different processes or on different nodes.

DECdtm aborts a transaction if the process executing a branch terminates. By default, it also aborts a transaction if the current program image terminates.

30.2.1 Calling DECdtm System Services for a Single Branch Application

An application using the DECdtm system services follows these steps:

1. Calls SYS$START_TRANSW. This starts a new transaction and returns the transaction identifier.
2. Instructs the resource managers to perform the required operations on their resources.
3. Ends the transaction in one of two ways:
   • Commit: To attempt to perform or commit the transaction, the application calls SYS$END_TRANSW. This checks whether all participants can commit their operations. If any participant cannot commit an operation, the transaction is aborted.
     When SYS$END_TRANSW returns, the application determines the outcome of the transaction by reading the completion status in the I/O status block.
   • Abort: To abort the transaction, the application calls SYS$ABORT_TRANSW. Typically, an application aborts a transaction if a resource manager returns an error or if the user enters invalid information during the transaction.

Edward Jessup, an employee of a computer company in Italy, is transferring to a subsidiary of the company in Japan. An application must remove his personal information from an Italian DBMS database and add it to a Japanese Rdb database. Both of these operations must happen, otherwise Edward's personal information may either end up cyber space (the application might remove him from the Italian database but then lose a network link while trying to add him to the Japanese database) or find that he is in both databases at the same time. Either way, the two databases would be out of step.

If the application used DECdtm to execute both operations as an atomic transaction, then this error could never happen; DECdtm would automatically detect the network link failure and abort the transaction. Neither of the databases would be updated, and the application could then try again.

Figure 30-1 shows the participants in the distributed transaction discussed in this sample transaction. The application is on node ITALY.

Figure 30-1 Participants in a Distributed Transaction

A transaction may have multiple branches. A separate branch is required for each process that takes part in a transaction, regardless of whether the processes run on the same node or on different nodes of the system.

The top branch of the transaction is created by $START_TRANS. A new branch can be requested in the following ways:

• By making explicit use of the $ADD_BRANCH and $START_BRANCH services. The application can use any suitable communication technique to pass application calls between the processes and nodes of the system. Such communication is not a function of DECdtm.
• By calling an RM such as Oracle Rdb that allows resource processing to be requested on another node of the system.
• By calling a transaction processing framework such as ACMS that allows processing tasks to be requested on other nodes of the system.

Note that in the last two cases, the RM or TP framework make the necessary branch service calls on behalf of the application. There is no difference in the three cases from the viewpoint of DECdtm.

The top branch of a transaction is created by calling $START_TRANS. A subordinate branch is authorized when an existing branch calls $ADD_BRANCH. This returns a globally unique branch identifier (BID). The application passes the BID and TID with an application-specific request to another process or node of the system. $START_BRANCH is then called on the target node to add a new branch to the transaction. A subordinate branch of a transaction may in turn create further branches.

DECdtm can connect the two parts of the transaction together because $ADD_BRANCH specifies the name of the target node while $START_BRANCH specifies the name of the parent node. Either the two nodes must be in the same OpenVMS Cluster or they must be able to communicate by DECnet. DECdtm operation is more efficient within an OpenVMS Cluster.

Unless DECdtm operation is confined to a single cluster, you must configure each node with the same DECnet node name as its cluster node name.

An application may complete its processing within a branch by calling $END_BRANCH.

On $START_BRANCH, DECdtm checks that the two nodes are able to communicate, but it does not validate that the branch is authorized until $END_BRANCH is called. At that point, an unauthorized branch is aborted without affecting the ability of the authorized branches to commit.

Be careful in situations in which an application attempts to access the same resource from different branches of a transaction. Some RMs can recognize that the branches form part of the same transaction and allow concurrent access to the resource. In that case, just like multiple threads in a process, the application may need to serialize its own operations on the shared resource. Other RMs may lock one branch against another. In that case, the application is likely to deadlock.

Multiple branches in a transaction can serialize their operations on a shared resource within an OpenVMS Cluster using the Lock Manager. Care is needed if two branches outside an OpenVMS Cluster implicitly share a resource, perhaps by each creating a subordinate branch on a third system.

A single process may have multiple branches. For example, a server process may execute parallel operations on behalf of different transactions.

30.3.1 Resource Manager Use of the Branch Services

Strictly defined, an RM provides access to resources on the same process as an AP that has started a transaction or added a branch. However an RM may perform work for a transaction in a different process to the original request. In that case, it must use the branch services to join the transaction in the worker process.

Similarly, an RM such as Oracle Rdb may provide an application interface that allows remote resources to be accessed. In that case, the RM uses the branch services to add a branch on the local node and start a branch on the remote node.

30.3.2 Branch Synchronization

Processing in all branches of a transaction must be complete before calling $END_TRANS.

Normally DECdtm is used to ensure branch completion. In this case:

• The call to $START_BRANCH does not specify the DDTM$M_BRANCH_UNSYNCHED flag.
• Either $END_BRANCH or $ABORT_TRANS must be called to end the branch.
• $END_BRANCH and $END_TRANS calls are not completed with a success status until all synchronized subordinate branches of the transaction have initiated calls to $END_BRANCH and the top branch has initiated a call to $END_TRANS.
• $END_TRANS and $END_BRANCH are not completed with an SS$_ABORT status until all synchronized branches on the local node have initiated calls to $END_TRANS, $END_BRANCH, or $ABORT_TRANS.

In other words, when a transaction completes successfully, all synchronized branches complete together. When a transaction aborts, all synchronized branches on a single node complete together, but branches on different nodes complete at different times. Using synchronized branches does not add extra message overhead, because the synchronization events are implicit in the normal DECdtm commitment protocol.

DECdtm branch synchronization is redundant when branch processing is initiated by a synchronous call to a process or remote node, and that call does not return until processing is complete. For example, remote operations may be requested by Remote Procedure Call (RPC). In this case:

• The call to $START_BRANCH specifies the DDTM$M_BRANCH_UNSYNCHED flag.
• The branch must not call $END_BRANCH or $ABORT_TRANS. If the transaction is to be aborted, the branch must return an error status to its superior branch.

See Section 30.4 for a case in which unsynchronized branches are not advised.