HP OpenVMS Systems Documentation

• Loading the watchpoint driver
• Creating and deleting watchpoints
• Looking at watchpoint driver data
• Acquiring collected watchpoint data
• Looking at the protection attributes and access fault mechanism
• Looking at some watchpoint restrictions

The Alpha Watchpoint utility (WP) enables you to monitor write access to user-specified locations. The chapter contains the following sections:

Section 13.1 presents an introduction of the Watchpoint utility.

Section 13.2 describes how to load the watchpoint driver.

Section 13.3 describes the creation and deletion of watchpoints and the constraints upon watchpoint locations.

Section 13.4 contains detailed descriptions of the watchpoint driver data structures, which you might need to know to analyze collected watchpoint data.

Section 13.5 discusses acquiring collected watchpoint data.

Section 13.6 describes the watchpoint protection facility.

Section 13.7 describes the utility's restrictions.

13.1 Introduction

A watchpoint is a data field to which write access is monitored. The field is from 1 to 8 bytes long and must be contained within a single page. Typically, watchpoints are in nonpaged pool. However, subject to certain constraints (see Section 13.3.1), they can be defined in other areas of system space. The Watchpoint facility can simultaneously monitor a large number (50 or more) watchpoints.

The utility is implemented in the WPDRIVER device driver and the utility program WP. This document concentrates on the device driver, which can be invoked directly or through the WP utility.

For information on the WP utility, see its help files, which can be displayed with the following DCL command:

           $ HELP/LIBRARY=SYS$HELP:WP 

Once the driver has been loaded, a suitably privileged user can designate a watchpoint in system space. Any write to a location designated as a watchpoint is trapped. Information is recorded about the write, including its time, the register contents, and the program counter (PC) and processor status longword (PSL) of the writing instruction. Optionally, one or both of the following user-specified actions can be taken:

• An XDELTA breakpoint¹ or SCD breakpoint which occurs just after the write to the watchpoint
• A fatal watchpoint bugcheck which occurs just after the write to the watchpoint

You define a watchpoint by issuing QIO requests to the watchpoint driver; entering commands to the WP utility, which issues requests to the driver; or, from kernel mode code, invoking a routine within the watchpoint driver.

The WPDRIVER data structures store information about writes to a watchpoint. This information can be obtained either through QIO requests to the WPDRIVER, commands to the WP utility, XDELTA commands issued during a requested breakpoint, or SDA commands issued during the analysis of a requested crashdump.

13.2 Initializing the Watchpoint Utility

From a process with CMKRNL privilege, run the SYSMAN utility to load the watchpoint driver, SYS$WPDRIVER.EXE. Enter the following commands:

$ RUN SYS$SYSTEM:SYSMAN
SYSMAN> IO CONNECT WPA0:/NOADAPTER/DRIVER=SYS$WPDRIVER
SYSMAN> EXIT

SYSMAN creates system I/O data structures for the pseudo-device WPA0, loads WPDRIVER, and invokes its initialization routines. WPDRIVER initialization includes the following actions:

• Allocating nonpaged pool and physical memory for WPDRIVER data structures
• Appropriating the SCB vector specific to access violations
• Recording in system space the addresses of the WPDRIVER routines invoked by kernel mode code to create and delete watchpoints

Memory requirements for WPDRIVER and its data structures are:

• Device driver and UCB---approximately 3K bytes of nonpaged pool
• Trace table and a related array---36 bytes for each of system parameter WPTTE_SIZE trace table entries
• Watchpoint restore entries---system parameter WPRE_SIZE pages of physically contiguous memory
• Each watchpoint---176 bytes of nonpaged pool

It is advisable to load the watchpoint driver relatively soon after system initialization to ensure its allocation of physically contiguous memory. If the driver cannot allocate enough physically contiguous memory, it does not set WPA0: online. If the unit is offline, you will not be able to use the watchpoint utility.

13.3 Creating and Deleting Watchpoints

There are three different ways to create and delete watchpoints:

• An image can assign a channel to device WPA0: and then request the Queue I/O Request ($QIO) system service to create or delete a watchpoint.
• Code running in kernel mode can dispatch directly to routines within the WPDRIVER to create and delete watchpoints.
• You can enter commands to the WP utility.

The first two methods are described in detail in the sections that follow.

13.3.1 Using the $QIO Interface

An image first assigns a channel to the pseudo-device WPA0: and then issues a $QIO request on that channel. The process must have the privilege PHY_IO; otherwise, the $QIO request is rejected with the error SS$_NOPRIV.

Table 13-1 shows the functions that the driver supports.

The IO$_ACCESS function requires the following device/function dependent arguments:

• P2---Length of the watchpoint. A number larger than 8 is reduced to 8.
• P3---Starting address of the watchpoint area.

The following are the constraints on the watchpoint area. It must be:

• Nonpageable system space.
• Write-accessible from kernel mode.
• Within one page. If it is not, the requested length is reduced to what will fit within the page containing the starting address.
• Within a page accessed only from kernel mode and by instructions that incur no pagefaults.
• Within a page whose protection is not altered while the watchpoint is in place.
• Outside of certain address ranges. These are the WPDRIVER code, its data structures, and the system page table.

Because of the current behavior of the driver, there is an additional requirement that there be no "unexpected" access violations referencing a page containing a watchpoint. See Section 13.7 for further details.

To specify that an XDELTA breakpoint or a fatal bugcheck occur if the watchpoint is written, use the following I/O function code modifiers:

• IO$M_CTRL to request an XDELTA breakpoint
• IO$M_ABORT to request a fatal bugcheck

For an XDELTA breakpoint to be taken, OpenVMS must have been booted specifying that XDELTA and/or the SCD be resident (bit 1 or bit 15 in the boot flags must be set). If both watchpoint options are requested, the XDELTA breakpoint is taken first. At exit from the breakpoint, the driver crashes the system.

A request to create a watchpoint can succeed completely, succeed partially, or fail. Table 13-2 shows the status codes that can be returned in the I/O status block.

The following example MACRO program assigns a channel to the WPA0 device and creates a watchpoint of 4 bytes, at starting address 80001068. The program requests neither an XDELTA breakpoint nor a system crash for that watchpoint.

 
                  $IODEF 
                  .PSECT   RWDATA,NOEXE,RD,WRT,LONG 
                                          ; 
        WP_IOSB:  .BLKL    2              ; I/O status block. 
        WP_ADDR:  .LONG    ^X80001068     ; Address of watchpoint to create. 
        WP_NAM:   .ASCID   /WPA0:/        ; Device to which to assign channel. 
        WP_CHAN:  .BLKW    1              ; Channel number. 
                  .PSECT   PROG,EXE,NOWRT 
                                          ; 
        START:    .CALL_ENTRY 
 
                   $ASSIGN_S DEVNAM=WP_NAM,CHAN=WP_CHAN 
                   BLBC    R0,RETURN 
 
                   $QIOW_S CHAN=WP_CHAN,- 
                           FUNC=#IO$_ACCESS,- 
                           IOSB=WP_IOSB,- 
                           P2=#4,- 
                           P3=WP_ADDR 
                   BLBC    R0,RETURN                   
                   MOVL    WP_IOSB,R0     ; Move status to R0. 
        RETURN:    RET                    ; Return to caller. 
                  .END     START 

A watchpoint remains in effect until it is explicitly deleted. (Note, however, that watchpoint definitions do not persist across system reboots.) To delete an existing watchpoint, issue an IO$_DEACCESS QIO request.

The IO$_DEACCESS function requires the following device/function dependent argument: P3 - Starting address of the watchpoint to be deleted.

Table 13-3 shows the status values that are returned in the I/O status block.

Section 13.5 describes the use of the IO$_RDSTATS QIO request.

13.3.2 Invoking WPDRIVER Entry Points from System Routines

When the WPDRIVER is loaded, it initializes two locations in system space with the addresses of routines within the driver. These locations, WP$CREATE_WATCHPOINT and WP$DELETE_WATCHPOINT, enable dispatch to create and delete watchpoint routines within the loaded driver. Input arguments for both routines are passed in registers.

Code running in kernel mode can execute the following instructions:

                  JSB    @G^WP$CREATE_WATCHPOINT ; create a watchpoint 

and

                 JSB    @G^WP$DELETE_WATCHPOINT ; delete a watchpoint 

Both these routines save IPL at entry and set it to the fork IPL of the WPDRIVER, IPL 11. Thus, they should not be invoked by code threads running above IPL 11. At exit, the routines restore the entry IPL.

These two locations contain an RSB instruction prior to the loading of the driver. As a result, if a system routine tries to create or delete a watchpoint before the WPDRIVER is loaded, control immediately returns.

WP$CREATE_WATCHPOINT has the following register arguments:

• R0---User-specified watchpoint options
  • Bit 1 equal to 1 specifies that a fatal OPERCRASH bugcheck should occur after a write to the watchpoint area.
  • Bit 2 equal to 1 specifies that an XDELTA breakpoint should occur after a write to the watchpoint area.
• R1---Length of the watchpoint area
• R2---Starting address of the watchpoint area

Status is returned in R0. The status values and their interpretations are identical to those for the QIO interface to create a watchpoint. The only difference is that the SS$_NOPRIV status cannot be returned with this interface.

WPS$DELETE_WATCHPOINT has the following register argument:

• R2---Starting address of the watchpoint area

Status is returned in R0. The status values and their interpretations are identical to those for the QIO interface.

13.4 Data Structures

The WPDRIVER uses three different kinds of data structures:

• One watchpoint restore entry (WPRE) for each page of system space in which one or more active watchpoints are located
• One watchpoint control block (WPCB) for each active watchpoint
• Trace table entries (WPTTEs) in a circular trace buffer which maintains a history of watchpoint writes

These data structures are described in detail and illustrated in the sections that follow.

13.4.1 Watchpoint Restore Entry (WPRE)

There is one WPRE for each system page that contains a watchpoint. That is, if nine watchpoints are defined which are in four different system pages, four WPREs are required to describe those pages. When WPDRIVER is loaded, its initialization routine allocates physically contiguous memory for the maximum number of WPREs. The number of pages to be allocated is specified by system parameter WPRE_SIZE.

The WPDRIVER allocates WPREs starting at the beginning of the table and maintains a tightly packed list. That is, when a WPRE in the middle of those in use is "deallocated," its current contents are replaced with the contents of the last WPRE in use. The number in use at any given time is in the driver variable WP$L_WP_COUNT. The system global EXE$GA_WP_WPRE points to the beginning of the WPRE table.

The WPRE for a page contains information useful for:

• Determining whether a given access violation refers to an address in the page associated with this WPRE
• Restoring the original SPTE value for the associated page
• Reestablishing the modified SPTE value when watchpoints are reenabled
• Invalidating the translation buffer when the SPTE is modified
• Locating the data structures associated with individual watchpoints defined in this system page

The WPCBs associated with a given system page are singly-linked to a list header in the associated WPRE. A WPCB is allocated from a nonpaged pool when a watchpoint is created. A WPCB contains static information about the watchpoint such as the following:

• Its starting address and length
• Original contents of the watchpoint at the time it was established
• User-specified options for this watchpoint

In addition, the WPCB contains dynamic data associated with the most recent write reference to the watchpoint. This data includes the following:

• Number of times that the watchpoint has been written.
• Address of the first byte within the watchpoint that was modified at the last write reference.
• PC-PSL pair that made the last write reference.
• System time at the last write reference.
• Contents of the general registers at the time of the last write reference.
• A copy of up to 15 bytes of instruction stream data beginning at the program counter (PC) of the instruction that made the last write reference. The amount of instruction stream data that is copied here is the lesser of 15 bytes and the remaining bytes on the page containing the PC.
• Contents of the watchpoint before the last write reference.
• Contents of the watchpoint after the last write reference. This value is presumably the current contents of the watchpoint.
• A pointer to an entry in the global circular trace buffer where all recent references to watchpoints are traced.

Whenever a watchpoint is written, all the relevant data is recorded in the WPCB associated with the watchpoint. In addition, to maintain a history, the WPDRIVER copies a subset of the data to the oldest WPTTE in the circular trace buffer. Thus, the circular trace buffer contains a history of the last N references to watchpoints. The driver allocates nonpaged pool to accommodate the number of trace table entries specified by the system parameter WPTTE_SIZE. The WPTTEs for all watchpoints are together in the table, but the ones for a particular watchpoint are chained together.

The subset of data in a WPTTE includes the following:

• Starting address of the watchpoint
• Relative offset of the first byte modified on this reference
• Opcode of the instruction that modified the watchpoint
• A relative backpointer to the previous WPTTE of this watchpoint
• PC-PSL of the write reference
• System time of the write reference
• Contents of the watchpoint before this reference

Analyzing watchpoint results is a function of the mode in which the WPDRIVER is used. For example, if you have only one watchpoint and have specified that an XDELTA breakpoint and/or a bugcheck occur on a write to the watchpoint, then when the reference occurs, simply find the program counter (PC) that caused the reference.

This PC (actually the PC of the next instruction) and its processor status longword (PSL) are on the stack at the time of the breakpoint and/or bugcheck. The layout that follows is the stack as it appears within an XDELTA breakpoint. Examined from a crash dump, the stack is similar but does not contain the return address from the JSB to INI$BRK.

 
        +--------------------------------------+ 
        |address in WPDRIVER from JSB G^INI$BRK| :SP 
        |PC of next instruction                | 
        |PSL at watchpoint access              | 
        +--------------------------------------+ 

Furthermore, R0 contains the address of the WPCB associated with that watchpoint. You can examine the WPCB to determine the original contents of the watchpoint area and the registers at the time of the write.

Definitions for the watchpoint data structures are in SYS$LIBRARY:LIB.MLB. Build an object module with its symbol definitions by entering the following DCL commands:

$ MACRO/OBJ=SYS$LOGIN:WPDEFS SYS$INPUT: + SYS$LIBRARY:LIB/LIB 
        $WPCBDEF   GLOBAL   !n.b. GLOBAL must be capitalized 
        $WPREDEF   GLOBAL 
        $WPTTEDEF  GLOBAL 
        .END 
CTRL/Z 

Then, within SDA, you can format watchpoint data structures. For example, enter the following SDA commands:

SDA>READ SYS$LOGIN:WPDEFS.OBJ
SDA>FORMAT @R0 /TYPE=WPCB   !type definition is required
SDA>DEF WPTTE = @R0 + WPCB$L_TTE
SDA>FORMAT WPTTE /TYPE=WPTTE

An alternative to crashing the system or using XDELTA to get watchpoint information is the QIO function IO$_RDSTAT. This function returns watchpoint control block contents and trace table entries for a particular watchpoint.

It requires the following device/function dependent arguments:

• P1---Address of buffer to receive watchpoint data.
• P2---Length of the buffer. The minimum size buffer of 188 bytes is only large enough for WPCB contents.
• P3---Watchpoint address.

The data returned in the buffer has the format shown in Figure 13-1.

Figure 13-1 Format of Data Returned in Buffer

The overall design of the watchpoint facility uses protection attributes on system pages and the access violation fault mechanism. To establish a watchpoint within a page of system space, the WPDRIVER changes the protection of the page to disallow writes. The WPDRIVER modifies the access violation vector to point to its own routine, WP$ACCVIO.

Any subsequent write to this page causes an access violation and dispatch to WP$ACCVIO. Thus, the WPDRIVER gains control on all write references to watchpoints and can monitor such accesses.

When WP$ACCVIO is entered, it raises IPL to 31 to block all other threads of execution. It first must determine whether the faulting address (whose reference caused the access violation) is within a page containing a watchpoint. However, any major amount of CPU processing at this point might access an area in system space whose protection has been altered to establish watchpoints. As a result, such processing might cause a reentry into WP$ACCVIO. To avoid recursive reentry, WP$ACCVIO first restores all SPTEs that it had modified to their values prior to the establishment of any watchpoints. From this point until this set of SPTEs are remodified, no watchpoints are in effect. Now WP$ACCVIO can determine whether the reference was to a page containing a watchpoint.

To determine whether the reference is to a watchpoint page, WP$ACCVIO compares the faulting address to addresses of pages whose protection has been altered by WPDRIVER. If the faulting address is not in one of these pages, then WP$ACCVIO passes the access violation to the usual OpenVMS service routine, EXE$ACVIOLAT. If the faulting address is within a page containing a watchpoint, more extensive processing is required.

As a temporary measure, WP$ACCVIO first records all data related to the reference in its UCB. It cannot immediately associate the access violation with a particular watchpoint. This ambiguity arises from imprecision in the faulting virtual address recorded at the access violation. The CPU need merely place on the stack "some virtual address in the faulting page."

As a result, when a reference to a page with a watchpoint results in an access violation, the watchpoint driver first merely captures the data in its UCB. The data captured at this point includes the following:

• PC and PSL of the faulting instruction
• Current system time
• Values of all the general registers from R0 through SP
• A copy of up to 15 bytes of the instruction stream, beginning at the PC previously captured

If the reference later turns out not to be one to a watchpoint, the captured data is discarded. If the reference is to a watchpoint, the data is copied to the WPCB and circular trace buffer.

The watchpoint driver distinguishes between these two possibilities by reexecuting the faulting instruction under a controlled set of circumstances.

Once the instruction has reexecuted, WP$TBIT can determine whether watchpoint data has been modified by comparing the current contents of all watchpoints within the page of interest to the contents that they had prior to this reference. Because the driver has run at IPL 31 since the write access that caused an access violation, any change in the contents is attributable to the reexecuted instruction. If the contents of a watchpoint are different, WP$TBIT copies the data temporarily saved in its UCB to the WPCB associated with this watchpoint and records a subset of this data in a WPTTE.

The driver can cause either or both an XDELTA breakpoint or a bugcheck, depending on what action was requested with the watchpoint definition. If an XDELTA breakpoint was requested, the driver invokes XDELTA. After the user proceeds from the XDELTA breakpoint, if a bugcheck was not requested, the driver restores the SPTEs of pages containing watchpoints, the saved registers and IPL, and REIs to dismiss the exception.

13.7 Restrictions

The WPDRIVER can monitor only those write references to system space addresses that arise in a CPU. I/O devices can write to memory and thereby modify watchpoints without the WPDRIVER's becoming aware of the write.

Because a write access to a watchpoint is determined by comparing the contents of the watchpoint before and after the write, a write of data identical to the original contents is undetectable.

Because the WPDRIVER modifies SPTEs, a device page that directly interprets tables may experience access violations when it attempts to write into a memory page whose protection has been modified to monitor watchpoints. In other words, a page containing a watchpoint should not also contain a buffer for such a controller.

When you create a watchpoint, you should ensure that the system is quiet with respect to activity affecting the watchpoint area. Otherwise, an inconsistent copy of the original contents of the watchpoint area may be saved. WPDRIVER raises IPL to 11 to copy the watchpoint area's original contents. This means that if the area is modified from a thread of execution running as the result of an interrupt above 11, WPDRIVER can copy inconsistent contents. An inconsistent copy of the original contents may result in spuriously detected writes and missed writes.

If the page containing the watchpoint area is written by an instruction that incurs a page fault, the system can crash with a fatal PGFIPLHI bugcheck. As described in the previous section, after detecting an attempt to write to a page with a watchpoint, the WPDRIVER re-executes the writing instruction at IPL 31. Page faults at IPL 31 are not allowed.

If an outer access mode reference to a watchpointed page causes an access violation, the system will likely crash. When an access violation occurs on a page with a watchpoint, the current driver does not probe the intended access and faulting mode against the page's original protection code. Instead, it assumes that any access violation to that page represents a kernel mode instruction that can be reexecuted at IPL 31. The driver's subsequent attempt to REI, restoring a program status longword (PSL) with an outer mode and IPL 31, causes a reserved operand fault and, generally, a fatal INVEXCEPTN bugcheck.

You must be knowledgeable about the accesses to the page with the watchpoint and careful in using the driver. You should test the watchpoint creation on a standalone system. You should leave the watchpoint in effect long enough to have some confidence that pagefaults in instructions accessing that page are unlikely.

An attempt to CONNECT a WPA unit other than zero results in a fatal WPDRVRERR bugcheck.

The WPDRIVER is suitable for use only on a single CPU system. That is, it should not be used on a symmetric multiprocessing system. There are no plans to remove this restriction in the near future.