HP OpenVMS Systems Documentation

Configuration 4 offers most of the individual component advantages of configuration 3, plus:

• Physical damage to one StorageWorks cabinet is not likely to disable the cluster.
• Disk failover to another HSJ is not a problem.

Configuration 4 has the following disadvantages:

• Does not have redundant paths to each disk.
• Write I/Os to shadow sets use twice the CI bandwidth of other alternatives.
• Higher cost than configuration 2 and configuration 3.

Configuration 4 (Figure 9-4) provides all of the strategies of configuration 3. It also provides shadow set members that are in physically separate StorageWorks cabinets.

9.7 Summary

All four configurations illustrate how to obtain both availability and performance by:

• Adding redundant components
• Dual-porting critical disks
• Shadowing essential disks

An advanced technique, separating the CI path A and path B cables and associated hubs, is used in configuration 3 and configuration 4. This technique increases availability and maintains performance with no additional hardware. Configuration 4 provides even greater availability without compromising performance by physically separating shadow set members and their HSJ controllers.

Using these configurations as a guide, you can select the techniques that are appropriate for your computing needs and adapt your environment as conditions change. The techniques illustrated in these configurations can be scaled for larger CI configurations.

This chapter explains how to maximize scalability in many different kinds of OpenVMS Clusters.

10.1 What Is Scalability?

Scalability is the ability to expand an OpenVMS Cluster system in any system, storage, and interconnect dimension and at the same time fully use the initial configuration equipment. Your OpenVMS Cluster system can grow in many dimensions, as shown in Figure 10-1. Each dimension also enables your applications to expand.

Figure 10-1 OpenVMS Cluster Growth Dimensions

Table 10-1 describes the growth dimensions for systems, storage, and interconnects in OpenVMS Clusters.

The ability to add to the components listed in Table 10-1 in any way that you choose is an important feature that OpenVMS Clusters provide. You can add hardware and software in a wide variety of combinations by carefully following the suggestions and guidelines offered in this chapter and in the products' documentation. When you choose to expand your OpenVMS Cluster in a specific dimension, be aware of the advantages and tradeoffs with regard to the other dimensions. Table 10-2 describes strategies that promote OpenVMS Cluster scalability. Understanding these scalability strategies can help you maintain a higher level of performance and availability as your OpenVMS Cluster grows.

10.2 Strategies for Configuring a Highly Scalable OpenVMS Cluster

The hardware that you choose and the way that you configure it has a significant impact on the scalability of your OpenVMS Cluster. This section presents strategies for designing an OpenVMS Cluster configuration that promotes scalability.

10.2.1 Scalability Strategies

Table 10-2 lists strategies in order of importance that ensure scalability. This chapter contains many figures that show how these strategies are implemented.

Each CI star coupler can have up to 32 nodes attached; 16 can be systems and the rest can be storage controllers and storage. Figure 10-2, Figure 10-3, and Figure 10-4 show a progression from a two-node CI OpenVMS Cluster to a seven-node CI OpenVMS Cluster.

10.3.1 Two-Node CI OpenVMS Cluster

In Figure 10-2, two nodes have shared, direct access to storage that includes a quorum disk. The VAX and Alpha systems each have their own system disks.

Figure 10-2 Two-Node CI OpenVMS Cluster

The advantages and disadvantages of the configuration shown in Figure 10-2 include:

Advantages

• All nodes have shared, direct access to all storage.
• As the nodes and storage in this configuration grow, all nodes can still have shared, direct access to storage.
• The MSCP server is enabled for failover to the LAN interconnect in case the CI fails. Enabling the MSCP server also allows you to add satellites.
• This configuration has the lowest cost of the CI configurations shown in this section.

Disadvantage

• The single HSJ/HSC is a potential bottleneck and single point of failure.

An increased need for more storage or processing resources could lead to an OpenVMS Cluster configuration like the one shown in Figure 10-3.

10.3.2 Three-Node CI OpenVMS Cluster

In Figure 10-3, three nodes are connected to two HSC controllers by the CI interconnects. The critical system disk is dual ported and shadowed.

Figure 10-3 Three-Node CI OpenVMS Cluster

The advantages and disadvantages of the configuration shown in Figure 10-3 include:

Advantages

• All nodes have shared, direct access to all storage.
• As the nodes and storage in this configuration grow, all nodes can still have shared, direct access to storage.
• The MSCP server is enabled for failover to the LAN interconnect, in case the CI fails. Enabling the MSCP server also allows you to add satellites.
• Volume shadowed, dual-ported disks increase data availability.

Disadvantage

• The HSJs/HSCs are potential bottlenecks.

If the I/O activity exceeds the capacity of the CI interconnect, this could lead to an OpenVMS Cluster configuration like the one shown in Figure 10-4.

10.3.3 Seven-Node CI OpenVMS Cluster

In Figure 10-4, seven nodes each have a direct connection to two star couplers and to all storage.

Figure 10-4 Seven-Node CI OpenVMS Cluster

The advantages and disadvantages of the configuration shown in Figure 10-4 include:

Advantages

• All nodes have shared, direct access to all storage.
• This configuration has more than double the storage, processing, and CI interconnect capacity than a configuration like the one shown in Figure 10-3.
• Two CI interconnects between processors and storage provide twice the communication performance of one path.
• Volume shadowed, dual-ported disks increase data availability.

Disadvantage

• This configuration is complex and requires experienced personnel to configure, tune, and manage it properly.

The following guidelines can help you configure your CI OpenVMS Cluster:

• Every system should have shared, direct access to all storage.
• In a CI OpenVMS Cluster larger than four nodes, use a second system disk for increased system performance.
  Reference: For more information on system disks, see Section 11.2.
• Enable your systems, interconnects, and storage to work to their full capacity by eliminating bottlenecks. If any of these components is not able to handle the I/O capacity, none of the other components will work at its best. Ensure that the sum of all the I/O on your nodes is less than or equal to your CI capacity and to your storage capacity. In calculating the sum of the I/O on your nodes, factor in 5--10% extra for lock manager internode communications.
  In general, use the following rules of thumb:
  • The sum of all the I/Os on your nodes plus internode communications should be less than or equal to the sum of your CI capacity.
  • Your CI capacity should be less than or equal to the sum of your storage capacity.

Volume shadowing is intended to enhance availability, not performance. However, the following volume shadowing strategies enable you to utilize availability features while also maximizing I/O capacity. These examples show CI configurations, but they apply to DSSI and SCSI configurations, as well.

Figure 10-5 Volume Shadowing on a Single Controller

Figure 10-5 shows two nodes connected to an HSJ, with a two-member shadow set.

The disadvantage of this strategy is that the controller is a single point of failure. The configuration in Figure 10-6 shows examples of shadowing across controllers, which prevents one controller from being a single point of failure. Shadowing across HSJ and HSC controllers provides optimal scalability and availability within an OpenVMS Cluster system.

Figure 10-6 Volume Shadowing Across Controllers

As Figure 10-6 shows, shadowing across controllers has three variations:

• Strategy A shows each volume in the shadow set attached to separate controllers. This configuration is not optimal because each volume is not attached to each controller.
• Strategy B shows dual-ported devices that have two paths to the volumes through separate controllers. This strategy is an optimal variation because two HSC controllers have direct access to a single storage device.
• Strategy C shows HSJ controllers shadowed across SCSI buses. It also is an optimal variation because two HSJ controllers have direct access to a single storage device.

Figure 10-7 shows an example of shadowing across nodes.

Figure 10-7 Volume Shadowing Across Nodes

As Figure 10-7 shows, shadowing across nodes provides the advantage of flexibility in distance. However, it requires MSCP server overhead for write I/Os. In addition, the failure of one of the nodes and its subsequent return to the OpenVMS Cluster will cause a copy operation.

If you have multiple volumes, shadowing inside a controller and shadowing across controllers are more effective than shadowing across nodes.

Reference: See HP Volume Shadowing for OpenVMS for more information.

10.4 Scalability in DSSI OpenVMS Clusters

Each DSSI interconnect can have up to eight nodes attached; four can be systems and the rest can be storage devices. Figure 10-8, Figure 10-9, and Figure 10-10 show a progression from a two-node DSSI OpenVMS Cluster to a four-node DSSI OpenVMS Cluster.

10.4.1 Two-Node DSSI OpenVMS Cluster

In Figure 10-8, two nodes are connected to four disks by a common DSSI interconnect.

Figure 10-8 Two-Node DSSI OpenVMS Cluster

The advantages and disadvantages of the configuration shown in Figure 10-8 include:

Advantages

• Both nodes have shared, direct access to all storage.
• The Ethernet LAN ensures failover capability if the DSSI interconnect fails.

Disadvantages

• The amount of storage that is directly accessible to all nodes is limited.
• A single DSSI interconnect can become a single point of failure.

If the OpenVMS Cluster in Figure 10-8 required more processing power, more storage, and better redundancy, this could lead to a configuration like the one shown in Figure 10-9.

10.4.2 Four-Node DSSI OpenVMS Cluster with Shared Access

In Figure 10-9, four nodes have shared, direct access to eight disks through two DSSI interconnects. Two of the disks are shadowed across DSSI interconnects.

Figure 10-9 Four-Node DSSI OpenVMS Cluster with Shared Access

The advantages and disadvantages of the configuration shown in Figure 10-9 include:

Advantages

• All nodes have shared, direct access to all storage.
• The Ethernet LAN ensures failover capability if the DSSI interconnect fails.
• Shadowing across DSSI interconnects provides increased performance and availability.

Disadvantage

• The amount of storage that is directly accessible to all nodes is limited.

If the configuration in Figure 10-9 required more storage, this could lead to a configuration like the one shown in Figure 10-10.

10.4.3 Four-Node DSSI OpenVMS Cluster with Some Nonshared Access

Figure 10-10 shows an OpenVMS Cluster with 4 nodes and 10 disks. This model differs from Figure 10-8 and Figure 10-9 in that some of the nodes do not have shared, direct access to some of the disks, thus requiring these disks to MSCP served. For the best performance, place your highest-priority data on disks that are directly connected by common DSSI interconnects to your nodes. Volume shadowing across common DSSI interconnects provides the highest availability and may increase read performance.

Figure 10-10 DSSI OpenVMS Cluster with 10 Disks

The advantages and disadvantages of the configuration shown in Figure 10-10 include:

Advantages

• All nodes have shared, direct access to most of the storage.
• The MSCP server is enabled to allow failover to the alternate DSSI interconnect if one of the DSSI interconnects fails.
• Shadowing across DSSI interconnects provides increased performance and availability.
• The SCSI storage connected through the HSZ controllers provides good performance and scalability.

Disadvantages

• The amount of storage that is directly accessible to all nodes is limited.
• Shadow set 2 requires MSCP serving to coordinate shadowing activity.
• Some nodes do not have direct access to storage. For example, Alpha 2 and Alpha 4 do not have direct access to disks connected to Alpha 1 and Alpha 3.

Each MEMORY CHANNEL (MC) interconnect can have up to four nodes attached to each MEMORY CHANNEL hub. For two-hub configurations, each node must have two PCI adapters, and each adapter must be attached to a different hub. In a two-node configuration, no hub is required because one of the PCI adapters serves as a virtual hub.

Figure 10-11, Figure 10-12, and Figure 10-13 show a progression from a two-node MEMORY CHANNEL cluster to a four-node MEMORY CHANNEL cluster.

Reference: For additional configuration information and a more detailed technical summary of how MEMORY CHANNEL works, see Appendix B.