Planning the Network

This section describes the available network protocols for replication in VVR. It also explains how bandwidth requirement depends on the mode of replication---synchronous or asynchronous.

Choosing the Network Bandwidth

Bandwidth of the Available Network Connection

The type of connection determines the maximum bandwidth available between the two locations, for example, a T3 line provides 45 megabits/second. However, the important factor to consider is whether the available connection is to be used by any other applications or is exclusively reserved for replicating to a single Secondary. If other applications are using the same line, it is important to be aware of the bandwidth requirements of these applications and subtract them from the total network bandwidth. If any applications sharing the line have variations in their usage pattern, it is also necessary to consider whether their times of peak usage are likely to coincide with peak network usage by VVR. Additionally, overhead added by VVR and the various underlying network protocols reduces effective bandwidth by a small amount, typically 3% to 5%.

How Network Performance Depends on Mode of Replication

All replicated write requests must eventually travel over the network to one or more Secondary nodes. Whether or not this trip is on the critical path depends on the mode of replication.

Because replicating in synchronous mode requires that data reach the Secondary node before the write can complete, the network is always part of the critical path for synchronous mode. This means that for any period during which application write rate exceeds network capacity, write latency increases.

Conversely, replicating in asynchronous mode does not impose this requirement, so write requests are not delayed if network capacity is insufficient. Instead, excess requests accumulate on the SRL, as long as the SRL is large enough to hold them. If there is a persistent shortfall in network capacity, the SRL eventually overflows. However, this setup does allow the SRL to be used as a buffer to handle temporary shortfalls in network capacity, such as periods of peak usage, provided that these periods are followed by periods during which the Secondary can catch up as the SRL drains. If a configuration is planned with this functionality in mind, you must be aware that Secondary sites may be frequently out of date.

Several parameters can change the asynchronous mode behavior described above by placing the network round-trip on the critical path in certain situations. The latencyprot and srlprot features, when enabled, can both have this effect. These features are discussed fully in Choosing Latency and SRL Protection.

To avoid problems caused by insufficient network bandwidth, apply the following principles:

• If synchronous mode is used, the network bandwidth must at least match the application write rate during its peak usage period; otherwise, the application is throttled. However, this leaves excess capacity during non-peak periods, which is useful to allow synchronization of new volumes using checkpoints as described in Peak Usage Constraint.
• If only asynchronous mode is used, and you have the option of allowing the Secondary to fall behind during peak usage, then the network bandwidth only needs to match the overall average application write rate. This might require the application to be shut down during synchronization procedures, because there is no excess network capacity to handle the extra traffic generated by the synchronization.
• If asynchronous mode is used with latencyprot enabled to avoid falling too far behind, the requirements depend on how far the Secondary is allowed to fall behind. If the latency high mark is small, replication will be similar to synchronous mode and therefore must have a network bandwidth sufficient to match the application write rate during its peak usage period. If the latency high mark is large, the Secondary can fall behind by several hours. Thus, the bandwidth only has to match the average application write rate.

Choosing the Network Protocol

VVR exchanges two types of messages between the Primary and the Secondary: heartbeat messages and data messages. The heartbeat messages are transmitted using the UDP transport protocol. VVR can use either the TCP transport protocol or the UDP transport protocol to exchange data messages.

The choice of protocol to use for the data messages is based on the network characteristics. TCP has been found to perform better than UDP on networks that lose packets. However, you must experiment with both protocols to determine the one that performs better in your network environment.

VVR uses the UDP transport protocol by default.

For information on how to set the network protocol, refer to the VERITAS Volume Replicator Administrator's Guide.

Choosing the Network Ports Used by VVR

VVR uses the UDP and TCP transport protocols to communicate between the Primary and Secondary. This section lists the default ports used by VVR.

By default, VVR uses the following ports when replicating data using UDP.

By default, VVR uses the following ports when replicating data using TCP.

The vrport command enables you to view and change the port numbers used by VVR. For instructions, see the VERITAS Volume Replicator Administrator's Guide.

Configuring VVR in a Firewall Environment

This section explains how to configure VVR to work in a firewall environment. For information about the default port numbers used by VVR, see section, Choosing the Network Ports Used by VVR. For information about replicating over a Network Address Translation (NAT) based firewall, see section VVR and Network Address Translation Firewall.

To configure VVR in a firewall environment when using TCP:

Enable in the firewall the following ports: the port used for heartbeats, the port used by the vradmind daemon, and the port used by the in.vxrsyncd daemon. Use the vrport command to display information about the ports and to change the ports being used by VVR.

To configure VVR in a firewall environment when using UDP:

1. In the firewall, enable the following ports:
   • the port used for heartbeats
   • the port used by the vradmind daemon and
   • the port used by the in.vxrsyncd daemon.

   Use the vrport command to display information about the ports and to change the ports being used by VVR.

2. Set a restricted number of ports to replicate data between the Primary and the Secondary. The operating system assigns anonymous port numbers by default. Most operating systems assign anonymous port numbers between 32768 and 65535. For each Primary-Secondary connection, one data port is required. Use the vrport command to specify a list of ports or range of ports to use for VVR.
3. In the firewall, enable the ports that have been set in step 2.

Choosing the Packet Size

If you have selected the UDP transport protocol for replication, the UDP packet size used by VVR to communicate between hosts could be an important factor in the replication performance. By default, VVR uses a UDP packet size of 8400 bytes. In certain network environments, such as those that do not support fragmented IP packets, it may be necessary to change the packet size. For instructions on how to change the packet size, see the VERITAS Volume Replicator Administrator's Guide.

Choosing the Network Maximum Transmission Unit

The UDP packets or TCP packets transmitted by VVR that are of size greater than the network Maximum Transmission Unit (MTU) are broken up into IP packets of MTU size by the IP module of the operating system. Certain network environments, such as Virtual Private Networks (VPNs), may add information to each IP packet. This makes the size of the IP packet greater than the network MTU, thus resulting in additional fragmentation. This results in twice the number of packets on the network. To avoid the second fragmentation, identify the number of bytes added to each packet by the VPN controller and reduce the network MTU by that amount.