About Heartbeats
When heartbeat packets are sent to an inline tool, they are expected to be received back when the inline tool is healthy. Negative heartbeat packets complement heartbeat packets to verify the health of inline tools. When negative heartbeat packets are sent to an inline tool, they are not expected to be received back when the inline tool is healthy.
When some inline tools begin to fail, they allow packets though that should have been dropped. A negative heartbeat detects such a failure by sending a packet that should not pass through an inline tool. If the negative heartbeat packet passes through an inline tool, the tool is deemed to have failed. Therefore, a negative heartbeat packet received back from an inline tool indicates a tool failure.
Heartbeat packets and negative heartbeat packets can be used in any combination: heartbeat only, negative heartbeat only, neither, or both.
A heartbeat profile supports health monitoring of inline tools and inline networks. A heartbeat profile is a group of attributes applied to an inline tool or inline network to configure its heartbeat operation. Multiple inline tools can share a heartbeat profile. However, for inline network, heartbeat is supported only within an inline network pair; it is not supported on inline netlag or inline network groups.
To display the configured heartbeat profiles, select Inline Bypass > Heartbeats to open the Heartbeats page. An example is shown in Figure 1.
Figure 143 | Heartbeats Page with Heartbeat Profiles |
The Heartbeats page includes a default heartbeat profile that has the following settings:
Alias—default |
Type—Regular |
Packet Format—arp or custom |
Custom Packet—URL from which a PCAP file can be imported |
Direction—bi-directional |
Period—1000 milliseconds |
Timeout—500 milliseconds |
Recovery Period—30 seconds |
Retries—3 |
The highest frequency heartbeat that can be configured is as follows:
period—30 milliseconds |
timeout—20 milliseconds |
retry-count—0 |
The heartbeat mechanism supports the maximum number of inline tools, at the highest frequency, which is 48 on the GigaVUE‑HC3 and GigaVUE‑HC2, and 16 on the GigaVUE‑HC1.
To display the heartbeat profile associated with an inline tool and the heartbeat status, open the Inline Tools page by selecting Inline Bypass > Inline Tools. There is also a combined heartbeat status, which combines heartbeat and negative heartbeat statuses and indicates the tool health used for inline tool failover actions or SNMP traps. The combined heartbeat status is the combination of both heartbeat statuses. If both are configured and one is down, the combined status will be down.
The standard heartbeat is a packet sent by the GigaVUE node that passes through the inline tool to verify that it is passing traffic, even if the link is up. If the packet is not passed through the tool, the tool is considered to have failed and a bypass action is triggered.
Even when the tool is considered down, heartbeat packets continue to be sent so that the bypass action can be reversed when the tool is healthy again.
Heartbeats are sent bidirectionally to the inline tool.
The format of the heartbeat packet can be the standard ARP packet or a custom packet. For a custom packet, you must provide a URL from which a PCAP file can be imported. If the PCAP file contains several packets, the first packet present in the file is taken as the heartbeat packet. The size of a custom heartbeat packet must be less than 128 bytes.
Note: The system will overwrite the MAC address portion of the custom heartbeat packet.
If the inline tool through which the heartbeat packets are passed is expecting IPv6 traffic exclusively, you must select a custom heartbeat packet.
Custom heartbeat packets are needed in situations in which inline tools do not reliably pass standard ARP packets. For example, if an inline tool is configured to pass only IPv6 traffic, an ICMPv6 ARP packet might be appropriate.
If a custom heartbeat packet is specified, the Heartbeats page displays the name of the PCAP file from which it was imported.
The health of the inline tool is critical to the proper handling of traffic. An inline tool is determined to have failed if:
link is lost to the tool |
inline heartbeat fails |
When the tool fails in one direction, it is considered to have failed in both directions. For example, if the heartbeat stops flowing in the northbound direction, neither northbound or southbound packets are sent to the tool.
A negative heartbeat profile is a group of attributes applied to an inline tool to configure its negative heartbeat operation. Multiple inline tools can share a negative heartbeat profile. The content of a negative heartbeat is configurable using the same PCAP file mechanism as for a custom heartbeat packet.
Use the Add Heartbeats page shown in About Heartbeats to configure a negative heartbeat profile. A negative heartbeat profile can be created by selecting Negative in the Type field. The profile will have the following settings:
PacketFormat set to Custom |
Direction set to Bi-directional |
Period set to 1000 (period is specified in milliseconds) |
Recover Time set to 30 (recovery is set in seconds) |
You must provide a valid PCAP file when Packet Format is set to Custom before the negative heartbeat profile can be applied to an inline tool.
When a negative heartbeat is configured, the system will send packets specified by the Custom Packet Format, in the time specified by Period, in the direction specified by Direction. The inline tool absorbs the negative heartbeat packets until the number of seconds specified by Recovery Time has passed. Recovery Time specifies the number of seconds of not receiving negative heartbeat packets in order for the inline tool to be declared healthy.
The negative heartbeat mechanism supports the maximum number of inline tools, which is 48 on the GigaVUE‑HC3 and GigaVUE‑HC2, and 16 on the GigaVUE‑HC1.
Use the Heartbeat page to display the configured negative heartbeat profiles. Use the Inline Tool page to display the negative heartbeat profile associated with an inline tool, the negative heartbeat status, and the counters of received packets in each direction. There is also a combined heartbeat status, which combines heartbeat and negative heartbeat statuses and indicates the tool health used for inline tool failover actions or SNMP polling. The combined heartbeat status is the combination of both heartbeat statuses. For example, if both are enabled and both are up, the combined status is up. If both are enabled and one is down, the combined status is down.
The heartbeat mechanism focuses on providing extended heartbeat capability to monitor the following types of devices when the devices are connected to the inline-tool pair of ports as a tool:
GigaVUE nodes |
GigaVUE nodes with GigaSMART operations configured |
Following figure illustrates an example of a topology with GigaVUE nodes placed at three different layers.
The GigaVUE node at the access layer accesses the network traffic, gets the traffic processed by the tools at the tool layer, and transmits the processed traffic back to the network.
The GigaVUE node at the distribution layer distributes the traffic from the access layer to the tool layer.
The GigaVUE node at the tool layer acts as the SSL decryption tool.
In this topology, heartbeats are essential to monitor the traffic integrity at the distribution layer and to ensure automatic failover in case of a tool failure. In the access layer device, the ports that are connected to the distribution layer device are configured as inline tool ports because they face the tool side. In the distribution layer device, the ports that are connected to the access layer device are configured as inline network ports because they face the network side of the topology. The heartbeat packets will be sent from the inline tool port pair of the access layer device to the inline network port pair of the distribution layer device. If the forwarding state of the inline network pair is normal, the heartbeat packet is sent back to the inline tool port pair of the access layer device. Else, the packet is dropped.
The heartbeat mechanism is extended to support the GigaVUE node at the distribution layer to monitor the GigaVUE node that acts as a tool at the tool layer. In the distribution layer device, the ports that are connected to the tool layer device are configured as inline tool ports. In the tool layer device, the ports that are connected to the distribution layer device are configured as inline network ports. The heartbeat packets that are sent from the distribution layer device to the tool layer device will monitor the availability of both, the tool layer device and its GigaSMART engines.
Keep in mind the following rules and notes when you configure the heartbeat mechanism between GigaVUE nodes:
The heartbeat mechanism is supported between GigaVUE-HC1, GigaVUE‑HC2, and GigaVUE‑HC3 devices. |
Negative heartbeat profiles are not supported. |
Both standard and custom heartbeat packets are supported. |
Both protected and unprotected inline network pairs are supported. |
The heartbeat mechanism is not supported on inline netlag and inline network group components. However, it is supported on the underlying inline networks. |
The heartbeat mechanism is not supported for classic inline bypass solution. |
The minimum timeout for heartbeat sessions is 200 milliseconds, if the tool type is configured as GigaVUE Node. |
In your topology, you may sometimes notice heartbeat packets getting dropped at the inline network or the heartbeat status of the inline tool is not up. For details about how to isolate and troubleshoot such failures, refer to the following sections:
Heartbeat Status of Inline Tool is Not Operationally Up |
Heartbeat Packets are Dropped |
Use the Add Heartbeats page shown in the below figure to configure a regular heartbeat profile or a negative heartbeat profile by selecting Inline Bypass > Heartbeats > Heartbeats, and then clicking New.
Statistics about heartbeat profiles are displayed on the heartbeats statistics page. To open the Statistics page, select Inline Bypass > Heartbeats > Statistics. The page shows the following information:
Column |
Description |
Alias |
The alias of the inline tool or inline network to which the heartbeat profile is associated with. |
Inline Type |
The inline type to which the heartbeat profile is associated with. The valid values are Inline Tool or Inline Network. |
Heartbeat Profile |
The alias of the heartbeat profile |
Heartbeat Type |
Indicates the type of heartbeat profile: Regular or Negative |
A to B Packets |
The number of packets sent/received from port A to port B of the inline tool. |
B to A Packets |
The number of packets sent/received from port B to port A of the inline tool. |
A to B Packet Drops |
The number of packets dropped from port A to port B of the inline tool or inline network. |
B to A Packet Drops |
The number of packets dropped from port B to port A of the inline tool or inline network. |
You can clear the statistics for a specific heartbeat profile or all heartbeat profile. To clear the statistics for a heartbeat profile, select the required row, and then click Clear.
To clear the statistics for all heartbeat profiles, select Clear All, and the choose one of the following options:
Select Clear All Inline Network Heartbeat Stats to clear the heartbeat statistics for all inline networks. |
Select Clear All Inline Tools Heartbeat Stats to clear the heartbeat statistics for all inline tools. |
Select Clear All Negative Heartbeat Stats to clear only the statistics for negative heartbeat profiles. The statistics for regular heartbeat profiles will remain. |
Following a reload, there is a 5-minute delay for the system to stabilize before heartbeat packets are sent or received. During this delay, the heartbeat status is down.