Rules and Recommendations for Nodes and Clusters

Keep in mind the following rules and recommendations when managing nodes and clusters:

■   If you add a node as a standalone node or as a cluster manually in GigaVUE-FM, the node will remain in that state unless you delete the standalone node/cluster. Refer to Using Command Line Interface for Managing Clusters for rules on adding and deleting the nodes using CLI.
■   You can add a cluster to GigaVUE‑FM using the cluster VIP or the IP address of the leader leader in clustering node relationship (formerly master).
■   The state of the cluster is decided based on the cluster API response from the device.
■   Before joining an existing GigaVUE node to a cluster, it is recommended to use the no traffic all or reset factory command to clean up existing traffic-related configuration. For example, in a cluster there is one leader and the other nodes are GigaVUE-TA nodes. When the leader is removed from the cluster, the GigaVUE-TA nodes moves to the unknown state. If another leader joins the cluster with a different database, the GigaVUE-TA nodes that are existing in the cluster will remain in the unknown state.
■   Remove all physical loops before enabling the cluster. An accurate cluster topology will help with this. The GigaVUE‑OS node automatically detects and prevents configurations that would cause loops, but it is best to avoid them in the first place.
■   Star configurations offer the most efficient use of bandwidth. In general, use one GigaVUE‑OS node at the hub of your star and then connect spokes off of that.
■   Create stack-links with enough capacity to match expected cross-node traffic. For example, you can use a 24x10Gb (PRT-HC0-X24) or 6x40Gb (PRT-HC0-Q06) for the GigaVUE‑HC2 node.
■   Configure only the stack type ports that you will use in the stack-link configuration. Loops can be created if stack type ports are configured but then not used in a stack-link.
■   The first node added to the cluster becomes the leader. This is important when creating a new cluster using an existing, already-configured node and a new node. If you want to keep the configuration on your existing node, use it as the first node in the cluster. This way, the existing node becomes the leader and the new node inherits its configuration, preserving your existing settings. GigaVUE TA Series nodes are an exception since they cannot be the leader.
■   When joining a new node to an existing cluster, give the new node a lower precedence than the leader. Once the database has synchronized to the existing leader, you can increase the precedence to make the newly joined node the leader, if that is required.
■   Connecting two leaders to the same cluster is not supported. This is why you must make physical connections for the cluster control traffic interfaces before issuing any cluster configuration commands. Because the first node added to a cluster becomes the leader, configuring cluster settings before physically connecting the cluster control network results in a situation with multiple leaders attempting to connect to the same cluster.

Note:  Merging clusters is not supported.

■   For Layer 3 Out of Band clustering, the leader and the standby nodes must reside within the same subnet and must have the Gigamon Discovery Protocol (GDP) enabled for auto discovery.
■   GigaVUE TA Series and Certified Traffic Aggregation White Box nodes in a cluster can have tool, network, hybrid, and stack ports.
■   A GigaVUE TA Series node cannot be a leader. It can only join a cluster with other node types, such as GigaVUE‑HC1, GigaVUE‑HC2, or GigaVUE‑HC3.
■   A GigaVUE TA Series node cannot be a standby node either. If the cluster has one leader and all other nodes are GigaVUE TA Series nodes, the cluster will not have a standby.
■   Since a GigaVUE TA Series node can never be a leader or a standby in a cluster, a database restore is not possible. The best option is a text restore that has the information of the other nodes in the cluster removed from the text backup of the GigaVUE TA Series.
■   If GigaVUE‑HC1 and GigaVUE‑HC3 devices are member nodes of a scaled cluster of approximately 10K map rules, then the following sequence of steps should be avoided:
    1. Removal of nodes from the cluster.
    2. Reload the removed node with 10K map rules.

If the above steps are done, the nodes may get locked out and you will not be able to log in to the devices. To remove a GigaVUE‑HC1 or GigaVUE‑HC3 node from a cluster and use it as a standalone node:

  • Remove the node out of cluster.
  • Erase the configuration using the "no traffic all" command on the standalone node.
  • Reload the node.

RBAC and Tag Control on Nodes and Clusters:

  • A user belonging to the Infrastructure Management resource category can create clusters in GigaVUE-FM using the nodes that he has access to, depending on the tag keys and the tag values assigned to him. The tag keys and the tag values of the user will be applied on the nodes and the cluster. Similarly, when the user removes a node from the cluster, the removed node gets the tags of the user.

  • If a user has tag key and tag value set to ALL (as in the case of a super admin or admin user), then the tag key and the tag value applied to the nodes in the cluster depends on the tag value of the leader of the cluster. Similarly, if a node is removed from the cluster, then the tag value of the leader is applied to the removed node.

    • Example:

    Consider the following:

    • tag key: Location
    • tag values: California, Texas, Washington and New York
    • User configured with tag key Location and has access to all tag values (ALL). Also, the user has access to the following devices.
    • dev1 - 10.115.46.11 - Tag-> Location: Texas
    • dev2 - 10.115.46.12 - Tag -> Location: California

    If the user creates a cluster C1 with dev1 and dev2, and dev1 as a seed node, as the user has access to all tag values of the tag Location, the tag value is derived from the seed device. In this example, the seed device is dev1 which has tag value Texas.

    The created cluster C1 will have the tag key: Location with value Texas.

  • For nodes with single-valued tags, if you change the tag value of the node, a confirmation message pops-up. Based on the response, the tag value of the node is changed. Refer to the Create User-defined Tag section in the GigaVUE Administration Guide.

Using Command Line Interface for Managing Clusters

The following table provides information on the behavioral changes observed in GigaVUE‑FM when the nodes are managed from CLI.

Process

Until Software Version 5.10.00

 

From Software Version 5.11.00

 
Add Nodes to Cluster
● If the node is already managed by GigaVUE‑FM: The node will be added to the cluster and stack mode will be updated.
● If the node is not already managed by GigaVUE-FM: The node will be added to GigaVUE‑FM.
● If the node is already managed by GigaVUE‑FM: Config sync will fail and is notified with appropriate messages. You must first delete the node from GigaVUE-FM and then add the node to the cluster.

For example, consider the following nodes and clusters managed by GigaVUE‑FM:

  • Stand alone node A
  • Cluster C with nodes B and C.

To add the standalone node A to cluster C, you must first delete the node A from GigaVUE-FM and then add the node to cluster C.

  • If the node is not already managed by GigaVUE-FM: The node will be added to GigaVUE‑FM.
Remove nodes from cluster The node removed will be managed as a standalone node in GigaVUE‑FM.

The node removed from the cluster will also be removed from GigaVUE-FM. If you want GigaVUE-FM to manage the node as a standalone device, you must add the node again. Refer to Add New Physical Node or Cluster to GigaVUE‑FM section.
Edit Cluster ID The new cluster id will get updated during the next config sync cycle. The next config sync will fail because of the change in cluster ID. You must remove the cluster and add it again, so that the cluster is rediscovered with the new cluster ID.
Moving nodes between clusters The node is removed from the existing cluster and added to the new cluster.

The node will be moved to the new cluster only if it is removed from the existing cluster. If the node is not removed from the existing cluster, config sync will fail until the node is removed from existing cluster.This could happen when the config sync for the cluster from which the device is moved out is triggered first.

For example, consider the following nodes and clusters managed by GigaVUE-FM:

Cluster C1: Nodes A, B, C

Cluster C2: Nodes E, F, G

If node E is moved from C2 to C1 through CLI, then following will be the behavior in the next config sync:

If cluster C1 first completes the config sync:

  • Config sync will fail. This is because a device which is already in C2 is now being claimed in C1.
  • As soon as cluster C2 completes the config sync, node E will be removed from C2.
  • The next config sync will succeed for cluster C1.

If cluster C2 first completes the config sync:

  • Node E will be removed from C2.
  • When config sync for C1 is completed, node E will be added to C1.

If you keep moving the nodes from C2 to C1, then all but the last node will get moved. To move the last node you must first delete the cluster C2. In the subsequent config sync, node G will get added to cluster C1.

Note:  If a node in a cluster does not report during config sync, GigaVUE-FM will remove the node, and an alarm with severity status ‘Critical’ is triggered (Alarm description: Device is not reported). If the node reports back to the same cluster, then the alarm will be cleared. However, if you acknowledge or delete the alarm and the node does not report back (if the node has been removed or added to another cluster) with in an hour after the alarm has been acknowledged, then GigaVUE-FM will clear the alarm.