Data Collector Groups

A data collector group consists of multiple data collectors working together to share the load that normally one data collector would handle. A data collector group helps load balance traffic, as you add more devices in SIP the amount of traffic messages being sent to a data collector will increase, a data collector group allows for improved load handling ability when processing the increased syslog and non-syslog traffic messages.

A data collector group will allow for child devices to be on the same data collector as their parent device. A data collector group can be assigned to the parent and child devices. It will determine which data collector will handle the devices associated with the data collector group.

Assigning three or more data collectors to a group also provides a level of redundancy. If a data collector is not able to communicate with another collector in the group, it will be considered down; the devices associated with the 'down' data collector will be divided between the remaining collectors in the group.

Notes:

  • The best practice is to select 3, 5 or another odd number of collectors per data collector group.
  • It is recommended that all data collectors in a data collector group be in the same data center.
  • When you add a new data collector, a new data collector group will also be created. It will have a system generated name of : <Data Collector Name>-Group and a description of: Generated automatically from <Data Collector Name> first time registration.
  • The newly added data collector will be automatically added to this new group.
  • If you have to do system restore, and no data collector groups are in the backup, then a data collector group will need to be created and devices associated to it .
  • Management Stations can be in a different data collector group as its child devices.
  • A load balancer is not required for a single data collector group with multiple data collectors but would improve redundancy. It would allow for log message redundancy because the firewall or CSS will send the log messages to the load balancer to forward to an available data collector for processing.

How syslog messages route in a data collector group

Firewalls should be configured to send logs to an IP address owned by a load balancer. The load balancer then distributes the messages to individual cluster members.

  • If you are sending syslog messages to a DC group that consists of a single data collector (the group has only one member), then the syslog messages should be sent directly to the data collector.

  • If you are sending syslog messages to a group which contains multiple data collectors, you should use a load balancer to distribute syslog messages among the data collectors.

  • If you're using Check Point and LEA, then syslog messages aren't a concern since LEA doesn't use syslog messages. In this case, there is still a benefit to having multiple data collectors in a group, since the group can assign LEA connections to various group members, and can reassign connections if a data collector goes down.
  • If you're using a mix of syslog and LEA, then you should adhere to the syslog message configuration requirements. If you want to send syslog messages to a data collector group containing more than one member, you must use a load balancer. If you have no load balancer, you should set up a separate data collector in its own group-of-one, to handle the syslog messages so that the multi-data collector group only handles LEA messages.
Syslog over TLS

You will need a TLS Certificate and Private Key (encoded in PEM format) to use Syslog over TLS, and the data collector must be able to listen on port 6514.

Permission Requirements

A user will need to be a member of a user group with the following minimum permissions granted:

  • Administration: Data Collectors

  • Module: Administration

Open the Data Collector Groups List

  • To view a list of data collector groups, click System > Data Collector Groups.