Clusters & load balancing - Storm Streaming Server

It is possible to configure multiple instances of Storm Streaming Server into a group (cluster). The main advantage of this approach is load balancing and redundancy capabilities. Servers can be configured in a hierarchical structure with one or more origin servers (this is where content is published) and multiple edge servers, or in a flat arrangement, where each server works as a gateway.

Origin configuration

In this approach, all incoming streams will be put on origin servers, while edge servers will copy the signal and redistribute it among viewers. In the preferences.xml file we must first configure its role.

For origin the configuration will look as follow:

roleIt can be either "origin" or "edge." In this example, we’re only presenting the origin configuration
allowedIPsList of IPs separated by a comma (space is optional). Only servers from these IPs will be allowed to connect. If the list is empty, you can still join servers using the password parameter. Both the allowedIPs and the password fields can be used together or separately, but at least one must be present for a cluster to work
passwordPassword for the cluster. If the field is empty, the password won’t be checked, but the system will rely on the allowedIPs list. Both the allowedIPs and the password fields can be used together or alone, but at least one must be present
Table 1. Origin configuration table

Edge configuration

Now we need to configure Edge server

roleAs mentioned above in this case, it must be set to "Edge"
allowedIPsIf a password is not required by the Origin server this field can be omitted
originIPsList of origin servers to which this edge server must connect to. IPs must be separated by a comma (space is optional)
Table 2. Edge configuration table

We’ll also have to setup the gateway:

<Gateway enabled="true" loadbalancing="true" checkInterval="3" />
enabledNot all edge servers must have a gateway enabled. Other gateways will decide on them regarding their load if necessary
loadbalancingIf this option is set to true a gateway will redirect the viewer to the least utilized server. Otherwise, stream data will be provided from a given server
checkIntervalSince all servers ask and hold information regarding each other’s loads, these are being checked and reported in intervals. For most cases, we advise keeping this value around 3 (seconds)
Table 3. Loadbalancing configuration table