Cluster Configuration

In Ignite, nodes can discover each other by using DiscoverySpi. Ignite provides TcpDiscoverySpi as a default implementation of DiscoverySpi that uses TCP/IP for node discovery. Discovery SPI can be configured for Multicast and Static IP based node discovery.

Multicast Based Discovery

TcpDiscoveryMulticastIpFinder uses Multicast to discover other nodes in the grid and is the default IP finder. You should not have to specify it unless you plan to override default settings. Here is an example of how to configure this finder via Spring XML file or programmatically:

var cfg = new IgniteConfiguration
{
    DiscoverySpi = new TcpDiscoverySpi
    {
        IpFinder = new TcpDiscoveryMulticastIpFinder
        {
            MulticastGroup = "228.10.10.157"
        }
    }
};

<igniteConfiguration>
    <discoverySpi type='TcpDiscoverySpi'>
        <ipFinder type='TcpDiscoveryMulticastIpFinder' multicastGroup='228.10.10.157' />
    </discoverySpi>
</igniteConfiguration>

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  ...
  <property name="discoverySpi">
    <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
      <property name="ipFinder">
        <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
          <property name="multicastGroup" value="228.10.10.157"/>
        </bean>
      </property>
    </bean>
  </property>
</bean>

Static IP Based Discovery

For cases when Multicast is disabled, TcpDiscoveryStaticIpFinder (TcpDiscoveryVmIpFinder in Java) should be used with pre-configured list of IP addresses.

You are only required to provide at least one IP address of a remote node, but usually it is advisable to provide 2 or 3 addresses of grid nodes that you plan to start at some point of time in the future. Once a connection to any of the provided IP addresses is established, Ignite will automatically discover all other grid nodes.

🚧
By default the TcpDiscoveryStaticIpFinder is used in non-shared mode. If you plan to start a server node then in this mode the list of IP addresses should contain an address of the local node as well. It will let the node not to wait while other nodes join the cluster but rather become the first cluster node and operate normally.

Here is an example of how to configure this finder via Spring XML file or programmatically from Java:

var cfg = new IgniteConfiguration
{
    DiscoverySpi = new TcpDiscoverySpi
    {
        IpFinder = new TcpDiscoveryStaticIpFinder
        {
            Endpoints = {"1.2.3.4", "1.2.3.5:47500..47509" }
        }
    }
};

<igniteConfiguration>
    <discoverySpi type='TcpDiscoverySpi'>
        <ipFinder type='TcpDiscoveryStaticIpFinder'>
            <endpoints>
                <string>1.2.3.4</string>
                <string>1.2.3.5:47500..47509</string>
            </endpoints>
        </ipFinder>
    </discoverySpi>
</igniteConfiguration>

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  ...
  <property name="discoverySpi">
    <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
      <property name="ipFinder">
        <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
          <property name="addresses">
            <list>
              <value>1.2.3.4</value>
              <value>1.2.3.5:47500..47509</value>
            </list>
          </property>
        </bean>
      </property>
    </bean>
  </property>
</bean>

Multicast and Static IP Based Discovery

You can use both, Multicast and Static IP based discovery together. In this case, in addition to addresses received via multicast, if any, TcpDiscoveryMulticastIpFinder can also work with pre-configured list of static IP addresses, just like Static IP-Based Discovery described above. Here is an example of how to configure Multicast IP finder with static IP addresses:

var cfg = new IgniteConfiguration
{
    DiscoverySpi = new TcpDiscoverySpi
    {
        IpFinder = new TcpDiscoveryMulticastIpFinder
        {
            MulticastGroup = "228.10.10.157",
            Endpoints = {"1.2.3.4", "1.2.3.5:47500..47509" }
        }
    }
};

<igniteConfiguration>
    <discoverySpi type='TcpDiscoverySpi'>
        <ipFinder type='TcpDiscoveryMulticastIpFinder' multicastGroup='228.10.10.157'>
            <endpoints>
                <string>1.2.3.4</string>
                <string>1.2.3.5:47500..47509</string>
            </endpoints>
        </ipFinder>
    </discoverySpi>
</igniteConfiguration>

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  ...
  <property name="discoverySpi">
    <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
      <property name="ipFinder">
        <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
          <property name="multicastGroup" value="228.10.10.157"/>
          <property name="addresses">
            <list>
              <value>1.2.3.4</value>
              <value>1.2.3.5:47500..47509</value>
            </list>
          </property>
        </bean>
      </property>
    </bean>
  </property>
</bean>

Isolated Ignite Clusters on the Same Set of Machines

There can be a case when you need to start two isolated Ignite clusters on the same set of machines due to testing purposes or by some other reason.

In Ignite this task is achievable if nodes from different clusters will use non intersecting local port ranges for TcpDiscoverySpi and TcpCommunicationSpi.

Let's say that you need to start two isolated clusters on a single machine for testing purposes.
Then for the nodes from the first cluster, you should use the following TcpDiscoverySpi and TcpCommunicationSpi configurations:

var cfg = new IgniteConfiguration
{
    // Explicitly configure TCP discovery SPI to provide list of initial nodes
    // from the first cluster.
    DiscoverySpi = new TcpDiscoverySpi
    {
        // Initial local port to listen to.
        LocalPort = 48500,
        // Changing local port range. This is an optional action.
        LocalPortRange = 20,
        IpFinder = new TcpDiscoveryStaticIpFinder
        {
            // Addresses and port range of the nodes from the first cluster.
            // 127.0.0.1 can be replaced with actual IP addresses or host names.
            // The port range is optional.
            Endpoints = new[] {"127.0.0.1:48500..48520"}
        }
    },
    // Explicitly configure TCP communication SPI changing 
    // local port number for the nodes from the first cluster.
    CommunicationSpi = new TcpCommunicationSpi
    {
        LocalPort = 48100
    }
};

<igniteConfiguration>
    <!--
        Explicitly configure TCP discovery SPI to provide list of initial
        nodes from the second cluster.    
    -->
    <discoverySpi type='TcpDiscoverySpi' localPort='48500' localPortRange='20'>
        <ipFinder type='TcpDiscoveryMulticastIpFinder'>
            <endpoints>
                <!--
                    Addresses and port range of the nodes from the second cluster.
                    127.0.0.1 can be replaced with actual IP addresses or host names. Port range is optional.
                -->
                <string>127.0.0.1:48500..48520</string>
            </endpoints>
        </ipFinder>
    </discoverySpi>
        
    <!--
        Explicitly configure TCP communication SPI changing local port number 
        for the nodes from the second cluster.
    -->
    <communicationSpi type='TcpCommunicationSpi' localPort='48100' />
</igniteConfiguration>

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  	...
    <!--
 				Explicitly configure TCP discovery SPI to provide list of 
				initial nodes from the first cluster.
 	  -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <!-- Initial local port to listen to. -->
            <property name="localPort" value="48500"/>

            <!-- Changing local port range. This is an optional action. -->
            <property name="localPortRange" value="20"/>

            <!-- Setting up IP finder for this cluster -->
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <!--
                                Addresses and port range of the nodes from the first
 																cluster.
                                127.0.0.1 can be replaced with actual IP addresses or
 																host names. Port range is optional.
                            -->
                            <value>127.0.0.1:48500..48520</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>

    <!--
        Explicitly configure TCP communication SPI changing local
 				port number for the nodes from the first cluster.
    -->
    <property name="communicationSpi">
        <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
            <property name="localPort" value="48100"/>
        </bean>
    </property>
</bean>

While for the nodes from the second cluster, the configuration can look like this:

var cfg = new IgniteConfiguration
{
    // Explicitly configure TCP discovery SPI to provide list of initial nodes
    // from the first cluster.
    DiscoverySpi = new TcpDiscoverySpi
    {
        // Initial local port to listen to.
        LocalPort = 49500,
        // Changing local port range. This is an optional action.
        LocalPortRange = 20,
        IpFinder = new TcpDiscoveryStaticIpFinder
        {
            // Addresses and port range of the nodes from the first cluster.
            // 127.0.0.1 can be replaced with actual IP addresses or host names.
            // The port range is optional.
            Endpoints = {"127.0.0.1:49500..49520"}
        }
    },
    // Explicitly configure TCP communication SPI changing 
    // local port number for the nodes from the first cluster.
    CommunicationSpi = new TcpCommunicationSpi
    {
        LocalPort = 49100
    }
};

<igniteConfiguration>
    <!--
        Explicitly configure TCP discovery SPI to provide list of initial
        nodes from the second cluster.    
    -->
    <discoverySpi type='TcpDiscoverySpi' localPort='49500' localPortRange='20'>
        <ipFinder type='TcpDiscoveryMulticastIpFinder'>
            <endpoints>
                <!--
                    Addresses and port range of the nodes from the second cluster.
                    127.0.0.1 can be replaced with actual IP addresses or host names. Port range is optional.
                -->
                <string>127.0.0.1:49500..49520</string>
            </endpoints>
        </ipFinder>
    </discoverySpi>
        
    <!--
        Explicitly configure TCP communication SPI changing local port number 
        for the nodes from the second cluster.
    -->
    <communicationSpi type='TcpCommunicationSpi' localPort='49100' />
</igniteConfiguration>

<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
    <!--
        Explicitly configure TCP discovery SPI to provide list of initial
         nodes from the second cluster.
    -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <!-- Initial local port to listen to. -->
            <property name="localPort" value="49500"/>

            <!-- Changing local port range. This is an optional action. -->
            <property name="localPortRange" value="20"/>

            <!-- Setting up IP finder for this cluster -->
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <!--
                                Addresses and port range of the nodes from the second
 																cluster.
                                127.0.0.1 can be replaced with actual IP addresses or
 																host names. Port range is optional.
                            -->
                            <value>127.0.0.1:49500..49520</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>

    <!--
        Explicitly configure TCP communication SPI changing local port number 
        for the nodes from the second cluster.
    -->
    <property name="communicationSpi">
        <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
            <property name="localPort" value="49100"/>
        </bean>
    </property>
</bean>

As you see from the configurations the difference between them is minor - only port numbers for SPIs and IP finder vary.

📘
If you want the nodes from different clusters are able to look for each other using multicast protocol then replace TcpDiscoveryStaticIpFinder with TcpDiscoveryMulticastIpFinder and set unique TcpDiscoveryMulticastIpFinder.MulticastGroups in each configuration above.

Failure Detection Timeout

Failure detection timeout is used to determine how long a cluster node should wait before considering a remote connection with other node failed. This timeout is the easiest way to tune discovery SPI's failure detection feature depending on the network and hardware conditions of your cluster.

🚧
The timeout automatically controls such configuration parameters of TcpDiscoverySpi as socket timeout, message acknowledgment timeout and others. If any of these parameters is set explicitly, then the failure timeout setting will be ignored.

Failure detection timeout is configured with IgniteConfiguration.FailureDetectionTimeout property. Default value, that is equal to 10 seconds, is chosen in a way to make it possible for discovery SPI to work reliably on most of hardware and virtual deployments, but this has made failure detection time worse. However, for stable low-latency networks the parameter may be set to ~200 milliseconds in order to detect and react on failures quicker.