Task Deployment

❗️

This is a legacy Apache Ignite documentation

The new documentation is hosted here: https://ignite.apache.org/docs/latest/

DeploymentSpi

Deployment functionality is provided via the DeploymentSpi interface.

Class loaders that are in charge of loading task classes (and other classes) can be deployed directly by calling the register(ClassLoader, Class) method or by SPI itself, for example by asynchronously scanning some folder for new tasks. When findResource(String) is called by the system, SPI must return a class loader associated with the given class. Every time a class loader gets (re)deployed or released, the DeploymentListener.onUnregistered(ClassLoader) method must be called.

If peer class loading is enabled, then it is usually enough to deploy a class loader only on one grid node. Once a task starts executing on the grid, all other nodes will automatically load all task classes from the node that initiated the execution. Hot redeployment is also supported with peer class loading. Every time a task changes and gets redeployed on a node, all other nodes will detect it and will redeploy this task as well. Note that peer class loading comes into effect only if a task was not locally deployed; otherwise, preference will always be given to local deployment.

Ignite provides these following DeploymentSpi implementations out of the box:

• UriDeploymentSpi
• LocalDeploymentSpi

🚧

SPI methods should never be used directly. SPIs provide an internal view on the subsystem and are used internally by Ignite. In rare use cases when access to a specific implementation of this SPI is required, an instance of this SPI can be obtained via the Ignite.configuration() method to check its configuration properties or call other non-SPI methods. Note again that calling methods from this interface on the obtained instance can lead to undefined behavior and explicitly is not supported.

UriDeploymentSpi

This implementation of DeploymentSpi can deploy tasks from different sources like file system folders, email, and HTTP. There are different ways to deploy tasks in grid and every deployment method depends on a selected source protocol. This SPI is configured to work with a list of URI's. Every URI contains all the data about protocol/transport plus configuration parameters like credentials, scan frequency, and others.

When SPI establishes a connection with a URI, it downloads deployable units to the temporary directory in order to protect it from any changes made while scanning. Use setTemporaryDirectoryPath(String) to set a custom temporary folder for downloaded deployment units. SPI will create the folder under the path with a name identical to local node ID.

SPI tracks all changes of every given URI. This means that if any file is changed or deleted, SPI will re-deploy or delete corresponding tasks. Note that the very first call to findResource(String) is blocked until SPI finishes scanning all URI's at least once.

There are several deployable unit types supported:

• GAR file
• Local disk folder with structure of unpacked GAR file
• Local disk folder containing only compiled Java classes

GAR file

GAR file is a deployable unit. GAR file is based on ZLIB compression format like simple JAR file and its structure is similar to WAR archive. GAR file uses the '.gar' extension.

GAR file structure (file or directory ending with '.gar'):

META-INF/
        |
         - ignite.xml
         - ...
lib/
   |
    -some-lib.jar
    - ...
xyz.class
...

• META-INF/ entry may contain ignite.xml file which is a task descriptor file. The purpose of the task descriptor XML file is to specify all tasks to be deployed. This file is a regular Spring XML definition file. META-INF/ may also contain any other files specific for JAR format.
• lib/ entry contains all library dependencies.
• Compiled Java classes must be placed in the root of a GAR file.

GAR file may be deployed without a descriptor file. If there is no descriptor file, SPI will scan all classes in the archive and instantiate those that implement the ComputeTask interface. In that case, all grid task classes must have a public no-argument constructor. Use the ComputeTaskAdapter adapter for convenience when creating grid tasks.

By default, all downloaded GAR files that have digital signature in the META-INF folder will be verified and deployed only if the signature is valid.

Example

The following examples demonstrate how the deployment SPI can be used. Different protocols can be used together as well.

// The example expects that you have a GAR file in
// `home/username/ignite/work/my_deployment/file` folder 
// which contains `myproject.HelloWorldTask` class.

IgniteConfiguration cfg = new IgniteConfiguration();

UriDeploymentSpi deploymentSpi = new UriDeploymentSpi();

deploymentSpi.setUriList(Arrays.asList("file:///home/username/ignite/work/my_deployment/file"));

cfg.setDeploymentSpi(deploymentSpi);

try(Ignite ignite = Ignition.start(cfg)) {
	  ignite.compute().execute("myproject.HelloWorldTask", "my args");
}

// The example expects that you have a HTMP under
// 'www.mysite.com:110/ignite/deployment'page which contains a link
// on GAR file which contains `myproject.HelloWorldTask` class.

IgniteConfiguration cfg = new IgniteConfiguration();

UriDeploymentSpi deploymentSpi = new UriDeploymentSpi();

deploymentSpi.setUriList(Arrays.asList("http://username:password;[email protected]:110/ignite/deployment"));

cfg.setDeploymentSpi(deploymentSpi);

try(Ignite ignite = Ignition.start(cfg)) {
	  ignite.compute().execute("myproject.HelloWorldTask", "my args");
}

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  ...
  <property name="deploymentSpi">
    <bean class="org.apache.ignite.spi.deployment.uri.UriDeploymentSpi">
     <property name="temporaryDirectoryPath" value="c:/tmp/grid"/>
      <property name="uriList">
        <list>
          <value>http://www.site.com/tasks</value>
          <value>file://freq=20000@localhost/c:/Program files/gg-deployment</value>
        </list>
      </property>
    </bean>
  </property>
</bean>

Configuration

Protocols

Following protocols are supported in SPI out of the box:

• file:// - File protocol
• http:// - HTTP protocol
• https:// - Secure HTTP protocol

📘

Custom Protocols

You can add support for additional protocols if needed. To do this, implement the UriDeploymentScanner interface and plug your implementation into the SPI via the setScanners(UriDeploymentScanner... scanners) method.

In addition to SPI configuration parameters, all necessary configuration parameters for selected URIs should be defined in the URI. Different protocols have different configuration parameters described below. Parameters are separated by the ';' character.

File

For this protocol, SPI scans a folder specified by the URI and downloads any GAR files or directories that end with .gar.

The following parameters are supported:

File Uri Example

The following example will scan the c:/Program files/ignite/deployment folder on the local computer every '2000' milliseconds. Note that since the path has spaces, the setEncodeUri(boolean) parameter must be set to true (which is default behavior).

file://freq=2000@localhost/c:/Program files/ignite/deployment

HTTP/HTTPS

The URI deployment scanner tries to read the DOM of the HTML file it points to and parses out the HREF attributes of all <a>-tags - this becomes the URL collection (to GAR files) to deploy: each 'A' link should be a URL to a GAR file. It's important that only HTTP scanner uses the URLConnection.getLastModified() method to check if there were any changes since last iteration for each GAR-file before redeploying.

The following parameters are supported:

Http Uri Example

The following example will download the page www.mysite.com/ignite/deployment, parse it, and download and deploy all GAR files specified by the HREF attributes of <a> elements on the page using authentication username:password every '10000' milliseconds (only new/updated GAR-s).

http://username:password;[email protected]:110/ignite/deployment

LocalDeploymentSpi

Local deployment SPI that implements only within VM deployment on local node via the register(ClassLoader, Class) method. This SPI requires no configuration.

There is no point to explicitly configure LocalDeploymentSpi as it is used by default and has no configuration parameters.