4.6.2.
The Selection Process and RMSelector.py
Up one level
The GRIA Job Service can be configured to use any number of Resource Managers. Deciding which one to use for a submitted job is done as a three-step selection process, outlined below.
In the screenshots below, we use five made-up Resource Manager Plugins - RM1, RM2, RM3, RM4 and RM5.
Figure 1: The decision process for selecting a Resource Manager
The Job Service starts by compiling a list of all the enabled RM Connector Plugins that are installed and enabled. You can see this list by looking at the main Job Service administration page (see Figure 2 below) - all the plugins in the list that are not greyed out are enabled, and will be used in this selection process.
Figure 2: A list of enabled plugins
Step 1: Filter by Application
When the service administrator deploys a new application on the job service, he can indicate which resource managers have the application installed. Any resource managers not selected will be immediately excluded from the selection process, and no jobs for that application will be able to run on them.
Figure 3: Selecting resource managers for an application
Step 2: Filter by Resources
At this stage, the Job Service asks each RM Connector Plugin whether it has enough resources to run the job being submitted. The plugins will typically look at the resource requirements section of the JSDL, and then query the actual resource manager to see if it is able to run the job. See Writing Custom Resource Manager Plugins for details of the canRunJob python function.
Some checks that the plugins might do include:
- Checking whether the operating system and system architecture requested by the submitter is available on any of the compute nodes.
- Checking whether there is a compute node with enough memory to run the job.
In our example, RM3 does not have enough resources to run the job and is excluded from further steps.
Note that support for this feature in the current set of RM plugins is quite limited. See column B in Support for Resource Elements for more information.
Step 3: Objective Function
The final decision as to which Resource Manager will be used for a job is made by a Python script - RMSelector.py. The default implementation of this function is to just choose the first plugin available, but administrators can override this behaviour.
The Python interface for selecting a plugin is very simple. The Job Service will look for a Python function called selectPlugin, and call it with two arguments:
- job - a Job object, containing information gathered from the JSDL including the job's name and its resource requirements.
- plugins - a list of RMConnector derived objects - representing all the plugins that made it to Step 3 in the selection process. The selectPlugin function is expected to return one of these objects.
A very simple example is given below. This always selects the plugin named "RM4".
#!/usr/bin/env python
def selectPlugin(job, plugins):
for plugin in plugins:
if plugin.__class__.__name__ == "RM4":
return plugin
return None
Once the administrator has written this script, he can instruct the Job Service to use it by entering its path in the configuration page:
Figure 4: Using another RMSelector
