4.4.1.
Overview
Up one level
Relation to the Job Service
Figure 1 - The Application Wrapper Scripts
The application wrapper scripts are deployed along with the applications - either on a shared file system or individually on each compute node. Their function is to provide a uniform interface to the Job Service for starting, monitoring and stopping applications.
The application model
The Job Service and RM connector scripts are designed to support a uniform model of application execution, shown in Figure 2:
Figure 2 - Application Model
The workspace for each job is set up by the Job Service when the job is initialised (this is one of the bookkeeping service operations that must precede the call to start the job). The workspace has a standard directory structure so the Job Service and RM connector scripts can create and find information stored in it, including a "work" sub-directory where the job will actually run.
When the user starts the job, the Job Service transfers input data files from Data Service URIs into the job's workspace. It then runs the RM connector script which submits the application to the execution platform (cluster, etc) where it will run (using the specified command line), possibly after some queuing delay. The application reads input data deposited in its workspace by the Job Service, and writes outputs back to the workspace once the job has finished. The Job Service can then find the outputs and transfer these to the correct output Data Service URIs.
When the user asks for the status of the job, the service queries the Resource Manager for status information (e.g. when the job started or finished, etc). The service may also run an associated monitoring application (using the specified command line) to gather application-specific status information (e.g. number of iterations completed, convergence plots, etc) from the workspace.
If the user asks for the job to be killed, the service uses issues a command to kill the job on the execution platform. The Job Service will detect that the job has finished, and will transfer any output produced. Note that the user (or their client-side application) should always check the status of a job to find out if it crashed or was killed, as some incomplete output may appear in the latter case.
Why are Application Wrapper Scripts Required?
In practice, few legacy applications behave exactly according to the model shown in Figure 2. It is rarely possible to change the application itself to fix this, so instead GRIA uses so-called wrapper scripts that do conform to the application model for starting and managing the application.
In practice, the wrapper scripts can do more than just make the underlying application work as indicated in Figure 2. They can also be used to handle and implement application specific features of the service.
One can also use (optional) wrapper scripts to look for application-specific status information in the working directory of the job. Without such scripts, the service can only obtain basic job status information from the job submission system.
Finally, wrapper scripts also provide a configurable mechanism for dealing with any application-specific security risks, e.g. checking for malicious input that may exploit a feature of the application. Few legacy applications were designed as network-accessible services, and since we can't change them to remove security loopholes, the use of wrapper scripts is essential to check for any exploits of application vulnerabilities. In the limit, one can configure the wrapper (and RM connector) scripts to run the application in a sandbox (e.g. chroot), with access only to a working sub-directory of the job workspace.
