Personal tools
You are here: Home GRIA Documentation Documentation 5.3 Reference Manuals Client User Guide

Client User Guide

Note: Return to reference manual view.

This guide describes how to use the Client Package to access GRIA management and application services via desktop applications, including a client API toolkit for integration of client applications.

1. Overview

The overview page

The GRIA client allows you to use GRIA services, such as data storage, batch processing and database systems. It also allows you to use the GRIA management services, which are needed so that use of the other services can be recorded and billed.

The client management package

There are two ways to use GRIA services:

  • The default GRIA client provides a Swing-based graphical interface which lets you manage accounts and service level agreements, run jobs and store data. The client supports plug-ins allowing support for new types of service to be added without recompiling the client.

  • GRIA services can also be used directly from your own applications, by using the GRIA client library. This provides a set of low-level Java objects which provide access to the remote services' operations and can be used by non-graphical applications (batch processing), or by applications where you wish to replace the standard GRIA interface components with your own. The client library also contains the default Swing components, which can be embedded into your application directly.

This documentation only describes the default Swing application; see the GRIA Developer Kit for information about integrating GRIA with your own applications.

2. Installation

Client package installation

Prerequisites

Unpacking the software

This document explains how to install the command-line client library. This program allows you to perform operations on the GRIA services by giving commands at a command line (Unix shell or Windows DOS prompt).

Unpack the gria-client-<version>.zip or gria-client-<version>.tar.gz archive into a new directory. The archive contains the following items:

conf
The configuration files go here. This can be set to another directory be setting an environment variable GRIA_CONF_DIR
gridcli (Linux) or gridcli.bat (Windows)
Scripts to run the command-line client easily.
lib
All the .jar files needed to run the client.
doc
The documentation files.
README.html
A file telling you where the documentation files are.
LICENSE.txt
License terms.

Upgrading

To upgrade from a GRIA Client 5.x simply set the GRIA_CONF_DIR directory environment variable to the location of your old configuration directory . There is then no need to generate a keystore but you should edit the gridcli or gridcli.bat if by default your system uses an unoffical version of Java.

 

Running the client

To test the client, try running it with the following command, from within the installation directory:

Windows
c:\grid-client-<version> gridcli
Linux
$ ./gridcli

Note that you must use an official Sun version of Java. On systems which default to their own version (such as Fedora Core), you may need to edit the gridcli script to run the Sun version rather than the system default. For example, if you have installed java in /opt then modify

java -Dgrid.config.dir=$CLIDIR/conf -jar $CLIDIR/lib/itinnov-grid-client-cli-4.jar "$@"
to
/opt/java/bin/java -Dgrid.config.dir=$CLIDIR/conf -jar $CLIDIR/lib/itinnov-grid-client-cli-4.jar "$@"

Creating an alias

You'll probably want to use the gridcli command from other directories. To do this:

Windows
Copy gridcli.bat into a directory in your path and then edit gridcli.bat to set the original installation path as explained in the file. Alternatively, set your Path system variable (Control Panel -> System -> Advanced -> Environment Variables).
Linux or Unix
Copy the shell script called gridcli to your /usr/local/bin directory and edit it as described in the comments inside the file. You may have to type rehash before your shell will notice the new command.

Configuring a web-proxy

In some network configurations, the site firewall prevents you from connecting to services directly. Instead, all requests must go via a proxy, which is set up by your network administrator. In this case, you need to edit the conf/HTTP-proxy.properties file and set the indicated values.

If you are not sure whether you are using a proxy, consult your network adminstrator. You may be able to find out whether you are using a proxy by checking your browser's settings:

Checking settings (Firefox browser)

  1. Open then Edit menu and choose Preferences (use Tools -> Options on Windows).
  2. In the General section, click on Connection Settings....
  3. The hostname and port for SSL Proxy are the values you need.

3. Client User's Tutorial

How to use the client

Introduction

This tutorial shows how to use the basic GRIA application services (the Job and Data services) as a client.

Running the client

Once you have installed the client, run it to open the Grid resource browser:

> gridcli

The initial display will not show any resources or services. The main client window is made up of three main areas.

  • The area on the left of the client is the resource browsing and this area allows you to view the relationships between resources and between services and resources. The are currently two views that can be used :
    • Service view is a tree view displaying resources under the service at which they are created.
    • Management view is a tree view displaying resources under the resource which they are manged by
  • The area on the right of the client (initially displaying the GRIA logo) is the resource viewing area which allows more detailed information about a resource or service.
  • The lower area on the client is the logging and information area.

 

Initial resource browser window

Adding services

To add a service you need to know the URL of its WSDL, which describes what the service can do and how to access it. Typically, your service provider will have a public web page containing links to the WSDL of each service they run.

The GRIA Demo Services (https://griademo1.it-innovation.soton.ac.uk) WSDL's are :

The easiest way to add the service is to drag the link from your web-browser into the the Grid client application's list of services:

Add a service using drag-and-drop

If you can't do this (e.g., because you have been given the WSDL URL on a piece of paper) then you can add the service using the menu instead:

  1. Select the Services menu from the top menu bar.
  2. Choose Add a service.
  3. Enter the URL of the service's WSDL.

After adding a service it will appear in the list and the client will contact it to find any resources at the service to which you have access. If you haven't used the service before then typically there won't be any, unless someone has granted you access to some resources already or there are public resources at the service.

Add at least one JobService and at least one DataService now. If you don't know of any existing service providers you can access the Basic Application Services package on the GRIA demo systems or install the package yourself.

 

Management requirements

Services may be set up so that you don't require any pre-existing agreement in order to use them, or they may require you to have a service level agreement with the supplier first. There are three cases:

  1. The service can be used without any pre-existing agreement:

    single domain

  2. The service requires an account or Service Level Agreement (SLA), and you have direct access to this:

    Gria Client Trusted Domain

  3. The service requires an account or Service Level Agreement (SLA), and you have access to a local client management service which manages your access to these resources for you:

    federated domain

To find out how the service you are using has been configured, try creating a new data stager now:

  1. Right-click on the data service you added above and choose Create New data stager.
  2. Enter a name for the new stager if permitted.

Creating a data stager

When you do this, the client queries the data service to find out what it needs in order to use the service. The service may reply that it needs an SLA at the service provider's SLA service(s), or it may report that no pre-existing resource is necessary to create new stagers.

If a new stager is created, then the service does not require an agreement before use. You can skip the next section and jump to Using a data stager.

Otherwise, you should see a message dialog box explaining that you need an SLA (Service Level Agreement) in order to use the service. The next section explains how to get an SLA.

Getting access to an SLA

If you try to use a service and get told that you need an account or SLA then follow these instructions to get one. Normally SLAs are created by a different person in your organisation and you are granted access to them. There are three ways to get access to the required resource:

You are responsible for making SLA agreements

Follow the client management guide to create the required SLA.

Someone else agrees SLAs for you and there is no client management service

If someone else in your organisation is responsible for agreeing SLAs then ask them to create one (by following the client management guide) and give you access to it. They will tell you to add the SLA service to your client, at which point the client will discover the SLA. If you have already added the SLA service to the client before you had access to the SLA, right-click on the service and choose Discover existing resources to check again. If you have more than one SLA valid at a supplier, you can make the client prefer one of them by right-clicking on it and choosing Set as Default SLA from the menu. The SLA will go red to indicate that it is the default.

Your organisation uses a client management service

Ask the person in your organisation who is responsible for agreeing SLAs to create one (by following the client management guide) and give you access to it. They may tell you to add your organisation's local client management service to your client, at which point the client will discover your private account. If you have already added the service to the client before you had access to the private account, right-click on the client management service and choose Discover existing resources to check again. Right-click on the private account and choose Set default private account from the menu. The account will be shown with a tick mark:

Once you have followed one of the three routes above, you will be able to create new data stagers.

Using a data stager

You should now be able to create new data stagers:

  1. Right-click on the data service and choose New data stager.
  2. Enter a name for the new stager.

A new data stager is created and appears in the display. You can use the popup menu to upload or download data, to transfer data to or from another stager, or to grant others permission to read or modify the contents. On some systems, you can also upload data by dragging a file from your file manager to the data stager, though this is not currently available on Microsoft Windows systems.

To view more details about a Data Stager, click on it and its details will appear on the right hand side of the client in the resource viewing area.

Managing a data stager

You can copy data between stagers directly, without having to download from one and upload to another. To try this:

  1. Create another data stager, as described above.
  2. Drag the source data stager onto the new stager.

The client automatically opens an authorisation on the target stager, allowing the source stager to write to it.

To download the data from the target, bring up the menu over the target stager and choose Download data, then choose where to save it on your local computer.

Delegating access to a data stager

You can also give other users read or write permission on a data stager. For example:

  1. Click on the data stager to show it in the resource viewing area.
  2. Switch to the Access Control tab.
  3. Click 'Load Access Control Rules' button. This shows the current rules for this Data Stager. If you just created this stager it should show a rule giving you the owner role on the reource.
  4. Click on Add Rule to show the Access Control Wizard
  5. Choose a process role from the first drop down menu, for a Data Stager there should be a reader role.
  6. Choose a rule type from the second drop down menu.
  7. Choose the Subject DN is... option box
  8. In the dialog that appears open the certificate file (.crt) for the user you want this rule to apply to.
  9. In the second dialog that appears open the certificate file (.crt) of the issuer of the users certificate.

Granting access to a user

Access can later be revoked by choosing the access control rule in the table and clicking the Remove Rule button.

To use the data stager, another user will also need to discover the stager. They will right-click on the data service and choose Discover existing resources.

Note: having read or write access does not give you permission to grant read and write access to others. Since transferring data between stagers requires you to grant permission for one of the services to use the other, you will need to invoke the appropriate copy operation on whichever data stager you don't own:

  • If you own the source stager, perform a Copy from operation on the destination. The client will grant the destination read access on the source.
  • If you own the destination stager, perform a Copy to operation on the source. The client will grant the source write access on the destination.

Creating a job

Ensure you have a job service shown in your client (see Adding Services if not). Then create a new job:

  1. Right-click on the job service and choose New job.
  2. The client will check the service's policy. If you need an SLA to create a job and you don't have one you will get a message saying so. See Management requirements above in this case.
  3. The client will query the job service to find out which applications it supports. Choose one from the list.

Gria Client Create Job

The new job appears appears in the window under the job service. Under the job itself more resources appear, one for each input and one for each output. Inputs and outputs are just data stagers, and you can use the menu to upload the inputs for the job in the normal way. You can also transfer the data from another stager (including transferring from the output of another job).

Running a job

Once the inputs are uploaded, you can start the job running by right-clicking on it and choosing Start job from the menu or you can start the job and monitor its progress by choosing Start Job and Monitor. You will be prompted to enter any arguments for the job:

The sample applications don't take any arguments, so you can just click on OK to start the job. If you choose Start Job and Monitor a job progress monitor window appears in the resource viewing area.


If the job will run for a long time you can resume monitoring it later, by clicking on the job and selecting the Job Monitor tab.

Once the job has finished, you can download the result from the output(s) using the output's popup menu, as for a normal data stager.

4. Managing users and suppliers

Managing users and suppliers

Introduction

Services may be set up so that users don't require any pre-existing agreement in order to use them, or they may require a service level agreement with the supplier. When a user tries to access a service that requires an existing Service Level Agreement (SLA) they will get a message saying something like this:

None of the known trade accounts or SLAs are suitable for this service (no
local private account service is being used). An account or SLA is required at
one of these services:

- https://management.example.com/gria-service-provider-mgt/services/SLAService

From host: apps.example.com

The user should now talk to the person in their organisation who is responsible for setting up agreements with suppliers. This guide assumes that that person is you. The topics covered here are:

  1. An overview of trade accounts and service level agreements.
  2. How to set these up using the GRIA client.
  3. Granting other employees of your organisation access to SLAs.
  4. Using a client management service to manage large numbers of trade accounts and SLAs centrally.

Trade accounts and SLAs

An SLA is an agreement between two organsiations (a client and a service provider) stating what resources will be provided and what use of the service will cost. For example, an SLA may state that:

  • Up to 1 Tb of data may be uploaded per month, at 1 euro per Gb.
  • Up to 30 jobs may be running at the same time, at 1 euro per CPU hour.

When a user uploads data or starts a job, they indicate which SLA they are using, and the usage is recorded against this SLA. Periodically, usage on the SLA is converted into monetary terms (according to the terms of the SLA) and recorded on your trade account.

You will need a trade account at each non-free supplier you wish to use, and at least one SLA billed to each trade account. You can then grant other people permission to use resources under the terms of the SLA.

Running the client

You should already have the GRIA client installed. If not, consult the Client Installation first. Then run the client to open the Grid resource browser:

> gridcli

The initial display will not show any resources or services unless you used the same client before to access other services:

Gria Client Empty

Adding the Trade Account and SLA Service

Go to the web-site of the service provider you wish to use and follow the Adding Services guide to add their TradeAccountService and SLAService now. You should see some SLA templates listed under the SLA service. An SLA template is a set of terms that you must agree to in order to create an SLA.

Opening a trade account

Click on a trade account service (as added above) and a form appears in the resource viewing panel on the right hand side of the client :

Your new trade account will appear under the supplier's service. You can click on it and details will appear in the resource viewing panel, from here you perform actions on the trade account. Initially, the account's status will be pending-credit-checks; the account can't be used yet. Once the service provider has approved the account, its status changes to open. You can also use this dialog box to check the account statement, which will show any spending on the account:

Creating an SLA

Once your account is in the open state, you can use it to create SLAs. Click on one of the SLA templates discovered when you added the SLA service (to check for templates published after you added the service, right-click on the service and choose Discover existing resources from the menu). You will see the details of the service provider's offer:

After examining the available templates, pick the one(s) you want and click on the Propose SLA button to create an SLA. If accepted by the service provider, a new SLA resource will appear under the SLA service:

Granting access to an SLA

Click on an SLA to open the Properties dialog box. This reminds you of the details of the agreement, provides graphs showing usage, and lets you control access to the SLA:

 

First load the current access control rules by clicking Load Access Control Rules. This shows a list of rules that apply to the current SLA, initially this will show one rule granting you the owner role on the SLA. To grant other users access to the SLA, choose Add Rule from the menu. You will be prompted with the Access Control Wizard similar to the one used to delegate access to a Data Stager in the previous section Client User's Tutorial.

Gria Client Access Control Wizard

Users do not have full access to the SLA - they can use resources at other services that require an SLA but they can't close it or grant access to others, for example.

Viewing usage on an SLA

As people make use of services using the SLA, the SLA service records the usage. You can view graphs of the usage using the client:

  1. Cick on the SLA to show its details in the resource viewing panel.
  2. Go to the Usage tab.
  3. Select the time period you wish to view, leaving the fields as the default gets usage from the start of the SLA until the current time.

You can view different metrics that the service has been keeping track of, you can view Number of Data Stagers, Number of Activities, Amount of Disc Space Used. Other services may report different metrics depending on the services they monitor.

Note that usage within the last couple of minutes may not be shown and that this view is a summary of usage, you can click on the load button to view raw usage, which gets all usage for the current metric chosen.

Client Management

Granting users access to trade accounts and SLAs individually becomes more difficult as the number of users and suppliers increases. Each time a user joins a project they must be given access to every SLA. Every time a new supplier is added, every user must be given access to it.

The solution is to run a client management service within your organisation. This service keeps track of who is a member of which projects, and which SLAs each project uses. Installation of the client management service is covered in the Client Management Service Overview. The following sections assume that the service is already installed.

It is recommended to use the Membership and Registry Service to manage users and resources, but you may wish to use the a Private Account Service which is described below Using Private Accounts but this service is deprecated and users are not advised to use it.

Creating a Membership Group

You can use the membership service to control groups of users.

  1. Adding the membership service is done in the same way as adding the other services.
  2. Right-click on the membership service and choose  'Create Group' from the menu. Choose a name for the group, the name should signify the privileges that users get if the are a member of this group, i.e. 'Engineers'

 

Adding Members to a Group

  1. Click on the membership group
  2. Go to the Access Control tab and click on Load Access Control Rules. Click the 'Add Rule' button.
  3. You will be prompted with the Access Control Wizard similar to the one used to delegate access to a Data Stager in the previous section Client User's Tutorial
  4. More members can be added in the same way.

     

    Giving Members access to a Resource

    Once you have created your membership group and added all the members to it you need to give them access to resources.

    To give the users access to an SLA, drag the sla onto the membership group in the client.

     

    Then choose a role for which you want members of this group to have for this resource.

     

    Doing this adds a rule to the SLA's Access Control Rules which gives anyone bearing a token asserting they are a member of the group the chosen role on the sla. We can see this new rule on the Access Control tab of the SLA:

     

    Using a Membership Group

    If you have been given access to a Membership Group, then follow these steps to use it
    1. Adding the membership service is done in the same way as adding the other services
    2. Right click on the Membership Service and choose Discover Existing Resources. If your have been given access to a Membership Group it should appear in the client
    3. Right click on the new Group and choose Set as default Group. Now when accessing services a token from this membership group will be attached to the request to authorise you.

    Creating a Registry Resource

    1. Adding the registry service is done in the same way as adding the other services.
    2. To create a new Registry you need to have been given the 'manager' role on the Registry Service.
    3. Click on the registry service and create a new registry resource by right-clicking the Registry Service and selecting the 'Create New Registry' option:

       

      Adding a Resource to a Registry

      1. Click on the Registry Resource in the client.
      2. Click on the 'Resources' tab on the resource viewing panel.
      3. Click on the 'Load Resources' button to load the current resources into the table. The table should be empty if you just created the registry.
      4. Click on the 'Add Resource' button and choose a resource to put in the registry.

       

      The resource should appear in the table

       

      Using a Registry

      If you have been (perhaps by your project manager) given access to a Registry then follow these steps to use it
      1. Adding the registry service is done in the same way as adding the other services.
      2. Right click on the Registry Service and choose Discover Existing Resources. If your have been given access to a registry it should appear in the client
      3. Right click on the new Registry and choose Discover Registered Resources. This will add to the client all the resources that you have access to, if you set a default membership group then it will use a token from that group as authorisation.
      4. If you right click the registry and choose Set as default registry then the registry can be used to select an appropriate SLA when creating resources on managed services.

      5. Resources Views and Filters

      A description of the features of the GRIA Client that allow the user to customize the view of services and resources

      Resource Views

      In the current GRIA Client there are two views which display services, resources and the relationships between them. Each resource also has an associated state which is displayed beside the resource. The state can be updated by right clicking a resource and selecting Update EPR. If you get an error that looks like this

      uk.ac.soton.grid.pbac2.pdp.InvalidActionException: Unknown action 'getEPR'
      
      From host: remotehost
      

      then the service does not support this functionality. You can refresh the state of all services by selected Refresh Resource EPRs from the Service menu.

      Service View

      The service view displays resources at the service at which they are located. The layout of this view is a tree and each top level item in the tree is a service, and the leaves of each service represent a resource at that service. The relationship is an owner relationship which means the services own each of the resources. This applies to most resources one exception being Data Stagers which have been created in the context of a Job, these are displayed under the job.

      The resources are displayed under the service at which the are located. The data stagers belonging to a job appear under the job.

      Management View

      The management view displays resources under the resource which they are managed by. This relationship is a parent-child relationship and this can be seen in the resources endpoint reference, right click a resource and select View EPR there is a managingresource element which tells us the endpoint of the managing resource. If a resource is unmanaged then it appears under the Management Resources node otherwise it appears under its managing resource.

      There are seven unmanaged resources

      • Group1 , Trade Account 1, Trade Account 2, , Unlimited Package, paint job1 and reg1 are all unmanaged

      There are five managed resources

      • SLA for Data and Job Package is manged by Trade Account 2
      • SLA for Unlimited Package 1 is manged by Trade Account 2
      • Data1 is managed by SLA for Unlimited Package 1
      • painted-image and source-image are managed by paint job 1

      The managing view also displays a list of known services so that there is access to the service functionality.

       

      Unknown Resources

      If a resource appears under a Services Icon Icon and resource URL similar to the following pattern  'https://service:8443/gria-service-provider-mgt/services/SLAService#1234-1234-1234-1234'

      then the resource is managed but the managing resource is unknown. Try adding the the service by choosing Add a Service from the Services menu and entering the WSDL of the service

      i.e. https://service:8443/gria-service-provider-mgt/services/SLAService?wsdl

      Resource Filters

      Your can customize the display of the client by filtering the resources and services you wish to view. Filters are accessed via the Filters menu on the main menu bar of the client. The GRIA client comes with some built in filters.

      Built in Filters

      • Show only active SLAs - this filter hides all inactive SLAs.
      • Show only full data stagers - this filter hides all empty data stagers.
      • Show only empty data stagers - this filter hides all full data stagers.

      Each filter can be started by choosing it from the Filters menu and selecting active. Filters are not exclusive so two can be active at once, this can cause problems if for example we turn on Show only full data stagers and Show only empty data stagers which means no data stagers would be shown in the client.

      Template Filters

      There are also some template filters included with the client which can be setup

      • State Filter - allows the user to filter a resource based on its state
      • Host Filter - allows the user to filter a resource or service based on the host at which it is located

      A new filter is setup by choosing Add New Filter from the filters menu. Each filter must be configured before it is used

      • State Filter - the type of resource and state must be set
      • Host Filter - the host must be set

      Once a name is choosen for the filter then it is added to the Filters menu and activated

      6. Contextualised Discovery

      The Client Management Service provides possibilities for contextualised service discovery.

      6.1. Background - Client Management Registry Service

      Background of Client Management Registry Service (CltMgtRS).

       

      The Client Management Registry Service (CltMgtRS) is part of the Client Management Component of GRIA. This service allows contextualized discovery across multiple resources based on metadata. The focus of the CltMgtRS is to support users in querying resources based on management contexts like SLAs or Trade Accounts. For instance users are able to ask queries like “Find SLA for application <name> where <CPU seconds of SLA> is greater than 1000 and <usage of SLA> is lower than 500”.

      The context information is defined within the Registry Domain Model (RDM) of the CltMgtRS. This model specifies concepts and relationships between concepts used within the Client Management Registry. XML documents are stored and can be retrieved with respect to these concepts. Relationships define dependencies between concepts that can be used for specifying join queries and sub queries. A special is-a relationship is introduced for specifying hierarchies of concepts. Sub concepts inherit relationships from their super concepts. Examples of concepts are SLAs, usage reports or endpoint references (EPRs); the relationship hasUsage can for instance combine usage reports with the SLAs they belong to.

       

      RDM of the Client Management Registry Service

       

      RDM of the Client Management Registry Service

       

      Queries against the CltMgtRS can be formulated using XQuery directly, but it is recommended to use ooXmlQL queries for ease of use. ooXmlQL is a query language designed to supports join queries and sub queries based on the underlying RDM. Thereby the selection and filter statements (where clause) are language independent, and can for instance be defined by XPath or XQuery. The specification of ooXmlQL can be found in the Appendix.

      6.2. Client Management Registry Client

      User guide for the Client Management Registry Client.

      Introduction

      This tutorial shows how to use the Client Management Registry Client.

      Creating a Registry Resource

      Before you can register services and other resources to the CltMgtRS, you have to create a concrete Registry Resource, in short a Registry, which is an instance of the CltMgtRS.

      1. Adding the registry service is done in the same way as adding the other services. See Adding Services.
      2. Click right on the registry service to open a popup menu and create a new registry resource by clicking the ‘Create new registry’ menu button:

       

      Creating a new Registry

       

       

       

      Adding a Resource to a Registry

      Resources (EPRs) can be registered with a concrete Registry using drag-and-drop on the service tree or by using the Registration interface at the ‘Resources’ tab

      1. Click on the Registry Resource in the client.
      2. Click on the 'Resources' tab on the panel on the left.
      3. Click on the 'Load Resources' button to load the current resources into the table.
      4. Click on the 'Add Resource' button and choose a resource to put in the registry.

       

      Adding resources to the registry

       

       

       

      The user of the registry is able to register EPRs of resources. However in order to make the concepts and relationships of the RDM usable, different strategies are implemented to extract additional knowledge from or related to the EPR. For instance, adding a concrete job results in adding the metadata of the application, the parent job service with its WSDL document, as well as the resources managing this job service.

      N.B. when registering job resources created at a 5.0.1 Job Service, the 'applicationuri' meta information is not part of the job's EPR, therefore querying for jobs supporting a specific application like "paint" will not work correctly. This is true for all example queries containing the concept 'Application'.

      After adding the resource should appear in the table, however, if you press ‘Load Resources’ again, you should also see resources that are inserted behind the scene.

       

      Load all registered resources

       

       

      Setting up the Registry for the following tutorial

      In order to follow the tutorial used in this guide you have to setup the client as follows:

      1. Add a Data Service and a Job Service to the list of services. These services should be managed by an SLA Service and a corresponding Trade Account Service.
      2. Add the SLA Service and the Trade Account Service to the list of services. Propose an SLA.
      3. Create a Registry resource.
      4. Create a data stager and a job.
      5. Register the data stager, the job and the proposed SLA with the Registry.

      The resulting list of services should look like this:

      Service tree

       

       

      Discovering Registered Resources

      Discovery is the process of finding entities previously registered. The ‘Resources’ tab provides a simple discovery mechanism that loads all registered resources into the client. The ‘Discovery’ tab provides more sophisticated discovery possibilities using ooXmlQL queries.

      In the top of the Discovery panel you have a menu that provides possibilities to load/save queries (‘File’ menu), to create new queries using a Query Wizard (‘Wizard’ menu) and to create queries based on predefined examples (‘Examples’ menu).

      Note: If you execute some examples by submitting the query without setting up the registry for the examples, it might happen that they produce an error, due to missing concepts in the RDM. This is correct since some concepts are added dynamically to the RDM when concrete resources are registered, like the concept ‘Application’ or different sub concepts of the concept ‘Reference’.

      'Discovery' tab

       

       

      In order to create your own queries, you can use the Query Wizard, explained in the next paragraph, modify provided examples or type in ooXmlQL queries directly into the corresponding text field. The ‘refresh view’ button allows you to recolor the XML-based ooXmlQL query, as well as it provides you with the information if the XML-based query is well-formed. It doesn’t check the syntax of the query at the moment.

      After submitting the query to the registry resource (using ‘submit query’ button) the resulting resources are displayed in the resource view:

      Discovered resources and additional documents

       

       

      From here you can select discovered resources and add them to the list of services (using ‘add to services’ button). If any additional documents are discovered you can view them in an XML viewer (using ‘show additional documents’ button):

      View of additional documents

       

       

      Using the Query Wizard

      The Query Wizard supports the user in creating ooXmlQL queries without knowing the syntax of ooXmlQL at all. The only part of the query the user has to write down manually is the where clause, where the XPath or XQuery filter goes in. Users can open the wizard in selecting the ‘Wizard’ menu and pressing ‘Open Query Wizard…’.

       

      Specifying the 'From' part of the query

       

      The Query Wizard pops up with the ‘From’ part specification of the query. The ‘From’ part in ooXmlQL has the same function than ‘from’ in SQL statements. From is used to specify the concept or a set of concepts of interest.

      Specification of the 'from' part

       

       

      First a requested concept and a variable for this concept have to be specified. This can be done by pressing the ‘select concept’ button. This button opens up a Concept Browser, where users can select the appropriate concept.

      Note: the concepts shown in the Concept Browser are discovered dynamically from the registry service. Therefore the number of concepts can change when new concepts are generated during registration.

      Concept browser

       

       

      After specifying a concept the corresponding fields are filled in and a variable will be suggested, if not provided beforehand. From here, users can continue creating the query pressing the ‘next’ button or they can extend the ‘From’ part by adding set operations and other concepts to the ‘From’ part using the ‘+’ button, e.g.:

      Building combined concepts using set operations

       

       

      The registry provides three different types of set operations: union, intersect, except, which can be specified by the corresponding combo box before pressing the ‘+’ button. The ‘-‘ button allows to remove a concept from the specification.

      After pressing the buttons ‘next’, ‘+’ or ‘-‘the user can see a preview of the resulting ooXmlQL on the right of the Query Wizard window.

      Tutorial: The query we will define is, ‘find SLA and its managed service, where the usage of this SLA on the metric ‘disk space’ is not greater than 0.0 and the service contains the term ‘Job Service’ in its address’.

      1. Open the Query Wizard.
      2. Select concept ‘ReferenceSLA’ and add it to the set of concepts by pressing ‘+’.
      3. Press ‘next’.

       

      Specifying Joins

      The next step in the Query Wizard after pressing the ‘next’ button is the definition of joins if this is required:

      Specification of joins

       

      Joins provide the possibility to navigate in the RDM using defined relationships between concepts.

      Note: depending on the specification of the ‘From’ part, e.g. when using set operations, it could happen that no relationships are available for building joins. For instance, the concept ‘Reference’ and the concept ‘Usage’ do not share any common relationship. Therefore combining them in the ‘From’ part will result in:

      Specification of joins is not possible

       

      The specification of a join is as follows:

      1. Select the relationship for navigation
        Join: select relationship

      2. Select sub concepts if required and specify a suitable variable for this concept
        Join: select sub concept

      3. Add additional joins from the concept selected beforehand by pressing the ‘+’ button
        Join: add additional join

      4. It is further possible to specify restrictions on a join concept using the ‘set / edit sub query’ button. This opens a new dialog window for specification of the sub query.
        Note
        : currently specified sub query cannot be edited afterwards. Creating sub queries is explained in a latter paragraph.

      The result should look like:

      Specification of a query navigation path using joins

       

       

      Tutorial:

      1. Select relation ‘holdBy’ and select sub concept ‘Sla’. Add the variable name ‘$slaContent’. Press ‘+’. (Due to the ‘From’ part, we are only interested in ReferenceSLAs hold by SLAs.)
      2. Select relation ‘hasParent’ and select sub concept ‘ManagerResource’. Add the variable name ‘$manager’. Press ‘+’. (An SLA is a child of a managing resource, a SLA Service.)
      3. Select relation ‘manages’ and add variable name ‘$managed’. Press ‘+’. (What resources are managed by this service?)
      4. Select relation ‘has’ and add variable name ‘$epr’. (What are the References of these managed resources?)
      5. Press ‘next’.

       

      Declaration of namespaces and variables

      After pressing the ‘next’ button on the ‘Join’ view users are able to declare variables and namespaces. Namespaces are required in where clauses when documents that should be filtered use namespaces. In order to support the user several namespaces are predefined. Variables can be used for instance to support aggregate functions used within a where clause. The ‘+’ button allows to specify a new namespace or variable respectively. If the user ticks the check box in the ‘use’ column the corresponding namespace or variable will be included into the query.

      Tutorial:

      1. Select the namespaces ‘gria’, ‘addressing’ and ‘metric’. (Used in the where clause later on.)
      2. Press ‘next’.

       

      1.1.1.      Specifying the where clause

      Specifying the where clause in XPath or XQuery has to be done manually. Currently there is no support in fulfilling this task. In the where clause user can use all variables defined in the scope of the actual query, but not on variables defined in sub queries or the other way around. Variables that can be used include the variable used in the ‘From’ part, the variables of the ‘Join’ and the variables explicitly declared in the namespace and variable declaration step.

      Specification of the where clause

       

       

      Additionally a sub query restriction can be defined on the concept of the ‘From’ part of the query, if the specification of the ‘From’ parts provides a common concept (see paragraph ‘Specifying Joins’). If not, the corresponding button is not selectable.

      Tutorial:

      1. Add where clause:
        fn:contains($epr//addressing:Address,"JobService")
      2. Press ‘next’.

       

      Selecting variables of the results

      In a final step users are able to select the elements of the result using multiple selections on the variables. This can be done by holding Ctrl or Shift during selection. If no variable is selected, elements of the variable of ‘From’ part are automatically assumed as the result.

      Selecting variables for the result

       

       

      Tutorial:

      1. Select variables ‘$referencesla’, ‘$slaContent’ and ‘$epr’.
      2. Before we finish this tutorial, we want to define a restriction on the usage reports of the SLA. Press ‘back’ until you reach the view where the ‘Join’ is specified. If you go to ‘From’ part of the query, you have to start from the beginning.
      3. Press ‘set / edit sub query’ at the relation ‘holdBy’ on concept ‘Sla’.


      Specifying restrictions on (join and from) concepts

      The ‘from’ concept as well as the ‘joins’ concepts can further restricted using sub queries. When pressing ‘set /edit sub query’ a new window pops up allows the specification of restrictions on the corresponding concept:

      Specification of restrictions on a concept

       

       

      1. Select a relationship (‘return’) that provides the domain concept you want to restrict (left component in the brackets).

        Selecting returned concept using a specific relationship

      2. Select a sub concept if required.
      3. Press ‘+’ button to add the restriction to the set of restrictions.

        Add restriction specification to set of restrictions

        This will open up a new Query Wizard dialog that allows specifying the sub query. The specification is done similar to the specification of the main query; beside there is no possibility to select variables for the result of the sub query.
      4. After finishing the declaration of the sub query, the wizard will return to the restriction dialog. Here it is possible to define more sub queries and combine them using the Boolean expressions ‘and’ and ‘or’ as well as invert restrictions pressing the ‘not’ checkbox.

        Finished sub query restriction

      5. Press ‘finish’ to finish the restriction specification and return back to the parent query, e.g. the main query. Press ‘cancel’ to discard the whole restriction.

       

      Tutorial:

      1. Select return ‘isUsageOf’ and press ‘+’. (Restrict SLAs by usage information.)
      2. In the new Query Wizard dialog, add variable ‘$usage’ in the ‘From’ part of the sub query.
      3. Press ‘next’ until you reach the where clause specification.
      4. Add the where clause:
        $usage//metric[@description="disk space"]//rate > 0.0
      5. Press ‘finish’.
      6. Select ‘not’ on ‘Usage’ return=’isUsageOf’. (Remark, in this example we could use ‘not’ directly within the where clause, like:
        not ($usage//metric[@description="disk space"]//rate > 0.0)
        However, ‘not’ in the interface allows you easily to invert whole Boolean expressions based on several sub queries.)
      7. Press ‘finish’.
      8. Press next until you reach the ‘Select’ view. (Otherwise the previous specifications of steps done beforehand are gone.)
      9. Press ‘finish’.

       

      Wizard result

      After pressing the ‘finish’ button in the Query Wizard the resulting query will be inserted into the ooXmlQL query field. Here the user can finally do some modifications and save the query to the file system using the ‘File’ menu.

      Submitting a query to the registry

       

      Tutorial:

      The resulting query should look like:

      <query>
        <select>$referencesla $slaContent $epr</select>
        <from as='$referencesla'>
          <class name='ReferenceSLA'/>
          <join on='holdBy' as='$slaContent' class='Sla'>
            <not>
              <query return='isUsageOf'>
                <from as='$usage'>
                   <class name='Usage'/>
                </from>
                <where>
                   $usage//metric[@description="disk space"]//rate > 0.0
                </where>
              </query>
            </not>
            <join on='hasParent' as='$manager' class='ManagerResource'>
              <join on='manages' as='$managed' class='ManagedResource'>
                <join on='has' as='$epr' class='Reference'>
                </join>
              </join>
            </join>
          </join>
        </from>
        <declare name='addressing'>
          http://www.w3.org/2005/08/addressing
        </declare>
        <declare name='gria'>
          http://it-innovation.soton.ac.uk/2005/grid
        </declare>
        <where>
          fn:contains($epr//addressing:Address, "JobService")
        </where>
      </query>

       

      1. Submit the query to the registry. The result should contain the SLA endpoint and the Job Service endpoint as well as the SLA as one additional document, the SLA content.
      2. Save the query using the ‘File’ menu for latter usage.
      3. You can reload this query latter on using the ‘File’ menu.

      6.3. Appendix

      This appendix contains information about the query language ooXmlQL

       

      ooXmlQL Specification

      The following EBNF grammar specifies sound ooXmlQL queries. Remark: A strange behaviour can be observed when using variables where one variable is a prefix of another variable, e.g. $ref and $refAlias. It seems there is a bug in the XQuery processor. Therefore this variable setting should be avoided.

       

      ooXmlQuery = query.
      query = ‘<query>’ ( queryelements ) ( declareNS )* ‘</query>’
      queryelements = ( select )?
                      from
                      ( declareVar )*
                      ( restriction )?
                      ( where )?.
      select = ‘<select>’ ( ‘*’ | xpath expression+ ) ‘</select>’.
      declareNS = ‘<declare name=“’name’”>’ namespace ‘</declare>’.
      from = ‘<from as=“’var‘”>’ ( class | setOperation ) ( join )? ‘</from>’.
      setOperation = ‘<union>’ ( class | setOperation )+ ‘</union>’ |
                     ‘<intersect>’ ( class | setOperation )+ ‘</intersect>’ |
                     ‘<except>’ ( class | setOperation )+ ‘</except>’.
      class = ‘<class name=“’classname’”/>.
      join = ‘<join on=“’relation’” as=“’var’” class=“’class’”>’
                  ( restriction )? ( join )?
             ‘</join>.
      declareVar = ‘<declare var=“’var’”>’ declaration ‘</declare>’.
      where = ‘<where>’ xpath or xquery expression ‘</where>’.
      restriction = subquery | and | or | not.
      subquery = ‘<query return=“’relationship’”>’
                   queryelements
                 ‘</query>’.
      and = ‘<and>’ restriction+ ‘</and>’
      or = ‘<or>’ restriction+ ‘</or>’.
      not = ‘<not>’ restriction ‘</not>’.

       

      Variables can be used in

      • where clause
      • declare variable clause
      • select clause

      Example ooXmlQL query:

       

      <query>
        <select>$sla</select> 
      
        <declare name="ns1">http://it-innovation.soton.ac.uk/2005/grid</declare>
        <declare name="ns2">http://www.w3.org/2005/08/addressing</declare> 
      
        <from as="$sla">
          <class name="Sla"/>
          <join on="provides" as="$res" class="Resource"/>
        </from> 
      
        <declare var="$min">fn:min($sla//cpu)</declare>
        <declare var="$max">fn:max($sla//cpu)</declare>
      
        <query return="providedBy">
           <from class="Resource" as="$res"/>
      
           <declare var="$avg">fn:avg($res//cpu)</declare>
      
           <where>
             $res//ns2:Metadata/ns1:type =
             "uk.ac.soton.ecs.iam.grid.comms.client.RemoteDataService"  
           </where>
        </query>
      
        <where>$sla//cpu = $min or $sla//cpu = $max</where>
      </query>