This documentation is for WSO2 Enterprise Integrator version 6.2.0 . View documentation for the latest release in the 6.x.x family and the latest release in the 7.x.x family.

All docs This doc

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

data service provides a Web service interface to access data that is stored in relational databases, CSV files, Microsoft Excel sheets, Google spreadsheets, and more. The following sections describe how to you can use WSO2 EI Tooling to create artifacts for data integrationwork with data services' artifacts

Table of Contents
maxLevel4

Prerequisites

Creating a data service project

  • To demonstrate how data services work, we will use a MySQL database as the datasource. Follow the steps given below

...

  1. You can either use the link on the dashboard (click Developer Studio -> Open Dashboard) as shown below or use the File-> New -> Other -> Data Service Project option. 
  2. On the window that appears, specify a project name. 
  3. If you want to customize the Maven options (such as including parent POM information in the file from another project in this workspace), click Next and specify the options.
  4. Click Finish.

Once you have created the Data Service product, you can create multiple data services and add them to the same project. For example, if you create a data service project called DSS_Project, the folder structure will be as follows:
Image Removed

Creating a data service

A data service provides a Web service interface to access data that is stored in relational databases, CSV files, Microsoft Excel sheets, Google spreadsheets, and more. You can create a new data service following the steps given below. Alternatively, you can import an existing data service using the data service descriptor (DBS) file.  

Step 1: Creating the datasource connection

Follow these steps given below.

...

  • to set up a MySQL database:

    1. Install the MySQL server.
    2. Download the JDBC driver for MySQL from here and copy it to your <EI_HOME>/lib directory.

      Note

      If the driver class does not exist in the relevant folders when you create the datasource, you will get an exception such as 'Cannot load JDBC driver class com.mysql.jdbc.Driver'.

    3. Create a database named Employees.

      Code Block
      CREATE DATABASE Employees;
    4. Create the Employee table inside the Employees database:

      Code Block
      USE Employees;
      CREATE TABLE Employees (EmployeeNumber int(11) NOT NULL, FirstName varchar(255) NOT NULL, LastName varchar(255) DEFAULT NULL, Email varchar(255) DEFAULT NULL, Salary varchar(255));

Creating a data service

Follow the steps given below to create a new data service.

Step 1: Creating a data service project

All the data services' artifacts that you create should be stored in a Data Service project. Follow the steps given below to create a project:

  1. Open the Developer Studio Dashboard (click Developer Studio > Open Dashboard) and click Data Service Project.
  2. Enter a name for the project and click Next.
  3. If required, change the Maven information about the project.
  4. Click Finish. The new project will be listed in the project explorer.

Step 2: Creating the datasource connection

Follow the steps given below to create the data service file:

  1. You can start creating a new data service in two ways:
    1. Open the dashboard (click Developer Studio -> Open Dashboard) , and click click Data Service.  
    2. Select the already created Data Service project on Project in the navigator, right click and click go to New -> Data Service
    Select
    The New Data Service window will open as shown below.
    Image Added
  2. To start creating a data service from scratch, select Create New Data Service and click  and click Next.Image Removed

  3. Type unique names Enter a name for the project and data service.:

    Data Service NameRDBMSDataService
  4. Click Click Next and start adding the datasource connection the datasource connection details given below.

    Datasource IDDatasource
    Datasource TypeRDBMS
    Datasource Type (Default/External)Leave Default selected.
    Database EngineMySQL
    Driver Classcom.mysql.jdbc.Driver
    URLjdbc:mysql://localhost:3306/Employees
    User Nameroot
  5. Save the data service with the datasource connection. 

A data service file (DBS file) will not now be created in your data service project. Shown below is the project directory.

Step

...

3: Creating a query

Now, letLet's add a write an SQL query to the data service.GET data from the MySQL datasource that you configured in the previous step:

  1. Select the data service you created in the previous step.
  2. Right-click and click Add Query.
  3. Enter the following query details:

    Query IDGetEmployeeDetails
    DatasourceDatasource
  4. Save the query. The query element is now added to the data service:
  5. Right-click the the GetEmployeeDetails query,  query and click click Add SQL SQL to add the following SQL statement:

    Code Block
    select EmployeeNumber, FirstName, LastName, Email from Employees where EmployeeNumber=:EmployeeNumber
  6. Save the SQL statement. The SQL statement is now added to the query.
  7. Right-click the query again , and click click Add Input Mapping

  8. Enter the following input mapping details:

    Mapping NameEmployeeNumber
    Parameter TypeSCALAR
    SQL TypeSTRING
  9. Save the input mapping.
  10. Right-click the query again , and click Add Output Mapping.

  11. Enter the following value to group the output mapping:

    Grouped by ElementEmployees
  12. Save the output mapping.

  13. Right-click the output mapping , and go to to Add Output Mapping → Add Element to  to create an element.

  14. Enter the following element details.

    Datasource Typecolumn
    Output Field NameEmployeeNumber
    Datasource Column NameEmployeeNumber
    Schema TypeString
  15. Save the element.
  16. Follow the same steps to create the following output elements:

    Datasource TypeOutput Field NameDatasource Column NameSchema Type
    columnFirstNameFirstNamestring
    columnLastNameLastNamestring
    columnEmailEmailstring
  17. Save the output elements.

The data service should now have the query element added as shown below.

Step

...

4: Creating a resource to invoke the query

Now, let's create a REST resource that can be used to invoke the query.

  1. Right-click the datasource, data service and click click Add Resource. Add the following resource details.

    Resource MethodGET
    Resource PathEmployee/{EmployeeNumber}
  2. Expand the GET resource, and click the the GetEmployeeDetails (call-query). Connect the query to the resource by adding the following:

    Query IDGetEmployeeDetails
  3. Save the resource.

The data service should now have the resource added as shown below.

Step 5: Testing the data service

You can test the data service artifacts by following the steps given below.

  1. Package the data service file (.dbs file) into a Composite Application (CApp). See the instructions in Packaging Artifacts into Composite Applications.
  2. Add your WSO2 EI product instance to the Tooling environment and deploy the CApp in the server. See the instructions in Deploying Composite Applications in the Server.
  3. Start the ESB profile. The data service is now deployed in WSO2 EI.
  4. Send a GET request to invoke the service. You can use curl as shown below.

    Code Block
    curl -X GET http://localhost:8280/services/RDBMSDataService.HTTPEndpoint/Employee/3

    You will receive the employee's details in the response.

Importing a data service

Follow these steps to import an existing data service from a data service descriptor file (DBS ) filefile) to a Data Service project in Tooling. Alternatively, you can can create a new data service.

  1. You can either use the Data Service link on the dashboard (click Developer Studio -> Open Dashboard) or right-click on the Data Service project in the project explorer and click New -> Data Service
  2. Select Select Import Data Service and click  and click Next.
  3. Specify the data service descriptor (DBS) file by typing its full pathname or clicking by clicking Browse and  and navigating to the file.
  4. Optionally, specify the location and working sets for this project. 
  5. A Maven POM file will be generated automatically for this project. If you want to customize the Maven options (such as including parent POM information in the file from another project in this workspace), click click Next and  and specify the options.
  6. Click Finish Finish. The data service is created, added and the data service is open in the editor. You can now right-click the data service in the outline and add queries, operations, additional data sources, and so on, or click the Source view to add components using XML.

Anchor
secure_data_service
secure_data_service
Applying security to a data service

WSO2 supports WS-Security, WS-Policy, and WS-Security Policy specifications and these . These specifications define a behavioral model for Web services. To enable a security policy for a data service, you need to first create a security policy file, and then add it to the data service.

Step 1: Creating a

...

Follow the instructions given below.

...

Open your data service in Developer Studio as explained in the Creating a new data service section given above. You can either create a new data service or import an existing DBS file.

...

registry resource project

Registry artifacts (such as security policy files) should be stored in a Registry Resource project. Follow the steps given below to create a project:

  1. Open the Developer Studio Dashboard (click Developer Studio > Open Dashboard) and click Registry Resource Project.Image Removed
  2. Enter a name for the project and click Next.enter details of the new projectImage Removed
  3. Enter If required, change the Maven information about the project and click .
  4. Click Finish.
    enter maven information about the projectImage RemovedRight-click on the SecurityPolicyProject in the left navigation panel, click New and then select Registry Resource.
    right click and create registry resourceImage RemovedThe new project will be listed in the project explorer.

Step 2: Creating a security policy as a registry resource

  1. You can start creating a registry resource in two ways:
    1. Open the dashboard (click Developer Studio -> Open Dashboard) and click Registry Resource.  
    2. Select the already created Registry Resource Project in the navigator, right click and go to New -> Registry Resource
    The New Registry Resource window will open as shown below.
    Image Added
  2. Select From existing template and click Next.

  3. Anchor
    step5
    step5
    Enter a resource name and select the WS-Policy template along with the preferred registry path. 
    Image Removedthe following details:

    Resource NameSample_Policy
    Artifact NameSample_Policy
    TemplateWS-Policy
    Registrygov
    Registry pathws-policy/
  4. Click Finish and the policy file will be listed in the navigator.
      Image RemovedDouble-click on the policy file to view the list of security policies. You can
      1. Let's use the Design View

       or the Source View.
      Localtab Group
      Localtab
      titleDesign View

      Image Removed

      Localtab
      titleSource View

      Image Removed

      Enable required scenario.
      1.  to enable the required security scenario. For example, enable the Sign and Encyrpt security scenario as shown below.

        Tip

        Click the icon next to

      each scenario for more information
      1. the scenario to get details of the scenario.

      Image Removed

      1. Image Added

      2. You can

      provide service information as a private store
      1. also provide encryption properties, signature properties, and advanced

      configuration information as a rampart configuration.
      Image RemovedImage Removed
      1. rampart configurations as shown below.

        Localtab Group
        Localtab
        titleEncryption/Signature Properties

        Image Added

        Localtab
        titleRampart Properties

        Image Added

        Info
        titleUsing role-based permissions?

        For certain scenarios, you can specify user roles. After you select the scenario, scroll to the right to see the User Roles button.
        Image Modified

        Either define the user roles inline or retrieve the user roles from the server.

        Localtab Group
        Localtab
        titleDefine Inline

        Image Modified

        Localtab
        titleGet from the
      Server
      1. server

        Image Modified

    1. The security policy will now be saved to the resource project as shown below.
      Image Removed

    ...

    1. Save the policy file.

    Step 3: Adding the security policy to the data service

    Once you have configured the policy file, you can add the security policy to the data service as explained below. 

    1. Select the If you have already created a data service using Tooling, select the file from the Project Explorer. Alternatively, you can download this sample data service (.dbs file) that is already stored in the workspace, and click the browse icon file and import it to your Tooling environment. 

    2. Once you have select the data service file, click the browse icon for the Policy field.

      Image Modified

      InfoThere are two ways to add a security policy to the data service: If you have a policy file already stored in your product registry, you can add it by clicking registry. If you want to use a new policy file that you have created and stored in your current workspace, you can add it by clicking workspace.

    3. Click workspace, to add the security policy from the current workspace. You can select the path to the sample_policy.xml file that you created in the previous steps.
      Image Modified
    4. Click OK, and the security policy will be added to the data service.
      Image Modified

    ...

    1. Save the data service.

    Step 4: Testing the data service with the security policy

    Follow the steps given below.

    1. Package the data service file (.dbs file) and the security policy file into a Composite Application (CApp). See the instructions in Packaging Artifacts into Composite Applications.

      Warning

      Once you create the CApp project, note that the information about each of the projects and artifacts that you packaged into the CApp will be listed (under Dependencies in the Composite Application Project POM Editor).

      Be sure to set the server role to Enterprise Service Bus

    ...

    1.  for the security policy file as shown below

      Image Added

    2. Add your WSO2 EI product instance to the Tooling environment and deploy the CApp in the server. See the instructions in Deploying Composite Applications in the Server.
    3. Start the ESB profile. The data service and the security policy is now deployed in WSO2 EI.

    Creating a custom validator

    An input validator allows a data service to validate the input parameters in a request and stop the execution of the request if the input doesn’t meet the required criteria. In addition to the default validators provided by Data Service Server, you can create your own custom validators by creating a Java class that implements the org.wso2.carbon.dataservices.core.validation.Validator interface. You can create a new custom validator or import an existing validator project.  

    ...

    1. You can either use the Data Services Validator Project link on the dashboard (click Developer Studio -> Open Dashboard) or click File-> New -> Other -> Data Services Validator Project option. 
    2. Select Create New Data Services Validator Project and click Next.
    3. Type a unique name for the project and specify the package and class name for this validator.
    4. Optionally, specify the location and working set for this project.
    5. A Maven POM file will be generated automatically for this project. If you want to customize the Maven options (such as including parent POM information in the file from another project in this workspace), click Next and specify the options.
    6. Click Finish. The project is created, and the new validator class is open in the editor , where you can add your validation logic.  

    ...

    Follow these steps to import an existing custom validator project. Alternatively, you can create a new custom validator.

    1. You can either use the Data Services Validator Project link on the dashboard (click Developer Studio -> Open Dashboard) or click File-> New -> Other -> Data Services Validator Project option. 
    2. Select Import Project From Workspace and click Next.
    3. Select the existing validator project, and optionally specify the location and working sets for the new project. 
    4. A Maven POM file will be generated automatically for this project. If you want to customize the Maven options (such as including parent POM information in the file from another project in this workspace), click Next and specify the options.
    5. Click Finish. The project is imported, and the validator class is open in the editor, where you can modify the validation logic as needed.

    ...

    Once you have created the data service artifacts as explained in the previous topics, you need to create a Composite Application project, and then create a CAR file to deploy it in the ESB server.

    1. Right-click the Project Explorer and click New -> Project.
    2. From the window that opens, click Composite Application Project.
    3. Give a name to the Composite Application project and select the projects that you need to group into your C-App from the list of available projects.
    4. In the Composite Application Project POM Editor that opens, under Dependencies, note the information for each of the projects you selected earlier. If you have added a security policy to your data service using a registry resource as explained above, be sure that EnterpriseServiceBus is selected as the server role as shown below.

      Image Removed

    5. To deploy the artifacts using the ESB's management console, you need to create a CAR file as follows:
      • Right-click the C-App project and select Export Composite Application Project from the pop-up menu. 
        OR
      • Open the pom.xml file in the Composite Application Project POM Editor and click the button for creating an archive in the upper-right corner.
      Info

      The CAR file is now saved in the location you specify. See Packaging Artifacts into Composite Applications, for more information on using CAR files

    6. Next, deploy the CAR file in the server.