This documentation is for WSO2 API Manager 2.6.0. View documentation for the latest release.

All docs This doc
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 25 Current »

Organizations can have multiple environments like Development, Testing, QA, Staging, Production with own instance of API Managers. These environments often have different configurations for APIs.

WSO2 API Manager supports creating fully automated CI/CD pipelines for APIs. It supports a variety of features including maintaining the lifecycle of API, changing environment-specific configurations throughout the CI/CD process.  This allows seamlessly migrating APIs across different environments reducing human intervene on the migration process.


Overview
















CI/CD for API Manager relies on a Version Control system which acts as a Single Source of Truth for the pipeline. After APIs are exported from one environment, promoting it to the other environment is done via the CLI tool for API Manager known as apimcli. It is capable of handling environment-related configurations and can promote the API seamlessly to other environments via a single command.  

Preparing Environments

Running the CLI tool

Step 1 - Deploy the API import/export tool

  1. Download the latest version of WSO2 API Manager from http://wso2.com/products/api-manager/.

  2. Start WSO2 API Manager.
  3. Download the latest WSO2 API import/export tool (api-import-export-2.6.0-v17.war) from  here.

    • Note that the import/export tool attached is specific to this version of WSO2 API Manager.

    • Make sure to delete all previous versions of the web app (e.g., api-import-export-2.6.0-v17.war), both the compressed and the extracted files, before copying over the new web app.

    • This version of the API Import and Export web application is compatible with the WSO2 WUM updates from July 8, 2019, onwards. For more information on updating WSO2 API Manager, see Updating WSO2 API Manager.
    • The previous version of the web application which is compatible with WUM updates that are prior to the above-mentioned date can be download from here.
    • Note that API update support in the targeted environment is available only with the api-import-export-2.6.0-v14.war version onwards.
  4. Copy the downloaded api-import-export-2.6.0-v17.war file to the <API-M_HOME>/repository/deployment/server/webapps folder.
    The file is automatically deployed as hot deployment is enabled.


Step 2 - Optionally, set the proxy environment variables for CLI

You can set proxy related HTTP_PROXY, HTTPS_PROXY, http_proxy, and https_proxy standard environment variables, with or without basic authentication as shown below to send the requests initiated from CLI via a proxy server. After one of the following environment variables is set in your environment where CLI is used, all the requests will go through the proxy server specified.

export HTTP_PROXY="http://<host-name>:<port>"
export HTTPS_PROXY="https://<host-name>:<port>"
export http_proxy="http://<host-name>:<port>"
export https_proxy="https://<host-name>:<port>"
export HTTP_PROXY="http://<username>:<password>@<host-name>:<port>"
export HTTPS_PROXY="https://<username>:<password>@<host-name>:<port>"
export http_proxy="http://<username>:<password>@<host-name>:<port>"
export https_proxy="https://<username>:<password>@<host-name>:<port>"
export HTTP_PROXY="http://localhost:3128"
export HTTPS_PROXY="https://localhost:3128"
export http_proxy="http://localhost:3128"
export https_proxy="https://localhost:3128"
export HTTP_PROXY="http://testuser:[email protected]:3128"
export HTTPS_PROXY="https://testuser:[email protected]:3128"
export http_proxy="http://testuser:[email protected]:3128"
export https_proxy="https://testuser:[email protected]:3128"


In conjunction with the above proxy configurations, calls from the CLI to particular hosts that need to bypass proxies can be configured via either the NO_PROXY or no_proxy environment variables.

export NO_PROXY="https://<host-name>:<port>"
export no_proxy="https://<host-name>:<port>"
export NO_PROXY="https://localhost:3128"
export no_proxy="https://localhost:3128"


Step 3 - Run the CLI tool
  1. Download API Controller based on your preferred platform (i.e., Mac, Windows, Linux).

  2. Extract the downloaded archive containing the CLI Tool to a preferred location and navigate to it from the terminal (using cd ).
  3. Navigate to the working directory where the executable CLI Tool resides.
  4. Execute the following command to start the CLI tool.

    ./apimcli
  5. Add the location of the extracted folder to your system's $PATH variable to be able to access the executable from anywhere.

    For further instructions execute the following command.

    apimcli --help
Global flags for CLI tool

The following are some global flags that you can use with the CLI tool.

      --verbose
           Enable verbose logs (Provides more information on execution)
      --insecure, -k
          Allow connections to SSL sites without certs
      --help, -h
          Display information and example usage of a command

Adding an Environment

  1. Then you need to install apimcli tool from here. After downloading the correct version for your OS, add the CLI to the $PATH of the system to access CLI tool from anywhere.
  2. Once the tool installed you can introduce API Manager environments using add-env  command.
    For example, you can configure an API Manager running on port 9443 as your development(dev) environment by issuing the following command.

    apimcli add-env -n dev \
                          --registration https://localhost:9443/client-registration/v0.14/register \
                          --apim https://localhost:9443 \
                          --token https://localhost:8243/token \
                          --import-export https://localhost:9443/api-import-export-{version} \
                          --admin https://localhost:9443/api/am/admin/v0.14 \
                          --api_list https://localhost:9443/api/am/publisher/v0.14/apis \
                          --app_list https://localhost:9443/api/am/store/v0.14/applications

    Let’s add another environment for production purposes

    apimcli add-env -n prod \
                          --registration https://localhost:9444/client-registration/v0.14/register \
                          --apim https://localhost:9444 \
                          --token https://localhost:8244/token \
                          --import-export https://localhost:9444/api-import-export-{version} \
                          --admin https://localhost:9444/api/am/admin/v0.14 \
                          --api_list https://localhost:9444/api/am/publisher/v0.14/apis \
                          --app_list https://localhost:9444/api/am/store/v0.14/applications

Exporting APIs with CLI

Now we have to add two different environments. Our end goal is to automate the API migration between dev and prod environments. Publish an API in dev environment using Publisher UI of API Manager. If you don’t know how to follow Quick Start Guide to learn how to deploy an API in the API Manager.


For this example, we are gonna use the Swagger Petstore API. 

  1. Start creating it by using “Add New API” and selecting an existing API option. Then create the API and set its name as SwaggerPetstore and Version as 1.0.0

2. Next, put backend URLs for Production and Sandbox and next page Publish the API.

For this example, we can use

Production: http://dev.wso2.com

Sandbox: http://dev.sandbox.wso2.com


Login to API Manager from CLI

Now before exporting this API as a project, we need to login to the API Manager from the CLI.

Sample:

apimcli login <Envionment> -u <Username> -p <Password>


Example:


apimcli login dev -u admin -p admin

Export an API

After successfully logged in we can export the API via CLI using a simple command,

Sample:

apimcli export-api -e <Env>  -n <APINAME> -v <VERSION> -r <PROVIDER> -k  


Example:

apimcli export-api -e dev  -n SwaggerPetstore -v 1.0.0 -r admin -k 


After providing name, version and provider you can see the API is downloaded as an archive to your machine.


We are using insecure(-k) flag because we are using self-signed certificates.

Preparing API Project for CI/CD

  1. After exporting the API as an archive, extract the content. You will find a directory called SwaggerPetstore-1.0.0 

  2. Copy this directory into your Version Control Repository. Rename it to SwaggerPetstore to reference it easily.

  3. Next, when we promoting API to the production environment we want to have different backend URLs for Production and Sandbox. For easily do this CLI tool supports a special file called api_params


Adding environment-specific parameters for APIs

  1. Create a file called api_params.yaml inside your SwaggerPetstore directory. Open this api_params.yaml file in your favourite text editor.

  2. Adding environment-specific parameters for APIs is done via this file. We are going to change backends in the production environment to point for production endpoints

    environments: 
      - name: dev 
        endpoints: 
            production: 
                url: http://dev.wso2.com
            sandbox:
                url: http://dev.sandbox.wso2.com
      - name: prod
        endpoints:
            production:
                url: http://prod.wso2.com
            sandbox:
                url: http://prod.sandbox.wso2.com

    As you can see we have defined prod.wso2.com and prod.sandbox.wso2.com as backend URLs for the production environment.

    Setup Multiple Endpoints

    Setup Load Balancing Endpoints

    To setup load balancing endpoints, you need to specify the field loadBalanceEndpoints instead of the usual endpoints field in the api_params.yaml file. Under the loadBalanceEndpoints, you can specify the following fields.

    • production: An array which consists the multiple production endpoints
    • sandbox: An array which consists the multiple sandbox endpoints
    • sessionManagement: Values can be "none", "transport", "soap", "simpleClientSession" and if not specified the default value would be "transport"
    • sessionTimeout: The number of milliseconds after which the session would time out
    Example api_params.yaml for load balancing
    environments:
     - name: dev
       endpoints:
           production:
               url: http://dev.wso2.com
           sandbox:
               url: http://dev.sandbox.wso2.com
     - name: prod
       loadBalanceEndpoints:
           production:
               - url: http://prod.wso2.com
               - url: http://prod.next.wso2.com
               - url: http://prod.later.wso2.com
           sandbox:
               - url: http://prod.sandbox.wso2.com
               - url: http://prod.next.sandbox.wso2.com
    Setup Failover Endpoints

    To setup failover endpoints, you need to specify the field failoverEndpoints instead of the usual endpoints field in the api_params.yaml file. Under the failoverEndpoints, you can specify the following fields.

    • production: The primary production endpoint (not an array)
    • sandbox: The primary sandbox endpoint (not an array)
    • productionFailovers: An array which consists of failover production endpoints
    • sandboxFailovers: An array which consists failover sandbox endpoints
    Example api_params.yaml for failover
    environments:
      - name: dev
        endpoints:
            production:
            sandbox:
      - name: prod
        failoverEndpoints:
           production:
               url: http://prod.wso2.com
           productionFailovers:
               - url: http://prod.next.wso2.com
               - url: http://prod.later.wso2.com
           sandbox:
               url: http://prod.sandbox.wso2.com
           sandboxFailovers:
               - url: http://prod.sandbox.next.wso2.com

    Advanced Endpoint Configurations

    Setting Up Advanced Endpoint Configurations

    You can set up advanced configurations related to endpoints by specifying the field config for the corresponding endpoint URL as shown with the below fields. (Note that, the only supported values for actionSelect is "discard" and "fault")

    Example for setting up advanced endpoint configurations in the api_params.yaml
    environments: 
      - name: prod
        endpoints:
            production:
                url: http://prod.wso2.com
                config: 
                    factor: 3
                    retryTimeOut: 13000
                    retryDelay: 23000
                    retryErroCode: 101504
                    actionSelect: discard
                    actionDuration: 75000
                    suspendErrorCode: 101504
                    suspendMaxDuration: 25000
                    suspendDuration: 45000
            sandbox:
                url: http://prod.sandbox.wso2.com

    Import Certificates

    Importing Endpoint Certificates

    You can import endpoint certificates using the api_params.yaml file by specifying the field certs. This field is an array where each item contains the following fields.

    • host: The endpoint URL which the certificate belongs to
    • alias: Alias for the certificate
    • path: File path to locate the certificate file
    Example api_params.yaml to import endpoint certificates
    environments:
      - name: dev
        endpoints:
            production:
            sandbox:
      - name: prod
        endpoints:
            production:
                url: http://prod1.wso2.com
            sandbox:
                url: http://prod2.wso2.com
        certs:
            - hostName: https://prod1.wso2.com
              alias: Prod1
              path: ~/.certs/prod1.pem
            - hostName: https://prod2.wso2.com
              alias: Prod2
              path: ~/.certs/prod2.pem 
    Importing Mutual SSL Certificates

    You can import Mutual SSL certificates using the api_params.yaml file by specifying the field mutualSslCerts. This field is an array where each item contains the following fields.

    • tierName: Name of the tier of the certificate
    • alias: Alias for the certificate
    • path: File path to locate the certificate file
    Example api_params.yaml to import mutual ssl certificates
    environments:
      - name: dev
        endpoints:
            production:
            sandbox:
      - name: prod
        endpoints:
            production:
                url: http://prod1.wso2.com
            sandbox:
                url: http://prod2.wso2.com
        mutualSslCerts:
            - tierName: Unlimited
              alias: Prod1
              path: ~/.certs/prod1.pem
            - tierName: Gold
              alias: Prod2
              path: ~/.certs/prod2.pem 

    Endpoint Security

    Setting Up Endpoint Security

    You can set up endpoint security by defining the field security. Under the security field, you can specify the following fields. 

    • enabled: If this is set to true you need to specify the below attributes as well. If this is false, then non of the security parameters will be set. If the enabled attribute is not set (blank), then the security parameters in api.yaml file will be considered.
    • type: Endpoint authentication type (can be either only basic or digest)
    • username: Endpoint username
    • password: Endpoint password
    Example api_params.yaml to set up endpoint security
    environments:
      - name: prod
        endpoints:
            production:
                url: http://prod1.wso2.com
            sandbox:
                url: http://prod2.wso2.com
        security:
        	enabled: true
            type: basic
            username: admin
            password: admin

    Subscription Policies/Tiers

    Setting Up Subscription Policies for an API

    You can set up subscription policies of an API by defining the field policies which is an array where you can list down the policy names that are needed to be enforced to the API. (Make sure that the policy/policies should be available in the environment. Otherwise, you will get an error.)

    Example api_params.yaml to set up subscription policies of an API
    environments:
      - name: prod
        endpoints:
            production:
                url: http://prod1.wso2.com
            sandbox:
                url: http://prod2.wso2.com
        policies:
        	- Gold
            - Silver
  3. Now we can commit this project to a version control system.


Importing API into a different environment

  1. Now we can try out importing this API into the production environment and test if the API works. To do this issue

    apimcli import-api -f ./SwaggerPetstore -e prod -k

    (You need to login to prod environment before issuing the command)
    With -f flag we point to the archive or project directory and with -e flag we provide target environment for the import process.

2. Now the tool will automatically detect the target environment and prepare a new artifact containing environment-related details.

3. You can investigate that the API has been imported with correct environment-specific details you defined and you can also see the API is in the published state too.

If you try again importing the same API to prod environment it will fail as API Manager does not allow updating existing APIs by default, it is to ensure that any accidental modification not happen.

But don’t worry, you can issue the same command with an additional flag called --update to update an existing API


apimcli import-api -f ./SwaggerPetstore -e prod --update -k

Now if you try again you can see it successfully imports the API

Automating API promotion with CI/CD Pipeline

Now you know building blocks of API import/export. By using the above commands you can create an automated pipeline for API promotion between environments.

  1. After installing the CLI tool on the CI/CD server you can configure environments using add-env as we did earlier. As the next step, you can create a pipeline with your automation server. 
    For a reference, we are providing you with a Jenkins pipeline

    Jenkinsfile
    pipeline {
        agent any
        environment {
            CI = 'true'
            API = './SwaggerPetstore'
        }
        stages {
            stage('Deploy to Test') {
                environment{
                    ENV = 'test'
                }
                steps {
                    echo 'Deploying to Test'
                    sh 'apimcli import-api -f $API -e $ENV -k --preserve-provider=false --update --verbose'
                }
            }
            stage('Deploy to Production') {
                environment{
                    ENV = 'prod' 
                }
                steps {
                    echo 'Deploying to Production'
                    sh 'apimcli import-api -f $API -e $ENV -k --preserve-provider=false --update --verbose'
                }
            }
        }
    }

You can see we have used an additional flag called --preserveProvider in the Jenkinsfile. It is used because if the User exported API does not exist in the target environment provider will be set to the current user who executes the import command.

Using environment variables 

There are situations where hiding certain information is required in parameters file or reuse values across the script. You can define them as environment variables in the parameters file in usual notation

For example:

  - name: test
    endpoints:
      production:
        url: $TEST_BACKEND

When importing the CLI tool will pick these values from your shell environment and replace them.

An example project containing all these can be found here.


  • No labels