This documentation is for WSO2 API Manager 2.0.0 View documentation for the latest release.
Upgrading from the Previous Release - API Manager 2.0.0 - WSO2 Documentation
||
Skip to end of metadata
Go to start of metadata

To upgrade from a version older than 1.8.0, follow the instructions in the document that was released immediately after your current release and upgrade incrementally.

Migrating the configurations 

Do not copy entire configuration files from the current version of API Manager to the new one as some configuration files (for example, api-manager.xml) may have changed. Instead, redo the configuration changes in the new configuration files.

In this section, you move all existing API Manager configurations from the current environment to the new one.

  1. Back up all databases in your API Manager instances along with the synapse configs of all the tenants and super tenant.
    You find the synapse configs of the super tenant in the <CURRENT_API-M_HOME>/repository/deployment/server/synapse-configs/default directory. Synapse configs of tenants are in the <CURRENT_API-M_HOME>/repository/tenants directory. 
    If you use a clustered/distributed API Manager setup, back up the available configurations in the API Gateway node.

  2. Download API Manager 2.0.0 from http://wso2.com/products/api-manager/.
  3. Open the <API-M_2.0.0_HOME>/repository/conf/datasources/master-datasources.xml file and provide the datasource configurations for the following databases. You can copy the configuration values from the same file in the current API Manager instance already being used.

    • User Store
    • Registry database/s
    • API Manager Databases
  4. Edit the registry configurations in the <API-M_2.0.0_HOME>/repository/conf/registry.xml file and the user database in the <API-M_2.0.0_HOME>/repository/conf/user-mgt.xml file similar to the configurations of the current API Manager.

    Note that in a clustered/distributed API Manager setup, step 5 and 6 need to be done on the Gateway node.

  5. Move all your synapse configurations except the files mentioned below by copying and replacing the <CURRENT_API-M_HOME>/repository/deployment/server/synapse-configs/default directory to the <API-M_2.0.0_HOME>/repository/deployment/server/synapse-configs/default. 

    NOTE: Do not replace the files listed below from the <CURRENT_API-M_HOME>/repository/deployment/server/synapse-configs/default folder to APIM 2.0.0. These are application-specific APIs and sequences. If you made any custom changes to the files below, please merge them to the corresponding files in 2.0.0.

    /api/_AuthorizeAPI_.xml /api/_RevokeAPI_.xml /api/_TokenAPI_.xml /api/_UserInfoAPI_.xml /sequences/_auth_failure_handler_.xml /sequences/_build_.xml /sequences/_cors_request_handler_.xml /sequences/fault.xml /sequences/main.xml /sequences/_production_key_error_.xml /sequences/_resource_mismatch_handler_.xml /sequences/_sandbox_key_error_.xml /sequences/_throttle_out_handler_.xml /sequences/_token_fault_.xml /proxy-services/WorkflowCallbackService.xml
  6. Move all your tenant synapse configurations by updating the configurations made in the <CURRENT_API-M_HOME>/repository/tenants directory to the <API-M_2.0.0_HOME>/repository/tenants directory.

    NOTE: Get the files listed below from the <API-M_2.0.0_HOME>/repository/deployment/server/synapse-configs/default/sequences directory and replace the corresponding files in the <API-M_2.0.0_HOME>/repository/tenants/<tenant-id>/synapse-configs/default/sequences directory.

    _auth_failure_handler_.xml _build_.xml _cors_request_handler_.xml fault.xml main.xml _production_key_error_.xml _resource_mismatch_handler_.xml _sandbox_key_error_.xml _throttle_out_handler_.xml _token_fault_.xml
  7. If you have documentation defined for your APIs, make sure you add a value to the Documentation Summary field. 
    Documentation summary is mandatory in APIM 2.0 onwards. Trying to migrate with an empty documentation summary field from an older version can cause issues in the migration. 

Upgrading the API Manager to 2.0.0

  1. Stop all running API Manager server instances.
  2. Make sure you backed up all the databases and synapse configs as instructed in step 1 of the previous section.

  3. Before you run the migration client, open the <API-M_2.0.0_HOME>/repository/conf/datasources/master-datasources.xml file, and set the <username>, and <password> elements of the AM_DB JNDI to that of a user who has permissions to alter tables in the database.

    Tip: After you are done running the migration client, you can switch these credentials back to a user with lesser privileges.

    For example,

    <datasource>
            ...
            <definition type="RDBMS">
                <configuration>
                    ...
                    <username>xxxxxx</username>
                    <password>xxxxxx</password>
                    ...
                </configuration>
             </definition>
    </datasource>
  4. Download and extract the WSO2 API Manager Migration Client and do the following:
    1. Copy the org.wso2.carbon.apimgt.migrate.client-2.0.X.jar file to the <API-M_2.0.0_HOME>/repository/components/dropins directory. If you use a clustered/distributed API Manager setup, copy the JAR file to a Publisher and a Gateway nodes.
    2. Copy the migration-script folder into <API-M_2.0.0_HOME>. If you use a clustered/distributed API Manager setup, copy the migration-script folder to a Publisher node.
      1. Start API Manager 2.0.0 with the following command-line options to migrate the database, registry and the file system separately in the given order.

        CommandDescription
        -DmigrateDB=true -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION>To migrate the database only. This migrates the AM_DB database. Please ensure that the <API-M_PUBLISHER>/repository/conf/datasources/master-datasources.xml file has an entry for AM_DB.
        -DmigrateReg=true -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION>To migrate the registry only.  This migrates the registry-related resources such as .rxt and swagger definitions.
        -DmigrateFS=true -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION>To migrate the file system only.  This migrates the synapse config files such as APIs that reside in the file system. Therefore, you must run this command on the Gateway node/s of a distributed API Manager setup.
        -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> -Doptions=migrateThrottling

        To migrate to the new throttling engine and generate throttle policies.

        This migrates synapse config files, the APIM database with new throttle policies and generates throttle policies to the <API-M_HOME>/executionplans directory. Therefore, you must run this command against a node that has synapse config files and the AM_DB. After the migration, copy the <API-M_HOME>/executionplans directory to the repository/deployment/server/executionplans directory of the Traffic Manager node.  

        Note that you need to use either this command or the one below in order to migrate to the new throttling engine of APIM 2.0.0.

        -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> -Doptions=migrateThrottling,deployPolicies

        To migrate to the new throttling engine and deploy throttle policies to the Traffic Manager node.

        This migrates synapse config files, the APIM database with new throttle policies and deploys policies to the Traffic Manager node. Therefore, you must run this command against a node that has synapse config files, the AM_DB and is pointed to the Traffic Manager node.

        Use this command if you want to deploy the throttle policies directly to the Traffic Manager node while performing the migration.

        Tip: If you are using a clustered/distributed API Manager setup, run with the following options -DmigrateDB=true -DmigrateReg=true -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> in the API Publisher node and the -DmigrateFS=true -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> options in the API Gateway node.

        To migrate to the new throttling engine of APIM 2.0.0, you have to use a node which has AM_DB pointed as well as API gateway synapse configurations in place in the file system. If you use only "migrateThrottling" option, you can use API Gateway node. Make sure that AM_DB is pointed to the gateway during the migration time. You can the start the gateway server with the -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> -Doptions=migrateThrottling command. After completing this execution, make sure you copy the generated throttle policies in the <API-M_HOME>/executionplans directory to the <API-M_TRAFFIC_MANAGER>/repository/deployment/server/executionplans directory.

        If you use both the migrateThrottling,deployPolicies options, make sure that the Traffic Manager node is configured with the Gateway node and also make sure you avoid passing the -Dprofile option while performing the migration. With these options, you don't have to manually copy the throttling policies to the traffic manager. You can the start the gateway server with -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> -Doptions=migrateThrottling,deployPolicies command.  

        If you have enabled token encryption in your database, as part of database migration, those consumer keys need to be decrypted. If you see the error below, it means that the migration client has failed to decrypt some of the keys in the old database. TID: [-1234] [] [2016-04-11 08:27:16,249] ERROR {org.wso2.carbon.apimgt.migration.client.MigrateFrom19to110} -  Cannot decrypt consumer key : DRHbt68uSU4+7xtCEIzuf42CMaqXbNjYl3OYVJ0VL/H6EsFo8GBRaZGUhLHlTWIHzYrvoeOpb1YbauvRRIN/b6VqEd9m8HuYOIuLkkDd AM_APPLICATION_KEY_MAPPING table {org.wso2.carbon.apimgt.migration.client.MigrateFrom19to110} If this occurs, by default, the migration client will terminate without updating databases in order to maintain database integrity. However, you can change this behavior by adding the following argument and running the APIM migration client again from the beginning. Please note that the database is then updated with the keys where decryption was successful and failed keys are permanently deleted.
        -DremoveDecryptionFailedKeysFromDB=true

        You may notice the following exception being thrown in the console when API Manager 2.0.0 is started at this point,

        org.wso2.carbon.idp.mgt.IdentityProviderManagementException: Error occurred while retrieving Identity Provider information for tenant : carbon.super and Identity Provider name : LOCAL

        This is due to the fact that the IDP_METADATA table does not exist in the API Manager database. This is not a cause for alarm and is to be expected since we are yet to complete the migration process. The instructions given from step 9 onwards will address the creation of the IDP_METADATA table after which this exception will no longer be thrown.

  5. Do the following to re-index the artifacts in the registry:

    1. Rename the <lastAccessTimeLocation> element in the <API-M_2.0.0_HOME>/repository/conf/registry.xml file. If you use a clustered/distributed API Manager setup, change the file in the API Publisher node. For example, change the /_system/local/repository/components/org.wso2.carbon.registry/indexing/lastaccesstime registry path to /_system/local/repository/components/org.wso2.carbon.registry/indexing/lastaccesstime_1

    2. Shut down the API Manager 2.0.0, back up and delete the <API-M_2.0.0_HOME>/solr directory.

    Steps 6,7 and 8 needs to be executed ONLY if the setup is migrating from 1.8.0, 1.9.0 or 1.9.1. If the current setup is in 1.10.0, steps 6, 7 and 8 are NOT required to be executed.

  6. Execute the following update queries against the WSO2AM_DB to set the GRANT_TYPE column of the IDN_OAUTH_CONSUMER_APPS table:

    UPDATE IDN_OAUTH_CONSUMER_APPS set GRANT_TYPES='refresh_token urn:ietf:params:oauth:grant-type:saml2-bearer implicit password iwa:ntlm client_credentials authorization_code' where CALLBACK_URL is not NULL; 
    
    UPDATE IDN_OAUTH_CONSUMER_APPS set GRANT_TYPES='refresh_token urn:ietf:params:oauth:grant-type:saml2-bearer password iwa:ntlm client_credentials' where CALLBACK_URL is NULL;  
  7. Download the WSO2 Identity Server Migration Client. This is required to migrate the Identity and User Store Database schemas that is used by API Manager. Unzip the downloaded file to the local file system where the wso2is-5.1.0-migration directory is created and do the following:

    If you are running in a distributed setup, ensure that this step is only carried out for the Key Manager node.

    If you are running WSO2 Identity Server as a Key Manager in a distributed setup, first follow the instructions in Configuring the Identity Server 5.2.0 as a Key Manager with API Manager 2.0.0 before following the steps below. Make sure that the additional configurations you made in the following files for WSO2 Identity Server 5.1.0 are redone in the Identity Server 5.2.0 configuration files,

    master-datasources.xml registry.xml user-mgt.xml api-manager.xml

          a. Copy the <wso2is-5.1.0-migration>/dbscripts/migration­5.0.0_to_5.1.0 directory to the <API-M_2.0.0_HOME>/dbscripts directory.

    If you are using MySQL 5.7, carryout the following steps before proceeding to the next step.

    1. Navigate to the <wso2is-5.1.0-migration>/ dbscripts/identity/migration­5.0.0SP1_to_5.1.0 directory.

    2. Rename the mysql.sql file to mysql_old.sql or a name of your choice.

    3. Rename the mysql_57.sql file to - mysql.sql


          b. Copy the <wso2is-5.1.0-migration>/ dbscripts/identity/migration­5.0.0SP1_to_5.1.0 directory and the  <wso2is-5.1.0-migration>/ dbscripts/identity/migration­5.0.0_to_5.0.0SP1 directory to the <API-M_2.0.0_HOME>/dbscripts/identity directory.
          c. Copy the <wso2is-5.1.0-migration>/dropins/org.wso2.carbon.is.migrate.client-­5.1.0.jar file to the <API-M_2.0.0_HOME>/repository/components/dropins directory.    

  8. Before executing identity migration as given in the step below, verify whether the API Manager database has the IDN_ASSOCIATED_ID table. To do that, connect to the API Manager database and check whether the IDN_ASSOCIATED_ID table exists. If not, download the IDN Tables zip file and run the relevant script according to your RDBMS against your API Manager DB.  

  9. Start API Manager 2.0.0 with the command line option -Dmigrate -Dcomponent=identity to carry out the complete Identity and User Store DB migration. Alternatively, you can run it step by step by starting the server with the command line options given below in the following order. 
    • -Dmigrate -DmigrateIdentityDB -Dcomponent=identity
    • -Dmigrate -DmigrateUMDB -Dcomponent=identity
    • -Dmigrate -DmigrateIdentityData -Dcomponent=identity
    • -Dmigrate -DmigrateIdentityDBFinalize -Dcomponent=identity
    • -Dmigrate -DmigrateUMData -Dcomponent=identity
  10. Upgrading to IS 5.2.0 from 5.1.0

    1. Download 5.1.0 to 5.2.0 migration DB scripts from here.
    2. Unzip and copy it to the <API-M_2.0.0_HOME>/dbscripts/identity directory.
      Copying the DB scripts are optional as it is done for reference purposes to indicate the version to which you have upgraded your APIM IS component.
    3. Execute the DB scripts, which corresponds to the RDBMS that you are working with, manually.

      Apply the respective DB script that is inside the following directories against the respective DB as follows:

      DB script directoryApplicable DBDescription
      <wso2is-5.2.0-migration>/dbscripts/identity/migration-5.1.0_to_5.2.0 AM_DB
      <wso2is-5.2.0-migration>/dbscripts/migration-5.1.0_to_5.2.0 UM_DBThis script includes insert operations and some index creation scripts against the UM_CLAIM table which are required in order for you to upgrade from your previous APIM version to API-M 2.0.0.

      For example, if you are working with a MySQL DB run the mysql.sql script.

    During the Identity and User Store DB migration, you may notice warning messages similar to the following:

    [2016-07-22 07:14:49,386] WARN - AppDeployerServiceComponent Waiting for required OSGi services: org.wso2.carbon.application.deployer.synapse.service.SynapseAppDeployerService,org.wso2.carbon.mediation.
    initializer.services.SynapseEnvironmentService, [2016-07-22 07:14:49,418] WARN - StartupFinalizerServiceComponent Waiting for required OSGi services: org.wso2.carbon.application.deployer.service.CappDeploymentService,org.wso2.carbon.server.admin.
    common.IServerAdmin,org.wso2.carbon.throttling.agent.ThrottlingAgent,

    Do not abort the migration process at this point as data migration is still happening in the background and aborting could result in data corruption. The time taken to finish depends on the amount of tokens in the database and could take hours depending on the amount of data.

This concludes the upgrade process.

Tip: The migration client that you use in this guide automatically migrates your tenants, workflows, external user stores etc. to the upgrade environment. There is no need to migrate them manually.

Data migration of statistics from WSO2 DAS to API Manager Analytics

From APIM 2.0.0 onwards, statistics can be configured only for RDBMS since the API Manager 1.10.0 REST based analytics configuration no longer exists. This section walks you through how to migrate statistics data from previous versions of API Manager to 2.0.0. 

If you have configured analytics using RDBMS,

  1. Download and extract the WSO2 APIM stat migration ZIP file. 
  2. Take a backup of the API Manager statistics DB. 
  3. Execute the relevant script in the <wso2-api-stat-migration>/dbscripts directory against your API Manager statistics DB. 
  4. Add the API Manager statistics DB as a datasource in WSO2 API Manager Analytics.

If you configured analytics using the REST client in APIM 1.10.0 with DAS 3.0.x,

  1. Configure analytics using API Manager 1.10.0 and DAS 3.0.x with the RDBMS client according to the instructions in Publishing API Runtime Statistics Using RDBMS.
  2. Wait for data to appear on the RDBMS and API Manager statistics dashboard.
  3. Add the API Manager statistics DB as a datasource in WSO2 API Manager Analytics.

Finally, execute relevant db script found in the migration-scripts/110-200-migration/stat/ directory on the STAT_DB.

Once done, configure analytics in API Manager 2.0.0 according to the instructions in Configuring APIM Analytics.