This documentation is for WSO2 API Manager 2.0.0 View documentation for the latest release.
Page Comparison - Upgrading from the Previous Release (v.78 vs v.79) - API Manager 2.0.0 - WSO2 Documentation

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Added a note relative to MySQL 5.7 - https://support.wso2.com/jira/browse/WSODOCINTERNAL-1093

...

  1. Stop all running API Manager server instances.
  2. Make sure you backed up all the databases and synapse configs as instructed in step 1 of the previous section.

  3. Before you run the migration client, open the <API-M_2.0.0_HOME>/repository/conf/datasources/master-datasources.xml file, and set the <username>, and <password> elements of the AM_DB JNDI to that of a user who has permissions to alter tables in the database.

    Tip

    Tip: After you are done running the migration client, you can switch these credentials back to a user with lesser privileges.

    For example,

    Code Block
    <datasource>
            ...
            <definition type="RDBMS">
                <configuration>
                    ...
                    <username>xxxxxx</username>
                    <password>xxxxxx</password>
                    ...
                </configuration>
             </definition>
    </datasource>
  4. Download and extract the WSO2 API Manager Migration Client and do the following:
    1. Copy the org.wso2.carbon.apimgt.migrate.client-2.0.X.jar file to the <API-M_2.0.0_HOME>/repository/components/dropins directory. If you use a clustered/distributed API Manager setup, copy the JAR file to a Publisher and a Gateway nodes.
    2. Copy the migration-script folder into <API-M_2.0.0_HOME>. If you use a clustered/distributed API Manager setup, copy the migration-script folder to a Publisher node.
      1. Start API Manager 2.0.0 with the following command-line options to migrate the database, registry and the file system separately in the given order.

        CommandDescription
        -DmigrateDB=true -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION>To migrate the database only. This migrates the AM_DB database. Please ensure that the <API-M_PUBLISHER>/repository/conf/datasources/master-datasources.xml file has an entry for AM_DB.
        -DmigrateReg=true -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION>To migrate the registry only.  This migrates the registry-related resources such as .rxt and swagger definitions.
        -DmigrateFS=true -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION>To migrate the file system only.  This migrates the synapse config files such as APIs that reside in the file system. Therefore, you must run this command on the Gateway node/s of a distributed API Manager setup.
        -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> -Doptions=migrateThrottling

        To migrate to the new throttling engine and generate throttle policies.

        This migrates synapse config files, the APIM database with new throttle policies and generates throttle policies to the <API-M_HOME>/executionplans directory. Therefore, you must run this command against a node that has synapse config files and the AM_DB. After the migration, copy the <API-M_HOME>/executionplans directory to the repository/deployment/server/executionplans directory of the Traffic Manager node.  

        Note

        Note that you need to use either this command or the one below in order to migrate to the new throttling engine of APIM 2.0.0.

        -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> -Doptions=migrateThrottling,deployPolicies

        To migrate to the new throttling engine and deploy throttle policies to the Traffic Manager node.

        This migrates synapse config files, the APIM database with new throttle policies and deploys policies to the Traffic Manager node. Therefore, you must run this command against a node that has synapse config files, the AM_DB and is pointed to the Traffic Manager node.

        Note

        Use this command if you want to deploy the throttle policies directly to the Traffic Manager node while performing the migration.

        Tip

        Tip: If you are using a clustered/distributed API Manager setup, run with the following options -DmigrateDB=true -DmigrateReg=true -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> in the API Publisher node and the -DmigrateFS=true -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> options in the API Gateway node.

        Tip

        To migrate to the new throttling engine of APIM 2.0.0, you have to use a node which has AM_DB pointed as well as API gateway synapse configurations in place in the file system. If you use only "migrateThrottling" option, you can use API Gateway node. Make sure that AM_DB is pointed to the gateway during the migration time. You can the start the gateway server with the -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> -Doptions=migrateThrottling command. After completing this execution, make sure you copy the generated throttle policies in the <API-M_HOME>/executionplans directory to the <API-M_TRAFFIC_MANAGER>/repository/deployment/server/executionplans directory.

        If you use both the migrateThrottling,deployPolicies options, make sure that the Traffic Manager node is configured with the Gateway node and also make sure you avoid passing the -Dprofile option while performing the migration. With these options, you don't have to manually copy the throttling policies to the traffic manager. You can the start the gateway server with -Dcomponent=apim -DmigrateFromVersion=<CURRENT_API-M_VERSION> -Doptions=migrateThrottling,deployPolicies command.  

        Info
        If you have enabled token encryption in your database, as part of database migration, those consumer keys need to be decrypted. If you see the error below, it means that the migration client has failed to decrypt some of the keys in the old database. TID: [-1234] [] [2016-04-11 08:27:16,249] ERROR {org.wso2.carbon.apimgt.migration.client.MigrateFrom19to110} -  Cannot decrypt consumer key : DRHbt68uSU4+7xtCEIzuf42CMaqXbNjYl3OYVJ0VL/H6EsFo8GBRaZGUhLHlTWIHzYrvoeOpb1YbauvRRIN/b6VqEd9m8HuYOIuLkkDd AM_APPLICATION_KEY_MAPPING table {org.wso2.carbon.apimgt.migration.client.MigrateFrom19to110} If this occurs, by default, the migration client will terminate without updating databases in order to maintain database integrity. However, you can change this behavior by adding the following argument and running the APIM migration client again from the beginning. Please note that the database is then updated with the keys where decryption was successful and failed keys are permanently deleted.
        -DremoveDecryptionFailedKeysFromDB=true
        Info

        You may notice the following exception being thrown in the console when API Manager 2.0.0 is started at this point,

        org.wso2.carbon.idp.mgt.IdentityProviderManagementException: Error occurred while retrieving Identity Provider information for tenant : carbon.super and Identity Provider name : LOCAL

        This is due to the fact that the IDP_METADATA table does not exist in the API Manager database. This is not a cause for alarm and is to be expected since we are yet to complete the migration process. The instructions given from step 9 onwards will address the creation of the IDP_METADATA table after which this exception will no longer be thrown.

  5. Do the following to re-index the artifacts in the registry:

    1. Rename the <lastAccessTimeLocation> element in the <API-M_2.0.0_HOME>/repository/conf/registry.xml file. If you use a clustered/distributed API Manager setup, change the file in the API Publisher node. For example, change the /_system/local/repository/components/org.wso2.carbon.registry/indexing/lastaccesstime registry path to /_system/local/repository/components/org.wso2.carbon.registry/indexing/lastaccesstime_1

    2. Shut down the API Manager 2.0.0, back up and delete the <API-M_2.0.0_HOME>/solr directory.

    Info

    Steps 6,7 and 8 needs to be executed ONLY if the setup is migrating from 1.8.0, 1.9.0 or 1.9.1. If the current setup is in 1.10.0, steps 6, 7 and 8 are NOT required to be executed.

  6. Execute the following update queries against the WSO2AM_DB to set the GRANT_TYPE column of the IDN_OAUTH_CONSUMER_APPS table:

    Code Block
    UPDATE IDN_OAUTH_CONSUMER_APPS set GRANT_TYPES='refresh_token urn:ietf:params:oauth:grant-type:saml2-bearer implicit password iwa:ntlm client_credentials authorization_code' where CALLBACK_URL is not NULL; 
    
    UPDATE IDN_OAUTH_CONSUMER_APPS set GRANT_TYPES='refresh_token urn:ietf:params:oauth:grant-type:saml2-bearer password iwa:ntlm client_credentials' where CALLBACK_URL is NULL;  
  7. Download the WSO2 Identity Server Migration Client. This is required to migrate the Identity and User Store Database schemas that is used by API Manager. Unzip the downloaded file to the local file system where the wso2is-5.1.0-migration directory is created and do the following:

    Tip

    If you are running in a distributed setup, ensure that this step is only carried out for the Key Manager node.

    If you are running WSO2 Identity Server as a Key Manager in a distributed setup, first follow the instructions in Configuring the Identity Server 5.2.0 as a Key Manager with API Manager 2.0.0 before following the steps below. Make sure that the additional configurations you made in the following files for WSO2 Identity Server 5.1.0 are redone in the Identity Server 5.2.0 configuration files,

    master-datasources.xml registry.xml user-mgt.xml api-manager.xml

          a. Copy the <wso2is-5.1.0-migration>/dbscripts/migration­5.0.0_to_5.1.0 directory to the <API-M_2.0.0_HOME>/dbscripts directory.

    Note

    If you are using MySQL 5.7, carryout the following steps before proceeding to the next step.

    1. Navigate to the <wso2is-5.1.0-migration>/ dbscripts/identity/migration­5.0.0SP1_to_5.1.0 directory.

    2. Rename the mysql.sql file to mysql_old.sql or a name of your choice.

    3. Rename the mysql_57.sql file to - mysql.sql


          b. Copy the <wso2is-5.1.0-migration>/ dbscripts/identity/migration­5.0.0SP1_to_5.1.0 directory and the  <wso2is-5.1.0-migration>/ dbscripts/identity/migration­5.0.0_to_5.0.0SP1 directory to the <API-M_2.0.0_HOME>/dbscripts/identity directory.
          c. Copy the <wso2is-5.1.0-migration>/dropins/org.wso2.carbon.is.migrate.client-­5.1.0.jar file to the <API-M_2.0.0_HOME>/repository/components/dropins directory.    

  8. Before executing identity migration as given in the step below, verify whether the API Manager database has the IDN_ASSOCIATED_ID table. To do that, connect to the API Manager database and check whether the IDN_ASSOCIATED_ID table exists. If not, download the IDN Tables zip file and run the relevant script according to your RDBMS against your API Manager DB.  

  9. Start API Manager 2.0.0 with the command line option -Dmigrate -Dcomponent=identity to carry out the complete Identity and User Store DB migration. Alternatively, you can run it step by step by starting the server with the command line options given below in the following order. 
    • -Dmigrate -DmigrateIdentityDB -Dcomponent=identity
    • -Dmigrate -DmigrateUMDB -Dcomponent=identity
    • -Dmigrate -DmigrateIdentityData -Dcomponent=identity
    • -Dmigrate -DmigrateIdentityDBFinalize -Dcomponent=identity
    • -Dmigrate -DmigrateUMData -Dcomponent=identity
  10. Upgrading to IS 5.2.0 from 5.1.0

    1. Download 5.1.0 to 5.2.0 migration DB scripts from here.
    2. Unzip and copy it to the <API-M_2.0.0_HOME>/dbscripts/identity directory.
      Copying the DB scripts are optional as it is done for reference purposes to indicate the version to which you have upgraded your APIM IS component.
    3. Execute the DB scripts, which corresponds to the RDBMS that you are working with, manually.

      Note

      Apply the respective DB script that is inside the following directories against the respective DB as follows:

      DB script directoryApplicable DBDescription
      <wso2is-5.2.0-migration>/dbscripts/identity/migration-5.1.0_to_5.2.0 AM_DB
      <wso2is-5.2.0-migration>/dbscripts/migration-5.1.0_to_5.2.0 UM_DBThis script includes insert operations and some index creation scripts against the UM_CLAIM table which are required in order for you to upgrade from your previous APIM version to API-M 2.0.0.

      For example, if you are working with a MySQL DB run the mysql.sql script.

    Warning

    During the Identity and User Store DB migration, you may notice warning messages similar to the following:

    [2016-07-22 07:14:49,386] WARN - AppDeployerServiceComponent Waiting for required OSGi services: org.wso2.carbon.application.deployer.synapse.service.SynapseAppDeployerService,org.wso2.carbon.mediation.
    initializer.services.SynapseEnvironmentService, [2016-07-22 07:14:49,418] WARN - StartupFinalizerServiceComponent Waiting for required OSGi services: org.wso2.carbon.application.deployer.service.CappDeploymentService,org.wso2.carbon.server.admin.
    common.IServerAdmin,org.wso2.carbon.throttling.agent.ThrottlingAgent,

    Do not abort the migration process at this point as data migration is still happening in the background and aborting could result in data corruption. The time taken to finish depends on the amount of tokens in the database and could take hours depending on the amount of data.

...