This documentation is for WSO2 API Manager 1.10.0 View documentation for the latest release.
Upgrading from the Previous Release - API Manager 1.10.0 - WSO2 Documentation
||
Skip to end of metadata
Go to start of metadata

To upgrade from a version older than 1.8.0, follow the instructions in the document that was released immediately after your current release and upgrade incrementally.

Migrating the configurations

In this section, you move all existing API Manager configurations from the current environment to the new one.

  1. Back up the all databases in your API Manager instances and synapse configs of all the tenants and super tenant.
    You find the synapse configs of super tenant in the <CURRENT_APIM_HOME>/repository/deployment/server/synapse-config/default directory. Synapse configs of tenants are in the <CURRENT_APIM_HOME>/repository/tenants directory. 
    If you use a clustered/distributed API Manager setup, back up these configurations available in the API Gateway node.

  2. Download API Manager 1.10.0 from http://wso2.com/products/api-manager/.
  3. Open the <APIM_1.10.0_HOME>/repository/conf/datasources/master-datasources.xml file and provide the datasource configurations for the following databases. You can copy the configurations from the same file in the current API Manager instance already being used.

    • User Store
    • Registry database/s
    • API Manager Databases
  4. Edit the registry configurations in the <APIM_1.10.0_HOME>/repository/conf/registry.xml file and the user database in the <APIM_1.10.0_HOME>/repository/conf/user-mgt.xml file similar to same configurations of current API Manager.

    Note that in a clustered/distributed API Manager setup below 5, 6 steps needs to be done to the Gateway Node.

  5. Move all your synapse configurations except below mentioned files by copying and replacing the <CURRENT_APIM_HOME>/repository/deployment/server/synapse-config/default directory to the <APIM_1.10.0_HOME>/repository/deployment/server/synapse-config/default. 

    NOTE: Do not replace below files from <CURRENT_APIM_HOME>/repository/deployment/server/synapse-config/default to APIM 1.10. These are application-specific APIs and Sequences. If you had any custom changes to below files, please merge them to corresponding files in 1.10

    /api/_AuthorizeAPI_.xml
    /api/_RevokeAPI_.xml
    /api/_TokenAPI_.xml
    /sequences/_auth_failure_handler_.xml
    /sequences/_build_.xml
    /sequences/_cors_request_handler_.xml
    /sequences/fault.xml
    /sequences/main.xml
    /sequences/_production_key_error_.xml
    /sequences/_resource_mismatch_handler_.xml
    /sequences/_sandbox_key_error_.xml
    /sequences/_throttle_out_handler_.xml
    /sequences/_token_fault_.xml
    proxy-services/ WorkflowCallbackService.xml
  6. Move all your tenant synapse configurations by copying and replacing the  <CURRENT_APIM_HOME>/repository/tenants  directory to the  <APIM_1.10.0_HOME>/repository/tenants  directory.

    NOTE: Retrieve the list of files given below from the <APIM_1.10.0_HOME>/repository/deployment/server/synapse-configs/default/sequences directory and replace the corresponding files in each <APIM_1.10.0_HOME>/repository/tenants/<tenant-id>/synapse-configs/default/sequences directory.

    _auth_failure_handler_.xml
    _build_.xml
    _cors_request_handler_.xml
    fault.xml
    main.xml
    _production_key_error_.xml
    _resource_mismatch_handler_.xml
    _sandbox_key_error_.xml
    _throttle_out_handler_.xml
    _token_fault_.xml

Upgrading the API Manager to 1.10.0

  1. Stop all running API Manager server instances.
  2. Make sure that you have backed up all the databases and synapse configs as instructed in step 1 of the previous section.

  3. Download WSO2-CARBON-PATCH-4.4.0-0051 and apply it to all the APIM nodes by following the instructions given in the README.txt file inside. 
  4. Before you run the migration client, open the <APIM_1.10.0_HOME>/repository/conf/datasources/master-datasources.xml file, and set the <username>, and <password> elements of the AM_DB JNDI to that of a user who has permissions to alter tables in the database.

    Tip: After you are done running the migration client, you can switch these credentials back to a user with lesser privileges.

    For example,

    <datasource>
            ...
            <definition type="RDBMS">
                <configuration>
                    ...
                    <username>xxxxxx</username>
                    <password>xxxxxx</password>
                    ...
                </configuration>
             </definition>
    </datasource>
  5. Download and extract the WSO2 API Manager Migration Client and do the following:
    1. Copy the org.wso2.carbon.apimgt.migrate.client-1.10.X.jar file to the <APIM_1.10.0_HOME>/repository/components/dropins directory. If you use a clustered/distributed API Manager setup, copy the JAR file to all nodes.
    2. Copy the migration-scripts folder into <APIM_1.10.0_HOME>. If you use a clustered/distributed API Manager setup, copy the migration-scripts folder to the node that hosts your database.

      Note: If you are using MySQL 5.7, rename the following files found in the <APIM_1.10.0_HOME>/migration-scripts/18-19-migration directory, before proceeding.

      • mysql.sql to mysql_56.sql
      • mysql_57.sql to mysql.sql
  6. This step is required only if you are migrating from APIM 1.8.0. If you are not using the MySQL database with the API Manager, change the query inside the <APIM_1.10.0_HOME>/migration-scripts/18-19-migration/drop-fk.sql file according to your database type. The scripts for each database type are given below:

    Database typeScript
    MySQLNo changes are required as the default drop-fk.sql file already contains the scripts for MySQL.
    H2SELECT DISTINCT constraint_name FROM information_schema.constraints WHERE table_name = 'AM_APP_KEY_DOMAIN_MAPPING';
    ALTER TABLE AM_APP_KEY_DOMAIN_MAPPING DROP CONSTRAINT <temp_key_name>;
    OracleSELECT constraint_name FROM user_cons_columns WHERE table_name = 'AM_APP_KEY_DOMAIN_MAPPING' GROUP BY constraint_name HAVING COUNT(constraint_name) = 1;
    ALTER TABLE AM_APP_KEY_DOMAIN_MAPPING DROP CONSTRAINT <temp_key_name>;
    MS SQL ServerSELECT CONSTRAINT_NAME FROM INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE WHERE TABLE_NAME = 'AM_APP_KEY_DOMAIN_MAPPING' AND COLUMN_NAME='CONSUMER_KEY'; 
    ALTER TABLE AM_APP_KEY_DOMAIN_MAPPING DROP CONSTRAINT <temp_key_name>;
    Postgresql

    SELECT CONSTRAINT_NAME FROM information_schema.constraint_table_usage WHERE TABLE_NAME = 'AM_APP_KEY_DOMAIN_MAPPING';
    ALTER TABLE AM_APP_KEY_DOMAIN_MAPPING DROP CONSTRAINT <temp_key_name>;

  7. Start API Manager 1.10.0 with the following command-line options to migrate the database, registry and the file system together or separately.

    DescriptionCommand
    To migrate the database, registry and file system together, at the same time.  Ideally, you use this to migrate a standalone pack.-Dmigrate=true -Dcomponent=apim -DmigrateFromVersion=<1.8.0|1.9.0|1.9.1>
    To migrate the database only.  This migrates the AM_DB database. Please ensure that the <APIM_PUBLISHER>/repository/conf/datasources/master-datasources.xml file has an entry for AM_DB.-DmigrateDB=true -Dcomponent=apim -DmigrateFromVersion=<1.8.0|1.9.0|1.9.1>
    To migrate the registry only.  This migrates the registry-related resources such as .rxt and swagger definitions.-DmigrateReg=true -Dcomponent=apim -DmigrateFromVersion=<1.8.0|1.9.0|1.9.1>
    To migrate the file system only.  This migrates the synapse config files such as APIs that reside in the file system. Therefore, you must run this command on the Gateway node/s of a distributed APIM setup. -DmigrateFS=true -Dcomponent=apim -DmigrateFromVersion=<1.8.0|1.9.0|1.9.1>

    Tip: If you are using a standalone API Manager setup, you can migrate all the resources together with the -Dmigrate=true -Dcomponent=apim -DmigrateFromVersion=<1.8.0|1.9.0|1.9.1> command, or perform single-resource migrations using the other commands separately. If you execute -Dmigrate=true -Dcomponent=apim -DmigrateFromVersion=<1.8.0|1.9.0|1.9.1>, you do not need to execute the rest of the commands.

    Tip: If you are using a clustered/distributed API Manager setup, run with the following options -DmigrateDB=true -DmigrateReg=true -Dcomponent=apim -DmigrateFromVersion=<1.8.0|1.9.0|1.9.1> in the API Publisher node and the -DmigrateFS=true -Dcomponent=apim -DmigrateFromVersion=<1.8.0|1.9.0|1.9.1> options in the API Gateway node.

    If you have enabled token encryption in your database, as part of database migration, those consumer keys need to be decrypted. If you see the error below, it means that the migration client has failed to decrypt some of the keys in the old database.

    TID: [-1234] [] [2016-04-11 08:27:16,249] ERROR {org.wso2.carbon.apimgt.migration.client.MigrateFrom19to110} -  Cannot decrypt consumer key : DRHbt68uSU4+7xtCEIzuf42CMaqXbNjYl3OYVJ0VL/H6EsFo8GBRaZGUhLHlTWIHzYrvoeOpb1YbauvRRIN/b6VqEd9m8HuYOIuLkkDd AM_APPLICATION_KEY_MAPPING table {org.wso2.carbon.apimgt.migration.client.MigrateFrom19to110}

    If this occurs, by default, the migration client will terminate without updating databases in order to maintain database integrity. However, you can change this behavior by adding the following argument and running the APIM migration client again from the beginning. Please note that the database is then updated with the keys where decryption was successful and failed keys are permanently deleted.
    -DremoveDecryptionFailedKeysFromDB=true

    You may notice the following exception being thrown in the console when API Manager 1.10 is started at this point,

           org.wso2.carbon.idp.mgt.IdentityProviderManagementException: Error occurred while retrieving Identity Provider information for tenant : carbon.super and Identity Provider name : LOCAL

    This is due to the fact that the IDP_METADATA table does not exist in the API Manager database. This is not a cause for alarm and is to be expected since we are yet to complete the migration process. The instructions given from step 9 onwards will address the creation of the IDP_METADATA table after which this exception will no longer be thrown.

  8. Do the following to re-index the artifacts in the registry:

    1. Rename the <lastAccessTimeLocation> element in the <APIM_1.10.0_HOME>/repository/conf/registry.xml file. If you use a clustered/distributed API Manager setup, change the file in the API Publisher node. For example, change the /_system/local/repository/components/org.wso2.carbon.registry/indexing/lastaccesstime registry path to /_system/local/repository/components/org.wso2.carbon.registry/indexing/lastaccesstime_1

    2. Shut down the API Manager 1.10.0, back up and delete the <APIM_1.10.0_HOME>/solr directory.

  9. Download the WSO2 Identity Server Migration Client. This is required to migrate the Identity and User Store Database schemas that is used by API Manager. Unzip the downloaded file to the local file system where the wso2is-5.1.0-migration directory is created and do the following:

    Note: If you are using MySQL 5.7, rename the following files found in the <IS5.1.0_MIGRATION_HOME>/dbscripts/identity/migration­5.0.0SP1_to_5.1.0 directory, before proceeding.

    • mysql.sql to mysql_56.sql
    • mysql_57.sql to mysql.sql 

          a. Copy the wso2is-5.1.0-migration /dbscripts/migration­5.0.0_to_5.1.0 directory to the <APIM_1.10.0_HOME>/dbscripts directory
          b. Copy the wso2is-5.1.0-migration / dbscripts/identity/migration­5.0.0SP1_to_5.1.0 directory and the  wso2is-5.1.0-migration / dbscripts/identity/migration­5.0.0_to_5.0.0SP1 directory to the < APIM_1.10.0_HOME >/dbscripts/identity directory.
          c. Copy the wso2is-5.1.0-migration /dropins/org.wso2.carbon.is.migrate.client-­5.1.0.jar file to the < APIM_1.10.0_HOME >/repository/components/dropins directory.    

    If you are running in a distributed setup, ensure that the above step is only carried out for the key manager node.

  10. Before executing identity migration as given in the below step, verify whether the API Manager database has the IDN_ASSOCIATED_ID table. To do that, connect to API Manager database and check whether the IDN_ASSOCIATED_ID table exists. If not, download IDN Tables zip file and run the relevant script according to your RDBMS against your API Manager DB 

  11. Start API Manager 1.10.0 with the command line option -Dmigrate -Dcomponent=identity to carry out the complete Identity and User Store DB migration. Alternatively, you can run it step by step by starting the server with the command line options given below in the following order. 
    • -Dmigrate -DmigrateIdentityDB -Dcomponent=identity
    • -Dmigrate -DmigrateUMDB -Dcomponent=identity
    • -Dmigrate -DmigrateIdentityData -Dcomponent=identity
    • -Dmigrate -DmigrateIdentityDBFinalize -Dcomponent=identity
    • -Dmigrate -DmigrateUMData -Dcomponent=identity

    During the Identity and User Store DB migration, you may notice warning messages similar to the following:

    [2016-07-22 07:14:49,386] WARN - AppDeployerServiceComponent Waiting for required OSGi services: org.wso2.carbon.application.deployer.synapse.service.SynapseAppDeployerService,org.wso2.carbon.mediation.initializer.services.SynapseEnvironmentService,
    [2016-07-22 07:14:49,418] WARN - StartupFinalizerServiceComponent Waiting for required OSGi services: org.wso2.carbon.application.deployer.service.CappDeploymentService,org.wso2.carbon.server.admin.common.IServerAdmin,org.wso2.carbon.throttling.agent.ThrottlingAgent,

    Do not abort the migration process at this point as data migration is still happening in the background and aborting could result in data corruption. The time taken to finish depends on the amount of tokens in the database and could take hours depending on the amount of data.

This concludes the upgrade process.

Tip: The migration client that you use in this guide automatically migrates your tenants, workflows, external user stores etc. to the upgrade environment. There is no need to migrate them manually.

APIM Statistic Data Migration from WSO2 BAM to WSO2 DAS 

When moving from WSO2 BAM to WSO2 DAS, you have to follow the instructions below in order to migrate data from RDBMS to DAS. 

  1. Download and extract the WSO2 APIM Stat Migration file. 
  2. Take a backup of the APIM statistics DB. 
  3. Execute the relevant script in the wso2-api-stat-migration/dbscripts directory against your APIM statistics DB. 
  4. Add the APIM statistics DB as a datasource to WSO2 DAS.
  5. Start WSO2 DAS and copy the wso2-api-stat-migration/APIM_MIGRATION-1.9.0_to_1.10.0.car file to the <DAS_HOME>/repository/deployment/server/carbonapps directory.
  6. Login to DAS and click Batch Analytics > scripts and then click Edit on APIM_1_10_X_MIGRATION_MYSQL/APIM_1_10_X_MIGRATION_ORACLE based on your RDBMS. 

    If your RDBMS is oracle, then select APIM_1_10_X_MIGRATION_ORACLE. Else select APIM_1_10_X_MIGRATION_MYSQL.

  7. Make sure that the dataSourceName in the script (WSO2AM_STATS_DB) is the same as the JDBC name you have in the datasources (jdbc/WSO2AM_STATS_DB). If not, modify all occurrences of the datasource name in the script. 
  8. Click Execute Script
  9. Now all the old statistic data is in DAS. You can complete your DAS configuration to publish statistics by following Publishing API Runtime Statistics