||
Skip to end of metadata
Go to start of metadata

WSO2 API Manager is a complete API management solution, used for creating and publishing APIs, creating and managing a developer community, and scalably routing API traffic. The API Manager solution includes a Publisher, Store, Gateway, Key Manager, and Traffic Manager component.

Typically, when you get started with API Manager in a development environment, you deploy API Manager as a single, all-in-one instance with all its components on a single server. For details, see Deploying API Manager as an All-in-One Instance.

However, in a production deployment, these components will be deployed in a distributed manner. Before you set up a cluster for WSO2 API Manager, you can create a distributed deployment of its five main components: Publisher, Store, Gateway, Key Manager and Traffic Manager. This page describes how to set up the distributed deployment in the following sections.

Note that your configurations may vary depending on the API Manager clustering deployment pattern you choose. Additionally, these instructions are based on API Manager 2.0.0, which is based on Carbon 4.4.6. If you are using multi-tenancy, all nodes should use the same user store, as all servers are servicing the same set of tenants, and it has to share the same Governance Registry space across all nodes.

Understanding the API Manager architecture

API Manager uses the following main components:

Publisher

Enables API providers to easily publish their APIs, share documentation, provision API keys, and gather feedback on API features, quality, and usage.

Store

Enables consumers to self-register, discover API functionality, subscribe to APIs, evaluate them, and interact with API publishers.

Key Manager

Responsible for all security and key-related operations.

Gateway

Responsible for securing, protecting, managing, and scaling API calls. 

Traffic ManagerUsed to make a decision on throttling.

For more information on the above, see the main components of a distributed system.

Additionally, API Manager uses the following databases, which are shared among the server nodes.

  • User Manager database - Stores information related to users and user roles. This information is shared among the Key Manager Server, Store, and Publisher. Users can access the Publisher for API creation and the Store for consuming the APIs.
  • API Manager database - Stores information related to the APIs along with the API subscription details. The Key Manager Server uses this database to store user access tokens that are used for verification of API calls.
  • Registry database - Shares information between the Publisher and Store. When an API is published through the Publisher, it is made available in the Store via the sharing registry database. Although you would normally share information between the Publisher and Store components only, if you are planning to create this setup for a multi-tenanted environment (create and work with tenants), it is required to share the information in this database between the Gateway and Key Manager components as well. 
  • Statistics database - Stores information related to API statistics. After you configure APIM analytics, it writes summarized data to this database. The Publisher and Store can then query this database to display the statistics data.
  • Message Broker database - Traffic Manager uses this database as the message store for broker when advanced throttling is used.

The API Manager components use the databases as follows:



 

(API Manager Database)

apimgtdb

(User Manager Database)

userdb

(Registry Database)

regdb

(Statistics Database)

statdb

(Message Broker Database)

mbstoredb

Publisher

Used

Used

Used

UsedNot used

Store

Used

Used

Used

UsedNot used

Key Manager

Used

Used

Used (in multi-tenancy mode)

Not usedNot used

Gateway

Not used

Used (in multi-tenancy mode)

Used (in multi-tenancy mode)

Not usedNot used
Traffic ManagerNot usedNot usedNot usedNot usedUsed

Notes

  • Although the Gateway does not use the API Manager database, this connection is required so do not remove the default configuration in the <APIM_HOME>/repository/conf/datasources/master-datasources.xml file.
  • If you have more than one Traffic Manager node, each Traffic Manager node must have its own WSO2_MB_STORE_DB database.

When we consider distributed deployment of WSO2 API Manager, we have the option of separating the five components and clustering each component as needed. Let's look more closely at how the API Manager components are deployed separately.

Creating the distributed deployment

In the following diagram, the five components are set up in a distributed deployment, and the three databases are connected to the relevant components respectively. The entire setup is also fronted by a load balancer. Notice the port offset and the HTTPS port of each WSO2 server component.

Note: The default communication protocol of Key Manager is Thrift. If your setup has multiple Key Manager nodes that are fronted by a HTTP load balancer, change the key management protocol from Thrift to WSClient using the <KeyValidatorClientType> element in the <APIM_HOME>/repository/conf/api-manager.xml file. Thrift supports TCP load balancing.

To achieve the above deployment, do the following configurations.

Installing and configuring API Manager

The following steps describe how to download, install, and configure API Manager. We will create five instances of API Manager for this.

  1. Download the WSO2 API Manager in each of the five servers in the cluster for distributed deployment.
  2. Unzip the API Manager zipped archive, and rename each of those directories respectively as Key Manager, Gateway, Publisher, Store, and Traffic Manager. These five directories are located in a server of their own and will be used for each component of the API Manager. Each of these unzipped directories will be referred to as <APIM_HOME> or <PRODUCT_HOME> in this document.
  3. Only do this step if you wish to run WSO2 servers (TrafficManager, KeyManager, Gateway, Publisher, and Store) on the same machine. You need to set the Offset attribute to avoid any port usage conflict, as follows:

    1. In each of the created directories, open the <PRODUCT_HOME>/repository/conf/carbon.xml file.

    2. Edit the <Offset> attribute in each file as shown in the Port Offset column in the following table. The port value will automatically be increased as shown in the Port Value column, allowing all five WSO2 server instances to run on the same machine.

      WSO2 Server instance

      Port Offset

      Port Value

      Key Manager

      0

      9443

      Gateway

      1

      9444

      Publisher

      2

      9445

      Store

      3

      9446

      Traffic Manager49447

      This step is optional and only required if all server instances are running in the same machine. This is not a recommended approach for production environments. Note that you need to change all ports used in your configurations based on the offset value if you are to do this. See Changing the Default Ports with Offset for more information. This document does NOT use port offsets within the configurations.

  4. In each of the five servers, replace the default certificates (where CN=localhost) with new certificates generated with proper CN values, in order to avoid seeing an error stating that the hostname in the certificate didn't match.
  5. Optionally, you can remove the web apps that are unnecessary to each node from the <APIM_HOME>/repository/deployment/server/webapps directory. The web apps required for each node are listed below.

    ProfileRequired web apps
    Gateway Manageram#sample#pizzashack#v1 (only needed if the Pizzashack sample API is used)
    api#am#admin#v0.10 (only needed if you want to perform administrative tasks through Gateway Manager)
    authenticationendpoint
    Gateway Workeram#sample#pizzashack#v1 (only needed if the Pizzashack sample API is used)
    api#am#admin#v0.10 (only needed if you want to perform administrative tasks through Gateway Manager)
    Key Managerauthenticationendpoint
    client-registration#v0.10
    oauth2 throttle#data#v1
    Traffic Managershindig (only needed if you use the WSO2 Remove existing web apps and jaggery apps from the <APIM_HOME>/repository/deployment/server directory.CEP Dashboard)
    API Publisheram#sample#pizzashack#v1 (only needed if the Pizzashack sample API is used)
    api#am#publisher#v0.10
    authenticationendpoint
    API Store (Developer Portal)api#am#store#v0.10
    authenticationendpoint

Installing and configuring the databases

The following steps describe how to download and install MySQL Server, create the databases, configure the data sources, and configure the API Manager components to connect to them. Additionally, you may need to configure your API Analytics platform with API Manager in the distributed setup.

  1. Download and install MySQL Server.

  2. Download the MySQL JDBC driver.

  3. Unzip the downloaded MySQL driver zipped archive, and copy the MySQL JDBC driver JAR (mysql-connector-java-x.x.xx-bin.jar) into the <PRODUCT_HOME>/repository/components/lib directory of all the nodes in the cluster.

  4. Enter the following command in a terminal/command window, where username is the username you want to use to access the databases:
    mysql -u username -p
  5. When prompted, specify the password that will be used to access the databases with the username you specified.
  6. Create the databases using the following commands, where <APIM_HOME> is the path to any of the API Manager instances you installed, and username and password are the same as those you specified in the previous steps.

    About using MySQL in different operating systems

    For users of Microsoft Windows, when creating the database in MySQL, it is important to specify the character set as latin1. Failure to do this may result in an error (error code: 1709) when starting your cluster. This error occurs in certain versions of MySQL (5.6.x) and is related to the UTF-8 encoding. MySQL originally used the latin1 character set by default, which stored characters in a 2-byte sequence. However, in recent versions, MySQL defaults to UTF-8 to be friendlier to international users. Hence, you must use latin1 as the character set as indicated below in the database creation commands to avoid this problem. Note that this may result in issues with non-latin characters (like Hebrew, Japanese, etc.). The following is how your database creation command should look.

    mysql> create database <DATABASE_NAME> character set latin1;

    For users of other operating systems, the standard database creation commands will suffice. For these operating systems, the following is how your database creation command should look.

    mysql> create database <DATABASE_NAME>;

    Additional notes

    • Ensure that MySQL is configured so that all nodes can connect to it. 
    • If you are using MySQL version 5.7 or later, use the mysql5.7.sql script instead of the mysql.sql script.
    • Table creation of the statistics database is handled by the Analytics scripts when you configure APIM Analytics, so you will create the statistics database in this step but will not specify a source script.
    mysql> create database apimgtdb;
    mysql> use apimgtdb;
    mysql> source <APIM_HOME>/dbscripts/apimgt/mysql.sql;
    mysql> grant all on apimgtdb.* TO username@localhost identified by "password";
    
    mysql> create database userdb;
    mysql> use userdb;
    mysql> source <APIM_HOME>/dbscripts/mysql.sql;
    mysql> grant all on userdb.* TO username@localhost identified by "password";
    
    mysql> create database regdb;
    mysql> use regdb;
    mysql> source <APIM_HOME>/dbscripts/mysql.sql;
    mysql> grant all on regdb.* TO username@localhost identified by "password"; 
     
    mysql> create database statdb;
    mysql> grant all on statdb.* TO username@localhost identified by "password"; 
     
    mysql> create database mbstoredb;
    mysql> use mbstoredb;
    mysql> source <APIM_HOME>/dbscripts/mb-store/mysql-mb.sql;
    mysql> grant all on mbstoredb.* TO username@localhost identified by "password"; 
  7. Configure the data sources for the five databases as follows:

    1. Open the <APIM_HOME>/repository/conf/datasources/master-datasources.xml file in all five API Manager components.

      Note: When configuring clustering, ignore the WSO2_CARBON_DB data source configuration.

    2. Enable the components to access the API Manager database by modifying the WSO2AM_DB data source in the master-datasources.xml files in the Publisher, Store, and Key Manager nodes by changing the URL as indicated below, replacing apimgtdb.mysql-wso2.com with the hostname you specified in step 4 (carbondb.mysql-wso2.com). 

      <datasource>
       <name>WSO2AM_DB</name>
       <description>The datasource used for the API Manager database</description>
       <jndiConfig>
         <name>jdbc/WSO2AM_DB</name>
       </jndiConfig>
       <definition type="RDBMS">
         <configuration>
           <url>jdbc:mysql://apimgtdb.mysql-wso2.com:3306/apimgtdb?autoReconnect=true</url>
           <username>user</username>
           <password>password</password>
           <defaultAutoCommit>false</defaultAutoCommit>
           <driverClassName>com.mysql.jdbc.Driver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
         </configuration>
       </definition>
      </datasource>
    3. Enable the Key Manager, Publisher, and Store components to access the user management database by configuring the WSO2UM_DB data source in their master-datasources.xml files as follows:

      <datasource>
       <name>WSO2UM_DB</name>
       <description>The datasource used by user manager</description>
       <jndiConfig>
         <name>jdbc/WSO2UM_DB</name>
       </jndiConfig>
       <definition type="RDBMS">
         <configuration>
           <url>jdbc:mysql://userdb.mysql-wso2.com:3306/userdb?autoReconnect=true</url>
           <username>user</username>
           <password>password</password>
           <driverClassName>com.mysql.jdbc.Driver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
         </configuration>
       </definition>
      </datasource> 
    4. Enable the Publisher and Store components to access the registry and statistics databases by configuring the WSO2REG_DB and WSO2AM_STATS_DB data sources in their master-datasources.xml files as follows:

      <datasource>
       <name>WSO2REG_DB</name>
       <description>The datasource used by the registry</description>
       <jndiConfig>
         <name>jdbc/WSO2REG_DB</name>
       </jndiConfig>
       <definition type="RDBMS">
         <configuration>
           <url>jdbc:mysql://regdb.mysql-wso2.com:3306/regdb?autoReconnect=true</url>
           <username>user</username>
           <password>password</password>
           <driverClassName>com.mysql.jdbc.Driver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
         </configuration>
       </definition>
      </datasource> 
       
      <datasource>
       <name>WSO2AM_STATS_DB</name>
       <description>The datasource used for getting statistics to API Manager</description>
       <jndiConfig>
         <name>jdbc/WSO2AM_STATS_DB</name>
       </jndiConfig>
       <definition type="RDBMS">
         <configuration>
           <url>jdbc:mysql://statdb.mysql-wso2.com:3306/statdb?autoReconnect=true</url>
           <username>user</username>
           <password>password</password>
           <driverClassName>com.mysql.jdbc.Driver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
         </configuration>
       </definition>
      </datasource> 
    5. Enable the Traffic Manager component to access the Message Broker database by configuring the WSO2_MB_STORE_DB data source in its master-datasources.xml file as follows:

      <datasource>
       <name>WSO2_MB_STORE_DB</name>
       <description>The datasource used for message broker database</description>
       <jndiConfig>
         <name>WSO2MBStoreDB</name>
       </jndiConfig>
       <definition type="RDBMS">
         <configuration>
           <url>jdbc:mysql://mbstoredb.mysql-wso2.com:3306/mbstoredb?autoReconnect=true</url>
           <username>user</username>
           <password>password</password>
           <driverClassName>com.mysql.jdbc.Driver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
      	 <defaultAutoCommit>false</defaultAutoCommit>
         </configuration>
       </definition>
      </datasource> 
    6. If you created the databases on other servers instead of your local machine, map the IP addresses to the data source URLs by opening the /etc/hosts file and adding an entry for each remote database. The following is an example of this.

      • 127.0.0.1 apimgtdb.mysql-wso2.com

      • 127.0.0.2 userdb.mysql-wso2.com

      • 127.0.0.3 regdb.mysql-wso2.com

      • 127.0.0.4 statdb.mysql-wso2.com
      • 127.0.0.5 mbstoredb.mysql-wso2.com
  8. To give each of the components access to the API Manager database, open the <APIM_HOME>/repository/conf/api-manager.xml file in each of the components and add the following line as the first child node of the root element (if it is not already there): 
    <DataSourceName>
    jdbc/WSO2AM_DB </DataSourceName>
  9. To give the Key Manager, Publisher, and Store components access to the user management database with shared permissions, open the <APIM _HOME>/repository/conf/user-mgt.xml file in each of these three components and add or modify the dataSource property of the <configuration> element as follows:

    <configuration> 
    ...
    	<Property name="dataSource">jdbc/WSO2UM_DB</Property>
    </configuration>
     
    <UserStoreManager class="org.wso2.carbon.user.core.jdbc.JDBCUserStoreManager">
    	<Property name="TenantManager">org.wso2.carbon.user.core.tenant.JDBCTenantManager</Property>
        <Property name="ReadOnly">false</Property>
        <Property name="MaxUserNameListLength">100</Property>
        <Property name="IsEmailUserName">false</Property>
        <Property name="DomainCalculation">default</Property>
        <Property name="PasswordDigest">SHA-256</Property>
        <Property name="StoreSaltedPassword">true</Property>
        <Property name="ReadGroups">true</Property>
        <Property name="WriteGroups">true</Property>
        <Property name="UserNameUniqueAcrossTenants">false</Property>
        <Property name="PasswordJavaRegEx">^[\S]{5,30}$</Property>
        <Property name="PasswordJavaScriptRegEx">^[\S]{5,30}$</Property>
        <Property name="UsernameJavaRegEx">^[^~!#$;%^*+={}\\|\\\\&lt;&gt;,\'\"]{3,30}$</Property>
        <Property name="UsernameJavaScriptRegEx">^[\S]{3,30}$</Property>
        <Property name="RolenameJavaRegEx">^[^~!#$;%^*+={}\\|\\\\&lt;&gt;,\'\"]{3,30}$</Property>
        <Property name="RolenameJavaScriptRegEx">^[\S]{3,30}$</Property>
        <Property name="UserRolesCacheEnabled">true</Property>
        <Property name="MaxRoleNameListLength">100</Property>
        <Property name="MaxUserNameListLength">100</Property>
        <Property name="SharedGroupEnabled">false</Property>
        <Property name="SCIMEnabled">false</Property>
    </UserStoreManager>
  10. To give the Publisher and Store components access to the registry database, open the <APIM_HOME>/repository/conf/registry.xml file in each of these two components and configure them as follows:

    Although it is mentioned that you need to do this on the Publisher and Store components only, if you are planning to create this setup for a multi-tenanted environment (create and work with tenants), it is required to perform the steps below on the Gateway and Key-Manager components as well.

    Note: Do not replace the following configuration when adding in the mounting configurations. The registry mounting configurations mentioned in the below steps must be added in addition to the following.

    <dbConfig name="wso2registry">
    	<dataSource>jdbc/WSO2CarbonDB</dataSource>
    </dbConfig>
    1. In the Publisher component's registry.xml file, add or modify the dataSource attribute of the <dbConfig name="govregistry"> element as follows:

      <dbConfig name="govregistry">
        <dataSource>jdbc/WSO2REG_DB</dataSource>
      </dbConfig>
      <remoteInstance url="https://publisher.apim-wso2.com"> 
         <id>gov</id>
         <cacheId>user@jdbc:mysql://regdb.mysql-wso2.com:3306/regdb</cacheId>
         <dbConfig>govregistry</dbConfig>
         <readOnly>false</readOnly>
         <enableCache>true</enableCache>
         <registryRoot>/</registryRoot>
      </remoteInstance>
      <mount path="/_system/governance" overwrite="true">
         <instanceId>gov</instanceId>
         <targetPath>/_system/governance</targetPath>
      </mount>
      <mount path="/_system/config" overwrite="true">
         <instanceId>gov</instanceId>
         <targetPath>/_system/config</targetPath>
      </mount>
    2. In the Store component's registry.xml file, add or modify the dataSource attribute of the <dbConfig name="govregistry"> element as follows (note that this configuration is nearly identical to the previous step with the exception of the remoteInstance URL):

      <dbConfig name="govregistry">
        <dataSource>jdbc/WSO2REG_DB</dataSource>
      </dbConfig>
      <remoteInstance url="https://store.apim-wso2.com"> 
         <id>gov</id>
         <cacheId>user@jdbc:mysql://regdb.mysql-wso2.com:3306/regdb</cacheId> 
         <dbConfig>govregistry</dbConfig>
         <readOnly>false</readOnly>
         <enableCache>true</enableCache>
         <registryRoot>/</registryRoot>
      </remoteInstance>
      <mount path="/_system/governance" overwrite="true">
         <instanceId>gov</instanceId>
         <targetPath>/_system/governance</targetPath>
      </mount>
      <mount path="/_system/config" overwrite="true">
         <instanceId>gov</instanceId>
         <targetPath>/_system/config</targetPath>
      </mount>
    3. Modify the /etc/hosts file to map the relevant IP addresses to the remoteInstance URLs. For example:

      • 127.0.0.1 publisher.apim-wso2.com

      • 127.0.0.1 store.apim-wso2.com

  11. Once registry caching is enabled, sync the published APIs between the Publisher and Store nodes by enabling clustering in both Store and Publisher nodes. To do this, open the <APIM_HOME>/repository/conf/axis2/axis2.xml file in each of these two components and configure them as follows:
    <clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

Configuring the connections among the components

You will now configure the inter-component relationships of the distributed setup by modifying their <APIM_HOME>/repository/conf/api-manager.xml files.

In a clustered environment, you use Session Affinity to ensure that requests from the same client always get routed to the same server.

It is mandatory to set up Session Affinity in the load balancers that front the Publisher and Store clusters, and it is optional in the load balancer (if any) that fronts a Key Manager cluster or Gateway Cluster.

However, authentication via session ID fails when Session Affinity is disabled in the load balancer.

First time authentication happens via Basic Auth and the Gateway gets a cookie. This cookie is used in every consequent request along with the Basic Auth credentials. The admin service validates the cookie and if the validation fails it re-authenticate using basic auth and issues a new cookie.

This section includes the following sub-topics.

Configuring the API Publisher

This section involves setting up the API Publisher node and enabling it to work with the other components in the distributed setup.

  1. Open the <APIM_HOME>/repository/conf/api-manager.xml files in the API Publisher node.
  2. Configure the API Gateway node using the following configurations. This configuration is used to publish synapse definition in the Gateway node.

    <AuthManager>
    	<ServerURL>https://<IP of the Key Manager>:9443/services/</ServerURL>
    	<Username>admin</Username>
    	<Password>admin</Password>
    	<CheckPermissionsRemotely>false</CheckPermissionsRemotely>
    </AuthManager>
     
    <APIGateway>
       <Environments>
           <Environment type="hybrid" api-console="true">
                <Name>Production and Sandbox</Name>
                <Description>This is a hybrid gateway that handles both production and sandbox token traffic.</Description>           
    			<ServerURL>https://<API-Gateway-Manager-Host>:9443/services/</ServerURL>
                <Username>${admin.username}</Username>
                <Password>${admin.password}</Password>          
    			<GatewayEndpoint>http://<API-Gateway-Worker-Host>:8280,https://<API-Gateway-Worker-Host>:8243</GatewayEndpoint>
           </Environment>
       </Environments>
    </APIGateway>

    If you have set the CheckPermissionRemotely parameter as true, the permissions will be checked in the remote server set in ServerURL. If the parameter is set as false the permissions will be checked by the local server

  3. The following configurations are for configuring Traffic Manager node-related configuration in the API Publisher. This configuration is used to publish Throttling Policies, Custom Templates, and Block conditions into the Gateway node.

    <ThrottlingConfigurations> 
    	<EnableAdvanceThrottling>true</EnableAdvanceThrottling>
        <DataPublisher>
    	       <Enabled>false</Enabled>
    		  ……………………
        </DataPublisher>
        <PolicyDeployer>
            <ServiceURL>https://<Traffic-Manager-Host>:9443/services/</ServiceURL>
            <Username>admin</Username>
            <Password>admin</Password>
        </PolicyDeployer>
        <BlockCondition>
            <Enabled>false</Enabled>
    		………………
        </BlockCondition>
        <JMSConnectionDetails>
            <Enabled>false</Enabled>
       		 ……………………
        </JMSConnectionDetails>
        <JMSEventPublisherParameters>
                <java.naming.factory.initial>org.wso2.andes.jndi.PropertiesFileInitialContextFactory</java.naming.factory.initial>
                <java.naming.provider.url>repository/conf/jndi.properties</java.naming.provider.url>
                <transport.jms.DestinationType>topic</transport.jms.DestinationType>
                <transport.jms.Destination>throttleData</transport.jms.Destination>
                <transport.jms.ConcurrentPublishers>allow</transport.jms.ConcurrentPublishers>
                <transport.jms.ConnectionFactoryJNDIName>TopicConnectionFactory</transport.jms.ConnectionFactoryJNDIName>
        </JMSEventPublisherParameters>
    	 ………………………………
    </ThrottlingConfigurations>
  4. Set the <DisplayURL> to true and provide the URL of the Store.

    When using a distributed/clustered API Manager setup , this configuration must be done in the API Publisher node/s.

    Example
    <APIStore>  
           <DisplayURL>true</DisplayURL>     
           <URL>https://<hostname>:9443/store</URL>
    </APIStore>
    • <hostname> - The hostname of the API Store node.
  5. Open the <APIM_HOME>/repository/conf/jndi.properties file and make the following changes.

    connectionfactory.TopicConnectionFactory = amqp://admin:admin@clientid/carbon?brokerlist='tcp://<Traffic-Manager-host>:5672'
    topic.throttleData = throttleData
Configuring the API Store

This section involves setting up the API Store node and enabling it to work with the other components in the distributed setup.

  1. Open the <APIM_HOME>/repository/conf/api-manager.xml file in the Store component.
  2. Make the following changes to the api-manager.xml file.

    <RevokeAPIURL>https://<API-Gateway-Worker-Host>:8243/revoke</RevokeAPIURL>
     
    <APIKeyValidator>
    	<ServerURL>https://<IP of the Key Manager>:9443/services/</ServerURL>
    	<Username>admin</Username>
    	<Password>admin</Password>
    ...
    </APIKeyValidator>
    
    
    <AuthManager>
    	<ServerURL>https://<IP of the Key Manager>:9443/services/</ServerURL>
    	<Username>admin</Username>
    	<Password>admin</Password>
    </AuthManager>
    
    
    <APIGateway>
    	<Environments>
    		<Environment type="hybrid">
    		...
    			<ServerURL>https://<API-Gateway-Manager-Host>:9443/services/</ServerURL>
    			<Username>admin</Username>
    			<Password>admin</Password> 
    			<GatewayEndpoint>http://<API-Gateway-Worker-Host>:8280,https://<API-Gateway-Worker-Host>:8243</GatewayEndpoint>
    		</Environment>
    	</Environments> 
    ...
    </APIGateway>
  3. Additionally, do the following changes. These are related to the Traffic Manager.

    <ThrottlingConfigurations>
            <EnableAdvanceThrottling>true</EnableAdvanceThrottling>
            <DataPublisher>
                <Enabled>false</Enabled>
    	    ……………………
            </DataPublisher> 
    	    …………………
            <BlockCondition>
                <Enabled>false</Enabled>
    	    ………………………
            </BlockCondition>
            <JMSConnectionDetails>
                <Enabled>false</Enabled>
    	     …………………………………
            </JMSConnectionDetails>
         ………………………………
    </ThrottlingConfigurations>
Configuring the Gateway

This section involves setting up the Gateway node and enabling it to work with the other components in the distributed setup.

  1. Open the <APIM_HOME>/repository/conf/api-manager.xml file in the Gateway node.
  2. Modify the api-manager.xml file as follows. This configures the connection to the Key Manager component.

    <APIKeyValidator> 
      <ServerURL>https://<IP of the Key Manager>:9443/services/</ServerURL>
      <Username>admin</Username>
      <Password>admin</Password>
      ...
    </APIKeyValidator>
  3. Configure key management related communication.

    In a clustered setup if the Key Manager is fronted by a load balancer, you have to use WSClient as KeyValidatorClientType in <APIM_HOME>/repository/conf/api-manager.xml. This should be configured in all Gateway and Key Manager components.

    <KeyValidatorClientType>WSClient</KeyValidatorClientType>
    1. In a clustered setup if the Key Manager is NOT fronted by a load balancer, you have to use ThriftClient as KeyValidatorClientType in <APIM_HOME>/repository/conf/api-manager.xml. This should be configured in all Gateway and Key Manager components.

      <KeyValidatorClientType>ThriftClient</KeyValidatorClientType>
    2. Ensure that the Thrift server is enabled only in the Key Manager. This is enabled by default in all instances of the product, so you need to disable the Thrift server in the Gateway, the Publisherand the Store by setting EnableThriftServer to false in <APIM_HOME>/repository/conf/api-manager.xml for each node.

      <EnableThriftServer>false</EnableThriftServer>
    3. Specify the ThriftClientPort and ThriftServerPort values. 10397 is the default.

      <ThriftClientPort>10397</ThriftClientPort>
      <ThriftServerPort>10397</ThriftServerPort>
    4. Specify the host name or IP of the Key Manager. The default is localhost. In a distributed deployment we must set this parameter in both Key Manager nodes and Gateway nodes only if the Key Manager is running on a separate machine. Gateway uses this parameter to connect to the key validation thrift service.

      <ThriftServerHost>localhost</ThriftServerHost>
  4. If you need to enable JWT you have to enable it in all the Gateway and key-manager components. Refer Generating JSON Web Token on how to configure JWT.

  5. Configure throttling for the Traffic Manager node.

    Note: 9611 and 9711 are the Traffic Manager receiver ports for the Binary type.

    <ThrottlingConfigurations>
            <EnableAdvanceThrottling>true</EnableAdvanceThrottling>
            <DataPublisher>
                <Enabled>true</Enabled>
                <Type>Binary</Type>
                <ReceiverUrlGroup>tcp://<Traffic-Manager-host>:9611</ReceiverUrlGroup>
                <AuthUrlGroup>ssl://<Traffic-Manager-host>:9711</AuthUrlGroup>
    		……………………
            </DataPublisher>
            <PolicyDeployer> 
    			<ServiceURL>https://<Traffic-Manager-host>:9443/services/</ServiceURL>
    		………………
            </PolicyDeployer>
    		………………
            <JMSConnectionDetails>
                <Enabled>true</Enabled>
                <ServiceURL>tcp://<Traffic-Manager-host>:5672</ServiceURL>
    		…………
            </JMSConnectionDetails>
    </ThrottlingConfigurations>
  6. Comment out the following section to prevent warning messages during startup of the gateway node in the cluster setup:

    <JMSEventPublisherParameters>
    	<java.naming.factory.initial>org.wso2.andes.jndi.PropertiesFileInitialContextFactory</java.naming.factory.initial>
    	<java.naming.provider.url>repository/conf/jndi.properties</java.naming.provider.url>
    	<transport.jms.DestinationType>topic</transport.jms.DestinationType>
    	<transport.jms.Destination>throttleData</transport.jms.Destination>
    	<transport.jms.ConcurrentPublishers>allow</transport.jms.ConcurrentPublishers>
    	<transport.jms.ConnectionFactoryJNDIName>TopicConnectionFactory</transport.jms.ConnectionFactoryJNDIName>
    </JMSEventPublisherParameters>
Configuring the Key Manager

No changes are required in the Key Manager node to enable it to work with the other components in the distributed setup.

Configuring the Traffic Manager

This section involves setting up the Traffic Manager node and enabling it to work with the other components in the distributed setup.

  1. Replace the <APIM_HOME>/repository/conf/registry.xml file with the <APIM_HOME>/repository/conf/registry_TM.xml file.
  2. Replace the <APIM_HOME>/repository/conf/axis2/axis2.xml file with the <APIM_HOME>/repository/conf/axis2/axis2_TM.xml file.
  3. Remove existing web apps and jaggery apps from the <APIM_HOME>/repository/deployment/server directory.

    If you use the WSO2 CEP Dashboard, keep the shindig web app.

  4. Start the Traffic Manager node using the following command.
    sh wso2server.sh -Dprofile=traffic-manager

Configuring the API Publisher and API Store to be on the same cluster domain

As explained in the sections above, the API Publisher and API Store use a shared registry for sharing API artifacts. Whenever an artifact is added/modified on the Publisher, the change must be reflected on the API Store appropriately. Due to the fact that the API Store fetches artifacts from the registry cache, modifications to artifacts done on the Publisher may not immediately be reflected on the Store unless the relevant cache entry is explicitly cleared off or the cache expires. For the cache to be explicitly cleared off, the Publisher and Store need to be in the same cluster domain and need to be aware of each other's existence. See Configuring Caching for more information.

A Gateway node must be up and running for you to create APIs through the API Publisher.

Follow the steps below to configure the Publisher and Store to be in a single cluster domain. You need to configure both nodes as instructed below.

  1. Open the <AM_HOME>/repository/conf/axis2/axis2.xml file and scroll down to the 'Clustering' section.
  2. Set the 'enable' attribute of the <clustering> element to true.

    <clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
  3. Change the 'membershipScheme' parameter to 'wka'.

    <parameter name="membershipScheme">wka</parameter>
  4. Provide a domain for the cluster.

    <parameter name="domain">wso2.pub.store.domain</parameter>
  5. Specify the 'localMemberHost' and 'localMemberPort' parameters. If on the publisher, the 'localMemberHost' should be the publisher's IP address. The port value should be the port on which the Publisher will be listening for incoming cluster messages. Same applies to the Store.

    <parameter name="localMemberHost">192.168.10.1</parameter>
    <parameter name="localMemberPort">4000</parameter>
  6. Specify well known member. When specifying the well known member, the Publisher should specify the Store information and the Store should specify the Publisher information. The port that we provide here should be equal to the 'localMemberPort' of the other member.

    <members>
       <member>
           <hostName>192.168.10.2</hostName>
           <port>4000</port>
       </member>
    </members>
  7. Save and close the file and restart the servers (if running) for the changes to take effect.

Creating the clustered deployment

The distributed API Manager setup can be extended to a clustered deployment by adding a cluster of Gateway components or adding a cluster of Key Manager components.

Creating a cluster of Gateway components

To cluster API Manager, create a cluster of Gateway components. To cluster the Gateway component, use the steps in Clustering the Gateway.

Traffic Manager scalable deployment patterns

See the article on Scalable Traffic Manager deployment patterns part 1 and part 2.

Starting the cluster

You can start up the nodes in any order that you prefer, but it is recommended to setup the Traffic Manager first. You can start the nodes using profiles or using the standard startup script.

  • No labels