Deploying WSO2 API-M in a Distributed Setup

The following steps describe how to download, install, and configure WSO2 API Manager, with five instances.

1. Download the WSO2 API Manager in each of the five servers in the cluster for distributed deployment.

2. Unzip the WSO2 API Manager zipped archive, and rename each of those directories respectively as Key Manager, Gateway, Publisher, Store, and Traffic Manager.
   These five directories are located in a server of their own and are used for each component of WSO2 API-M. Each of these unzipped directories are referred to as <API-M_HOME> or <PRODUCT_HOME> in this document.

3. In each of the five servers, replace the default certificates (where CN=localhost) with new certificates generated with proper common name (CN) values.
   You need to do this in order to avoid getting an error with regard to the fact that the hostname in the certificate did not match.

   Note that you should use the same primary keystore for all the API Manager instances here in order to decrypt the registry resources. For more information, see Configuring Primary Keystores in the Administration Guide. When creating the keystore, always use a longer validity period so that it will avoid the need of migration on the registry data when shifting to a new keystore.

This section involves setting up the Key Manager node and enabling it to work with the other components in a distributed deployment.

1. Open the <API-M_HOME>/repository/conf/api-manager.xml file in the Key Manager node and change the <ServerURL> element that appears under the <APIGateway> section, so that it points to the API Manager Gateway.  
   You need to add these configurations so that when a user is deleted or when the role of a user is updated in the Key Manager, it will update the Gateway cache by clearing the cache entries of a particular user. 
   • If you are working with a single Gateway in distributed set up, you need to replace [GATEWAY_SERVER_HOST] with the host of the Gateway node.
   • If you are working with Gateways in a High Availability (HA) setup that uses a shared file system (e.g., NFS), you need to replace [GATEWAY_SERVER_HOST] with the host of the Gateway load balancer node.
   • If you are working with Gateways in a High Availability (HA) setup that uses rsync, you need to replace [GATEWAY_SERVER_HOST] with the host of the Gateway Manager node.

   • You need to replace [port] with the management transport port. For more information, see Default Product Ports.

     <ServerURL>https://$[GATEWAY_SERVER_HOST]:[port]/services/</ServerURL>

2. Configure the API key validator in the Key Manager.
   The Thrift protocol is normally enabled by default. However, if you have disabled the Thrift protocol, enable it as follows in the <API-M_HOME>/repository/conf/api-manager.xml file. 

   • Single Key Manager
   • Key Manager with HA

   When you are using a single Key Manager, you need to add WSClient for the <KeyValidatorClientType> element to use the Web Service Client, and change <EnableThriftServer> to false to optimize performance.

   <APIKeyValidator>    
   	
       <KeyValidatorClientType>WSClient</KeyValidatorClientType>   
       <EnableThriftServer>false</EnableThriftServer>
       <ThriftServerHost>localhost</ThriftServerHost>
       <!--ThriftServerPort>10397</ThriftServerPort-->
       ...
   </APIKeyValidator>

   When you are using multiple Key Managers fronted by a load balancer, you need to add WSClient for the <KeyValidatorClientType> element to use the Web Service Client, and change <EnableThriftServer> to false to optimize performance.

   <APIKeyValidator>
   	...
   	<KeyValidatorClientType>WSClient</KeyValidatorClientType>    
    
       <EnableThriftServer>false</EnableThriftServer>
       <ThriftServerHost>localhost</ThriftServerHost>
       <!--ThriftServerPort>10397</ThriftServerPort-->
       ...
   </APIKeyValidator>

   If you wish to encrypt the Auth Keys (access tokens, client secrets, and authorization codes), see Encrypting OAuth Keys.

3. Disable the Policy Deployer under the Throttling configurations.

   <ThrottlingConfigurations>
   		...
           <PolicyDeployer>
               <Enabled>false</Enabled>
   			...
           </PolicyDeployer>
   		...
       </ThrottlingConfigurations>

4. Optionally, configure High Availability (HA) for the Key Manager.

   These steps are ONLY applicable if you need to configure HA for the Key Manager.

   1. Make a copy of the active instance configured above and use this copy as the second Key Manager active instance.

   2. Configure a load balancer to front the two Key Manager nodes.
      For information on configuring the load balancer, see Configuring the Proxy Server and the Load Balancer.

5. Configure the Key Manager to communicate with the Traffic Manager.

   This is available only as a  WUM update and is effective from 30th September 2020 (2020-09-30). For more information on updating WSO2 API Manager, see Updating WSO2 API Manager.

   • Single Traffic Manager
   • HA of Traffic Manager

   Configure the Key Manager with a single Traffic Manager as follows:

   9611 and 9711 are the Traffic Manager receiver ports for the binary type.

   1. Update the Throttling configurations as follows:

      <ThrottlingConfigurations>
      		<TrafficManager>
      			<Type>Binary</Type>
      			<ReceiverUrlGroup>tcp://${carbon.local.ip}:${receiver.url.port}</ReceiverUrlGroup>
                  <AuthUrlGroup>ssl://${carbon.local.ip}:${auth.url.port}</AuthUrlGroup>
                  <Username>${admin.username}</Username>
                  <Password>${admin.password}</Password>
      		</TrafficManager>        
      </ThrottlingConfigurations>

   2. Configure JMSConnectionParameters to connect to the broker running within the Traffic Manager.

      Example

      <JMSConnectionParameters>
         <transport.jms.ConnectionFactoryJNDIName>TopicConnectionFactory</transport.jms.ConnectionFactoryJNDIName>
         <transport.jms.DestinationType>topic</transport.jms.DestinationType>
         <java.naming.factory.initial>org.wso2.andes.jndi.PropertiesFileInitialContextFactory</java.naming.factory.initial>
      	<connectionfactory.TopicConnectionFactory>amqp://${admin.username}:${admin.password}@clientid/carbon?brokerlist='tcp://[traffic-manager-hostname]:${jms.port}?retries='5'%26connectdelay='50''</connectionfactory.TopicConnectionFactory>   
      </JMSConnectionParameters>

   Configure the Key Manager with multiple Traffic Managers, which are fronted by a load balancer as follows:
   
   Follow the instructions below to configure the Key Manager to communicate with the Traffic Managers and to push token revocation events to both Traffic Manager instances.

   1. Configure the receiver URL group <ReceiverUrlGroup> and Authentication URL Group <AuthUrlGroup> values, which are under the <TrafficManager> element in the <API-M_HOME>/repository/conf/api-manager.xml file, in order to contain all the Traffic Manager receiver URLs.
      This is required when you have more than one Traffic Manager instance, and you are publishing to both as per the deployment pattern selected. As an example, if you are using two Traffic Manager instances and data should be published to both of them, the ReceiverUrlGroup and AuthUrlGroup should be configured as follows:

      Example

      <ThrottlingConfigurations>
         <EnableAdvanceThrottling>true</EnableAdvanceThrottling>
      	<TrafficManager>
      		<Type>Binary</Type>
            	<ReceiverUrlGroup>{tcp://[Traffic-Manager-1-host]:9611}, {tcp://[Traffic-Manager-2-host]:9611}</ReceiverUrlGroup>
            	<!--ReceiverUrlGroup>tcp://${carbon.local.ip}:9612</ReceiverUrlGroup-->
            	<AuthUrlGroup>{ssl://[Traffic-Manager-1-host]:9711}, {ssl://[Traffic-Manager-2-host]:9711}</AuthUrlGroup>
            	<!--AuthUrlGroup>ssl://${carbon.local.ip}:9712</AuthUrlGroup-->
      		<Username>${admin.username}</Username>
      		<Password>${admin.password}</Password>
      	</TrafficManager>
      ...
      </ThrottlingConfigurations>

      [Traffic-Manager-1-host] and [Traffic-Manager-2-host] are the IPs/hostnames of two Traffic Manager nodes.

      Based on the above configuration, the Key Manager publishes events to both the Traffic Managers.

   2. Configure JMSConnectionParameters to connect to multiple brokers running within each Traffic Manager using fail over mechanism.

      Example

      <JMSConnectionParameters>
         <transport.jms.ConnectionFactoryJNDIName>TopicConnectionFactory</transport.jms.ConnectionFactoryJNDIName>
         <transport.jms.DestinationType>topic</transport.jms.DestinationType>
         <java.naming.factory.initial>org.wso2.andes.jndi.PropertiesFileInitialContextFactory</java.naming.factory.initial>
         <connectionfactory.TopicConnectionFactory>amqp://${admin.username}:${admin.password}@clientid/carbon?failover='roundrobin'%26cyclecount='2'%26brokerlist='tcp://[Traffic-Manager-1-host]:${jms.port}?retries='5'%26connectdelay='50';tcp://[Traffic-Manager-2-host]:${jms.port}?retries='5'%26connectdelay='50''</connectionfactory.TopicConnectionFactory>
      </JMSConnectionParameters>

6. Add the following in the <API-M_HOME>/repository/conf/api-manager.xml file for each of the Key Manager nodes in the deployment.

   <TokenRevocationNotifier>
          <Enabled>true</Enabled>
   </TokenRevocationNotifier>

7. Start the WSO2 API-M Key Manager node(s).
   Make sure to run the product optimizer either before starting the server or while starting the server, so that the resource utilization can be optimized on each of the nodes. For more information on product profile optimization, see Product Profiles.

org.apache.solr.common.SolrException: 
SolrCore 'registry-indexing' is not available due to init failure: Index locked for write for core registry-indexing

This section involves setting up the API Publisher node and enabling it to work with the other components in the distributed deployment.

1. Open the <API-M_HOME>/repository/conf/api-manager.xml file in the API Publisher node and make the following changes.

   1. Configure the Publisher with the Key Manager.
      You need to update the following configuration ONLY when you do not wish to share the user stores with the WSO2 API-M instance.

      This step is not applicable if you are enabling Single Sign-on (SSO).

      • Single Key Manager
      • Key Manager with HA

      Configure the Publisher with a single Key Manager as follows:

      <AuthManager>    
      	<ServerURL>https://[Key-Manager-host]:9443/services/</ServerURL>
      	<Username>admin</Username>
      	<Password>admin</Password>
      </AuthManager>

      Configure the Publisher with multiple Key Managers that are fronted by a load balancer as follows:

      <AuthManager>
      	<ServerURL>https://[Key-Manager-LB-host]:9443/services/</ServerURL>
      	<Username>admin</Username>
      	<Password>admin</Password>
      </AuthManager>

   2. Configure the Publisher with the Traffic Manager.
      This configuration enables the publishing of throttling policies, custom templates, and block conditions to the Gateway node.

      • Single Traffic Manager
      • Traffic Manager with HA

      Configure the Publisher with a single Traffic Manager as follows:

      <ThrottlingConfigurations> 
      	<EnableAdvanceThrottling>true</EnableAdvanceThrottling>
      	<TrafficManager>
      			<Type>Binary</Type>
      			<ReceiverUrlGroup>tcp://[Traffic-Manager-host]:9611</ReceiverUrlGroup>
                  <AuthUrlGroup>ssl://[Traffic-Manager-host]:9711</AuthUrlGroup>
                  <Username>${admin.username}</Username>
                  <Password>${admin.password}</Password>
      	</TrafficManager>
          <DataPublisher>
      	       <Enabled>false</Enabled>
      		  ...
          </DataPublisher>
          <PolicyDeployer>
      		<Enabled>true</Enabled>
              <ServiceURL>https://[Traffic-Manager-Host]:9443/services/</ServiceURL>
              <Username>${admin.username}</Username>
              <Password>${admin.password}</Password>
          </PolicyDeployer>
          <BlockCondition>
              <Enabled>false</Enabled>
      		...
          </BlockCondition>
          <JMSConnectionDetails>
              <Enabled>false</Enabled>
         		 ...
          </JMSConnectionDetails>
      	 ...
      </ThrottlingConfigurations>

      Configure the Publisher with multiple Traffic Managers that are fronted by a load balancer as follows:

      <ThrottlingConfigurations>     
      <EnableAdvanceThrottling>true</EnableAdvanceThrottling>
      	<TrafficManager>
      			<Type>Binary</Type>
                	<ReceiverUrlGroup>{tcp://[Traffic-Manager-1-host]:9611}, {tcp://[Traffic-Manager-2-host]:9611}</ReceiverUrlGroup>
                	<AuthUrlGroup>{ssl://[Traffic-Manager-1-host]:9711}, {ssl://[Traffic-Manager-2-host]:9711}</AuthUrlGroup>
                  <Username>${admin.username}</Username>
                  <Password>${admin.password}</Password>
      	</TrafficManager>
          <DataPublisher>
                <Enabled>false</Enabled>
                ...
          </DataPublisher>
          <PolicyDeployer>
      		<Enabled>true</Enabled>
              <ServiceURL>https://[Traffic-Manager-LB-Host]:9443/services/</ServiceURL>
              <Username>${admin.username}</Username>
              <Password>${admin.password}</Password>
          </PolicyDeployer>
          <BlockCondition>
              <Enabled>false</Enabled>
              ...
          </BlockCondition>
          <JMSConnectionDetails>
              <Enabled>false</Enabled>
               ...
          </JMSConnectionDetails>
         
           ...
      </ThrottlingConfigurations>

   3. Configure the Publisher with the Gateway.
      You need to add these configurations, because when creating an API, it calls the Gateway endpoint to create the actual Synapse file. 

      
      

      • If you are using a single Gateway node, configure the Publisher with the Gateway as follows:

        <APIGateway>
           <Environments>
               <Environment type="hybrid" api-console="true">
                    <Name>Production and Sandbox</Name>
                    <Description>This is a hybrid gateway that handles both production and sandbox token traffic.</Description>           
        			<ServerURL>https://[API-Gateway-Host-or-IP]:9443/services/</ServerURL>
                    <Username>${admin.username}</Username>
                    <Password>${admin.password}</Password>          
        			<GatewayEndpoint>http://[API-Gateway-Host]:8280,https://[API-Gateway-Host]:8243</GatewayEndpoint>
               </Environment>
           </Environments>
        </APIGateway>

      • If you are using multiple Gateway nodes, configure the Publisher with the Gateway nodes as follows:

        • Gateway with Shared File System
        • Gateway with rsync

        Configure the Publisher when working with multiple Gateways that are fronted by a load balancer when using a shared file system (e.g., NFS), to synchronize the data between your Gateway nodes as follows:

        <APIGateway>
           <Environments>
               <Environment type="hybrid" api-console="true">
                    <Name>Production and Sandbox</Name>
                    <Description>This is a hybrid gateway that handles both production and sandbox token traffic.</Description>           
        			<ServerURL>https://[API-Gateway-LB-Host-or-IP]:9443/services/</ServerURL>
                    <Username>${admin.username}</Username>
                    <Password>${admin.password}</Password>          
        			<GatewayEndpoint>http://[API-Gateway-LB-Host]:8280,https://[API-Gateway-LB-Host]:8243</GatewayEndpoint>
               </Environment>
           </Environments>
        </APIGateway>

        Configure the Publisher with a multiple Gateways that are fronted by a load balancer when using Remote Synchronization (rsync), to synchronize the data between your Gateway nodes as follows:

        <APIGateway>
           <Environments>
               <Environment type="hybrid" api-console="true">
                    <Name>Production and Sandbox</Name>
                    <Description>This is a hybrid gateway that handles both production and sandbox token traffic.</Description>           
        			<ServerURL>https://[API-Gateway-Manager-Host-or-IP]:9443/services/</ServerURL>
                    <Username>${admin.username}</Username>
                    <Password>${admin.password}</Password>          
        			<GatewayEndpoint>http://[API-Gateway-Manager-Host]:8280,https://[API-Gateway-Manager-Host]:8243</GatewayEndpoint>
               </Environment>
           </Environments>
        </APIGateway>

      To change the admin password, see Changing the super admin password. Furthermore, make sure to adhere to the note given under step 2 in the latter mentioned section if your password has special characters.

   4. Configure the Store URL to appear in the Publisher UI.
      For this purpose you need to set the <DisplayURL> to true and provide the URL of the Store.

      • Single Store
      • Store with HA

      Configure the Publisher with a single API Store as follows:

      Example

      <APIStore>  
             <DisplayURL>true</DisplayURL>     
             <URL>https://[Store-hostname]:9443/store</URL>
      </APIStore>

      Configure the Publisher with multiple API Stores that are fronted by a load balancer as follows:

      Example

      <APIStore>  
             <DisplayURL>true</DisplayURL>     
             <URL>https://[Store-LB-hostname]:9443/store</URL>
      </APIStore>

2. Disable the Thrift Server to optimize performance.
   You need to configure this in the Publisher <API-M_HOME>/repository/conf/api-manager.xml file.

   <APIKeyValidator> 
   ...  
   	<EnableThriftServer>false</EnableThriftServer>
   </APIKeyValidator>

3. Optionally, configure High Availability (HA) for the Publisher.

   These instructions are ONLY applicable if you need to configure HA for the Publisher.

   1. Make a copy of the active Publisher instance configured above and use this copy as the second active Publisher instance.

   2. Configure a load balancer to front the two Publisher nodes.
      For information on configuring the load balancer, see Configuring the Proxy Server and the Load Balancer.

4. Start the WSO2 API-M Publisher node(s).
   Make sure to run the product optimizer either before starting the server or while starting the server, so that the resource utilization can be optimized on each of the nodes. For more information on product profile optimization, see Product Profiles.

This section involves setting up the API Store node and enabling it to work with the other components in the distributed deployment.

1. Open the  <API-M_HOME>/repository/conf/api-manager.xml  file in the API Store node and make the following changes.

   1. Configure the API Store with the Key Manager.

      • Single Key Manager
      • Key Manager with HA

      Configure the API Store with a single Key Manager as follows:

      1. Configure the API key validator.
         When you are connecting the API Store directly to the Key Manager, add WSClient for the <KeyValidatorClientType> element to use the Web Service Client in order to optimize performance.

         <APIKeyValidator>
         	<ServerURL>https://[Key-Manager-host]:9443/services/</ServerURL>
         	<Username>${admin.username}</Username>
         	<Password>${admin.password}</Password>
         	<KeyValidatorClientType>WSClient</KeyValidatorClientType>    
         	<ThriftClientConnectionTimeOut>10000</ThriftClientConnectionTimeOut>
             <!--ThriftClientPort>10397</ThriftClientPort-->
          
             <EnableThriftServer>false</EnableThriftServer>
             <ThriftServerHost>localhost</ThriftServerHost>
             <!--ThriftServerPort>10397</ThriftServerPort-->
             ...
         </APIKeyValidator>

      2. Configure the Authentication Manager, so that the API Store can connect to the Key Manager.
         You need to update the following configuration ONLY when you do not wish to share the user stores with the WSO2 API-M instance.

         This step is not applicable if you are enabling Single Sign-on (SSO).

         <AuthManager>
         	<ServerURL>https://[Key-Manager-host]:9443/services/</ServerURL>
         	<Username>${admin.username}</Username>
         	<Password>${admin.password}</Password>
         </AuthManager>

      Configure the Store with multiple Key Managers that are fronted by a load balancer as follows:

      1. Configure the API key validator.
         When you are using multiple Key Managers fronted by a load balancer, you need to add WSClient for the <KeyValidatorClientType> element to use the Web Service Client.

         <RevokeAPIURL>https://[Gateway-host]:8243/revoke</RevokeAPIURL>
          
         <APIKeyValidator>
         	<ServerURL>https://[Key-Manager-LB-host]:9443/services/</ServerURL>
         	<Username>${admin.username}</Username>
         	<Password>${admin.password}</Password>
         	<KeyValidatorClientType>WSClient</KeyValidatorClientType>    
         	<ThriftClientConnectionTimeOut>10000</ThriftClientConnectionTimeOut>
             <!--ThriftClientPort>10397</ThriftClientPort-->
          
             <EnableThriftServer>false</EnableThriftServer>
             <ThriftServerHost>localhost</ThriftServerHost>
             <!--ThriftServerPort>10397</ThriftServerPort-->
          
             ...
         </APIKeyValidator>

      2. Configure the Authentication Manager, so that the API Store can connect to the Key Manager.
         You need to update the following configuration ONLY when you do not wish to share the user stores with the WSO2 API-M instance.

         This step is not applicable if you are enabling Single Sign-on (SSO).

         <AuthManager>
         	<ServerURL>https://[Key-Manager-LB-host]:9443/services/</ServerURL>
         	<Username>${admin.username}</Username>
         	<Password>${admin.password}</Password>
         </AuthManager>

   2. Make the following throttling related changes that correspond to the Traffic Manager.

      <ThrottlingConfigurations>
              <EnableAdvanceThrottling>true</EnableAdvanceThrottling>
              <DataPublisher>
                  <Enabled>false</Enabled>
      	    ...
              </DataPublisher>
              <PolicyDeployer>
                  <Enabled>false</Enabled>
      			...
              </PolicyDeployer> 
      	    ...
              <BlockCondition>
                  <Enabled>false</Enabled>
      	    ...
              </BlockCondition>
              <JMSConnectionDetails>
                  <Enabled>false</Enabled>
      	     ...
              </JMSConnectionDetails>
           ...
      </ThrottlingConfigurations>

   3. Configure the Store with the Gateway. 

      • If you are using a single Gateway node, configure the Store with the Gateway as follows:

        <APIGateway>
        	<Environments>
        		<Environment type="hybrid">
        		...
        			<ServerURL>https://[API-Gateway-host-or-IP]:9443/services/</ServerURL>
        			<Username>${admin.username}</Username>
        			<Password>${admin.password}</Password> 
        			<GatewayEndpoint>http://[API-Gateway-host]:8280,https://[API-Gateway-host]:8243</GatewayEndpoint>
        		</Environment>
        	</Environments> 
        ...
        </APIGateway>

      • If you are using multiple Gateway nodes, configure the Store with the Gateway nodes as follows:

        • Gateway with Shared File System
        • Gateway with rsync

        Configure the Store when working with multiple Gateways that are fronted by a load balancer, and when using a shared file system (e.g., NFS), to synchronize the data between your Gateway nodes as follows:

        <APIGateway>
        	<Environments>
        		<Environment type="hybrid">
        		...
        			<ServerURL>https://[API-Gateway-LB-Host-or-IP]:9443/services/</ServerURL>
        			<Username>${admin.username}</Username>
        			<Password>${admin.password}</Password> 
        			<GatewayEndpoint>http://[API-Gateway-LB-Host]:8280,https://[API-Gateway-LB-Host]:8243</GatewayEndpoint>
        		</Environment>
        	</Environments> 
        ...
        </APIGateway>

        Configure the Store with multiple Gateways that are fronted by a load balancer, and when using Remote Synchronization (rsync), to synchronize the data between your Gateway nodes as follows:

        <APIGateway>
        	<Environments>
        		<Environment type="hybrid">
        		...
        			<ServerURL>https://[API-Gateway-Manager-host]:9443/services/</ServerURL>
        			<Username>${admin.username}</Username>
        			<Password>${admin.password}</Password> 
        			<GatewayEndpoint>http://[API-Gateway-Manager-host]:8280,https://[API-Gateway-Manager-host]:8243</GatewayEndpoint>
        		</Environment>
        	</Environments> 
        ...
        </APIGateway>

   4. Configure the Token Revoke endpoint to point to Gateway.

      • Single Gateway
      • Gateway with HA

      Configure the Token Revoke endpoint with a single Gateway as follows:

      <OAuthConfigurations>
          ...
          <RevokeAPIURL>https://[API-Gateway-host]:8243/revoke</RevokeAPIURL>
      </OAuthConfigurations>

      Configure the Token Revoke endpoint with multiple Gateways, which are fronted by a load balancer as follows:

      <OAuthConfigurations>
          ...
          <RevokeAPIURL>https://[API-Gateway-Worker-LB-host]:8243/revoke</RevokeAPIURL>
      </OAuthConfigurations>

2. Optionally, configure High Availability (HA) for the Store.

   This is ONLY applicable if you need to configure HA for the Store.

   Make a copy of the active instance configured above and use this copy as the second API Store active instance.

3. Start the API Store node(s).
   Make sure to run the product optimizer either before starting the server or while starting the server, so that the resource utilization can be optimized on each of the nodes. For more information on product profile optimization, see Product Profiles.

This section involves setting up the Traffic Manager node(s) and enabling it to work with the other components in a distributed deployment.

1. Delete the <API-M_HOME>/repository/conf/registry.xml file and rename the <API-M_HOME>/repository/conf/registry_TM.xml file as the registry.xml  file.
   To disable registry indexing when setting up the Traffic Manager, see Registry indexing configurations.

2. Delete the <API-M_HOME>/repository/conf/axis2/axis2.xml file and rename the <API-M_HOME>/repository/conf/axis2/axis2_TM.xml file as the axis2.xml file.

   Note that all the jaggery apps and webapps in the <API-M_HOME>/repository/deployment/server/ directory are removed when you optimize the profile. For more details, see Product Profiles.

3. Optionally, mount the <API-M_HOME>/repository/deployment/server directory of all the Traffic Manager nodes to the shared file system.

   This step is ONLY applicable if you are configuring the Traffic Manager with HA and shared file system as the content synchronization mechanism.

   You need to do this to share all the Throttling policies between traffic management nodes.

4. Disable the Thrift Server to optimize performance.
   You need to configure this in the Traffic Manager <API-M_HOME>/repository/conf/api-manager.xml file.

   <APIKeyValidator> 
   ...  
   	<EnableThriftServer>false</EnableThriftServer>
   </APIKeyValidator>

5. Make sure that the Policy Deployer is enabled under the Throttling configurations.

   <ThrottlingConfigurations>
   		...
           <PolicyDeployer>
               <Enabled>true</Enabled>
           </PolicyDeployer>
   		...
       </ThrottlingConfigurations>

6. Optionally, configure High Availability (HA) for the Traffic Manager.

   This is ONLY applicable if you need to configure HA for the Traffic Manager.

   Make a copy of the active instance configured above and use this copy as the second active Traffic Manager instance.

7. Start the WSO2 API-M Traffic Manager node(s).
   Make sure to run the product optimizer either before starting the server or while starting the server, so that the resource utilization can be optimized on each of the nodes. For more information on product profile optimization, see Product Profiles.

   Always start the Traffic Manager using the -Dprofile=traffic-manager profile to avoid FATAL errors such as the following.

   FATAL - ServiceBusInitializer Couldn't initialize the ESB...
   org.apache.synapse.SynapseException: The synapse.xml location ././
           ./repository/deployment/server/synapse-configs
       /default doesn't exist

<heartbeat>    
	<delay>60</delay>
    <timeoutFactor>2.0</timeoutFactor>
</heartbeat>

This section involves setting up the Gateway node and enabling it to work with the other components in the distributed deployment.

1. Open the <API-M_HOME>/repository/conf/api-manager.xml file in the Gateway node.

2. Modify the api-manager.xml file as follows. This configures the connection to the Key Manager component.

   • Single Key Manager
   • Key Managers with HA

   Configure the Gateway with a single Key Manager as follows:

   1. Configure the API Key Validator.

      <APIKeyValidator> 
        <ServerURL>https://[Key-Manager-host]:9443/services/</ServerURL>
        <Username>${admin.username}</Username>
        <Password>${admin.password}</Password>
        ...
      </APIKeyValidator>

      [Key-Manager-host] - If you have a single Key Manager node, this should be the host of the Key Manager (i.e., the host of the WSO2 Identity Server).

      To change the admin password, see Changing the super admin password. If your password has special characters, follow the guidelines mentioned as a note under step 2 in the latter mentioned section.

   2. Use WSClient as the  KeyValidatorClientType in the  <API-M_HOME>/repository/conf/api-manager.xml file.

      <KeyValidatorClientType>WSClient</KeyValidatorClientType>

   3. Disable the Thrift Server to optimize performance.
      You need to configure this in the Gateway <API-M_HOME>/repository/conf/api-manager.xml file

      <APIKeyValidator> 
      ...  
      	<EnableThriftServer>false</EnableThriftServer>
      </APIKeyValidator>

   Configure the Gateway with multiple Key Managers, which are fronted by a load balancer as follows:

   1. Configure the APIKeyValidator as follows:

      <APIKeyValidator> 
        <ServerURL>https://[Key-Manager-LB-host]:9443/services/</ServerURL>
        <Username>${admin.username}</Username>
        <Password>${admin.password}</Password>
        ...
      </APIKeyValidator>

      • [Key-Manager-LB-host] - If there are multiple Key Managers (i.e., Multiple WSO2 Identity Servers as the Key Manager) fronted by a load balancer, this should be the host of the Key Manager's load balancer. For example, in the configuration we have defined key-manager as the load balancer host in the Key Manager section.

      To change the admin password, see Changing the super admin password. If your password has special characters, follow the guidelines mentioned as a note under step 2 in the latter mentioned section.

   2. Use  WSClient  as  KeyValidatorClientType  in the  <API-M_HOME>/repository/conf/api-manager.xml file.
      Note that you can only use the Web Service Client when the Key Manager cluster is fronted by a load balancer.

      <KeyValidatorClientType>WSClient</KeyValidatorClientType>

   3. Ensure that Thrift is disabled in the Gateway.
      This is enabled by default in all instances of the product, so you need to disable the Thrift server by setting EnableThriftServer  to false in the <API-M_HOME>/repository/conf/api-manager.xml file of each node.

      <EnableThriftServer>false</EnableThriftServer>

3. If you need to enable JSON Web Token (JWT),  you have to enable it in all Gateway and Key Manager components.
   For more information on configuring JWT, see Generating JSON Web Token.

4. Configure the Gateway to communicate with the Traffic Manager.
   You need to do this to enable Throttling for the Traffic Manager node(s).

   These configurations vary based on whether you have a single Traffic Manager node or multiple Traffic Manager nodes.

   • Single Traffic Manager
   • HA of Traffic Manager

   Configure the Gateway with a single Traffic Manager as follows:

   9611 and 9711 are the Traffic Manager receiver ports for the binary type.

   1. Update the Throttling configurations as follows:

      <ThrottlingConfigurations>
              <EnableAdvanceThrottling>true</EnableAdvanceThrottling>
      		<TrafficManager>
      			<Type>Binary</Type>
      			<ReceiverUrlGroup>tcp://[Traffic-Manager-host]:9611</ReceiverUrlGroup>
                  <AuthUrlGroup>ssl://[Traffic-Manager-host]:9711</AuthUrlGroup>
                  <Username>${admin.username}</Username>
                  <Password>${admin.password}</Password>
      		</TrafficManager>
              <DataPublisher>
                  <Enabled>true</Enabled>
      		...
              </DataPublisher>
              <PolicyDeployer>
      			<Enabled>false</Enabled>
      			<ServiceURL>https://[Traffic-Manager-host]:9443/services/</ServiceURL>
      		...
              </PolicyDeployer>
      		...
              <JMSConnectionDetails>
                  <Enabled>true</Enabled>
                  <ServiceURL>tcp://[Traffic-Manager-host]:5672</ServiceURL>
      		...
              </JMSConnectionDetails>
      </ThrottlingConfigurations>

      In the Gateway profile of WSO2 API Manager, disable the <PolicyDeployer> configuration to prevent blocking the API Publisher.

   2. Configure JMSConnectionParameters to connect to the broker running within the Traffic Manager.

      Example

      <JMSConnectionParameters>
         <transport.jms.ConnectionFactoryJNDIName>TopicConnectionFactory</transport.jms.ConnectionFactoryJNDIName>
         <transport.jms.DestinationType>topic</transport.jms.DestinationType>
         <java.naming.factory.initial>org.wso2.andes.jndi.PropertiesFileInitialContextFactory</java.naming.factory.initial>
      	<connectionfactory.TopicConnectionFactory>amqp://${admin.username}:${admin.password}@clientid/carbon?brokerlist='tcp://[traffic-manager-hostname]:${jms.port}?retries='5'%26connectdelay='50''</connectionfactory.TopicConnectionFactory>   
      </JMSConnectionParameters>

   Configure the Gateway with multiple Traffic Managers, which are fronted by a load balancer as follows:
   
   The Gateway publishes all Throttling events to the two Traffic Manager instances, and it fetches the throttle decisions from the Traffic Manager instances. Follow the instructions below to configure the API Gateway worker to communicate with the Traffic Managers and to push throttle events to both Traffic Manager instances.

   1. Configure the receiver URL group <ReceiverUrlGroup> and Authentication URL Group <AuthUrlGroup> values, which are under the <TrafficManager> element in the <API-M_HOME>/repository/conf/api-manager.xml file, in order to contain all the Traffic Manager receiver URLs.
      This is required when you have more than one Traffic Manager instance, and you are publishing to both as per the deployment pattern selected. As an example, if you are using two Traffic Manager instances and data should be published to both of them, the ReceiverUrlGroup and AuthUrlGroup should be configured as follows:

      Example

      <ThrottlingConfigurations>
         <EnableAdvanceThrottling>true</EnableAdvanceThrottling>
      	<TrafficManager>
      		<Type>Binary</Type>
            	<ReceiverUrlGroup>{tcp://[Traffic-Manager-1-host]:9611}, {tcp://[Traffic-Manager-2-host]:9611}</ReceiverUrlGroup>
            	<!--ReceiverUrlGroup>tcp://${carbon.local.ip}:9612</ReceiverUrlGroup-->
            	<AuthUrlGroup>{ssl://[Traffic-Manager-1-host]:9711}, {ssl://[Traffic-Manager-2-host]:9711}</AuthUrlGroup>
            	<!--AuthUrlGroup>ssl://${carbon.local.ip}:9712</AuthUrlGroup-->
      		<Username>${admin.username}</Username>
      		<Password>${admin.password}</Password>
      	</TrafficManager>
      
      
         <DataPublisher>
            <Enabled>true</Enabled>
      		...
         </DataPublisher>
       	<PolicyDeployer>
      			<Enabled>false</Enabled>
      		...
              </PolicyDeployer>
      ...
      </ThrottlingConfigurations>

      [Traffic-Manager-1-host] and [Traffic-Manager-2-host] are the IPs/hostnames of two Traffic Manager nodes.

      Based on the above configuration, the API Gateway publishes events to both the Traffic Managers.

      In the Gateway profile of WSO2 API Manager, disable the <PolicyDeployer> configuration to prevent blocking the API Publisher.

   2. Configure JMSConnectionParameters to connect to multiple brokers running within each Traffic Manager using fail over mechanism.

      Example

      <JMSConnectionParameters>
         <transport.jms.ConnectionFactoryJNDIName>TopicConnectionFactory</transport.jms.ConnectionFactoryJNDIName>
         <transport.jms.DestinationType>topic</transport.jms.DestinationType>
         <java.naming.factory.initial>org.wso2.andes.jndi.PropertiesFileInitialContextFactory</java.naming.factory.initial>
         <connectionfactory.TopicConnectionFactory>amqp://${admin.username}:${admin.password}@clientid/carbon?failover='roundrobin'%26cyclecount='2'%26brokerlist='tcp://[Traffic-Manager-1-host]:${jms.port}?retries='5'%26connectdelay='50';tcp://[Traffic-Manager-2-host]:${jms.port}?retries='5'%26connectdelay='50''</connectionfactory.TopicConnectionFactory>
      </JMSConnectionParameters>

   By default, WSO2 API Manager is shipped with a keystore (wso2carbon.jks) and a trust store (client-truststore.jks). For more information on how to create a new key store and a trust store with a private key and a self-signed certificate, see Configuring Keystore and Truststore and also see the Administration guide for recommendations on setting up keystores in WSO2 products.

5. Start the WSO2 API-M Gateway node(s).
   Make sure to run the product optimizer either before starting the server or while starting the server, so that the resource utilization can be optimized on each of the nodes. For more information on product profile optimization, see Product Profiles.