This documentation is for WSO2 API Manager 2.0.0 View documentation for the latest release.
Tuning Performance - API Manager 2.0.0 - WSO2 Documentation
||
Skip to end of metadata
Go to start of metadata

This section describes some recommended performance tuning configurations to optimize the API Manager. It assumes that you have set up the API Manager on Unix/Linux, which is recommended for a production deployment. We also recommend a distributed API Manager setup for most production systems. Out of all components of an API Manager distributed setup, the API Gateway is the most critical, because it handles all inbound calls to APIs. Therefore, we recommend you to have at least a 2-node cluster of API Gateways in a distributed setup.

Important:

  • Performance tuning requires you to modify important system files, which affect all programs running on the server. We recommend you to familiarize yourself with these files using Unix/Linux documentation before editing them.
  • The values we discuss here are general recommendations. They might not be the optimal values for the specific hardware configurations in your environment. We recommend you to carry out load tests on your environment to tune the API Manager accordingly.

OS-level settings

When it comes to performance, the OS that the server runs plays an important role.

If you are running MacOS Sierra and experience long startup times for WSO2 products, try mapping your Mac hostname to 127.0.0.1 and ::1 in the /etc/hosts file as described in this blog post.

Following are the configurations you can apply to optimize OS-level performance:

  1. To optimize network and OS performance, configure the following settings in the /etc/sysctl.conf file of Linux. These settings specify a larger port range, a more effective TCP connection timeout value, and a number of other important parameters at the OS-level.

    It is not recommended to use net.ipv4.tcp_tw_recycle = 1 when working with network address translation (NAT), such as if you are deploying products in EC2 or any other environment configured with NAT.

    net.ipv4.tcp_fin_timeout = 30
    fs.file-max = 2097152
    net.ipv4.tcp_tw_recycle = 1
    net.ipv4.tcp_tw_reuse = 1
    net.core.rmem_default = 524288
    net.core.wmem_default = 524288
    net.core.rmem_max = 67108864
    net.core.wmem_max = 67108864
    net.ipv4.tcp_rmem = 4096 87380 16777216
    net.ipv4.tcp_wmem = 4096 65536 16777216
    net.ipv4.ip_local_port_range = 1024 65535      
  2. To alter the number of allowed open files for system users, configure the following settings in the /etc/security/limits.conf file of Linux (be sure to include the leading * character).

    * soft nofile 4096
    * hard nofile 65535

    Optimal values for these parameters depend on the environment.

  3. To alter the maximum number of processes your user is allowed to run at a given time, configure the following settings in the /etc/security/limits.conf file of Linux (be sure to include the leading * character). Each carbon server instance you run would require upto 1024 threads (with default thread pool configuration). Therefore, you need to increase the nproc value by 1024 per each carbon server (both hard and soft).

    * soft nproc 20000
    * hard nproc 20000

JVM-level settings

When an XML element has a large number of sub elements and the system tries to process all the sub elements, the system can become unstable due to a memory overhead. This is a security risk.

To avoid this issue, you can define a maximum level of entity substitutions that the XML parser allows in the system. You do this using the entity expansion limit as follows in the <API-M_HOME>/bin/wso2server.bat file (for Windows) or the <API-M_HOME>/bin/wso2server.sh file (for Linux/Solaris). The default entity expansion limit is 64000.

-DentityExpansionLimit=10000

In a clustered environment, the entity expansion limit has no dependency on the number of worker nodes.

WSO2 Carbon platform-level settings

In multi tenant mode, the WSO2 Carbon runtime limits the thread execution time. That is, if a thread is stuck or takes a long time to process, Carbon detects such threads, interrupts and stops them. Note that Carbon prints the current stack trace before interrupting the thread. This mechanism is implemented as an Apache Tomcat valve. Therefore, it should be configured in the <PRODUCT_HOME>/repository/conf/tomcat/catalina-server.xml file as shown below.

<Valve className="org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve" threshold="600"/>
  • The className is the Java class used for the implementation. Set it to org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.
  • The threshold gives the minimum duration in seconds after which a thread is considered stuck. The default value is 600 seconds.

APIM-level settings

Timeout configurations for an API call

The following diagram shows the communication/network paths that occur when an API is called. The timeout configurations for each network call are explained below.
 

  • Get throttle policy
    The throttle policy is taken from the registry and is based on your registry configurations. The differences are listed below:
    • Local Registry DB in the API Gateway - no network call occurs.
    • Mount directly to the Registry DB from the <API-M_HOME>/repository/conf/registry.xml file - A DB connection timeout occurs. This can be configured in the <API-M_HOME>/repository/conf/datasources/master-datasources.xml file and depends on the JDBC driver. For example,

      jdbc:mysql://localhost:3306/database?connectTimeout=60000&socketTimeout=60000
      jdbc:jtds:sqlserver://server:port/database;loginTimeout=60;socketTimeout=60
  • Key validation
     Key validation occurs via a Servlet HTTP call and the connection timeout can be configured by changing the following configuration details in the <API-M_HOME>/repository/conf/axis2/axis2_client.xml file. All timeout values are in milliseconds.

    <transportSender name="https" class="org.apache.axis2.transport.http.CommonsHTTPTransportSender">
    <parameter name="SO_TIMEOUT">60000</parameter>
    <parameter name="CONNECTION_TIMEOUT">60000</parameter>
    </transportSender>
  • Client call API Gateway + API Gateway call Backend
    For backend communication, the API Manager uses PassThrough transport. This is configured in the <API-M_HOME>/repository/conf/passthru-http.properties file. For more information, see Configuring passthru-http.properties in the ESB documentation.

    Note that the default value for http.socket.timeout differs between WSO2 products. In WSO2 API-M, the default value for http.socket.timeout is 60000ms.

General APIM-level recommendations

Some general APIM-level recommendations are listed below:

Improvement AreaPerformance Recommendations
API Gateway nodes

Increase memory allocated by modifying the /bin/wso2server. sh file with the following setting:

  • -Xms2048m -Xmx2048m -XX:MaxPermSize=1024m

Set the following in the <APIM_HOME>/repository/conf/axis2/axis2_client.xml file:

  • The following Axis2 client configurations are only applicable when WS key validation is enabled.
  • The default values mentioned in the API-M 2.0.0 pack are the values that we identified at the time of releasing API-M 2.0.0. However, if you want high concurrency, use the values mentioned below:
<parameter name="defaultMaxConnPerHost">1000</parameter> 
<parameter name="maxTotalConnections">30000</parameter> 
NHTTP transport of API Gateway

Recommended values for the <API-M_HOME>/repository/conf/nhttp.properties file are given below. Note that the commented out values in this file are the default values that will be applied if you do not change anything.

Property descriptions:

snd_t_coreTransport sender worker pool's initial thread count
snd_t_maxTransport sender worker pool's maximum thread count
snd_io_threadsSender-side IO workers, which is recommended to be equal to the number of CPU cores. I/O reactors usually employ a small number of dispatch threads (often as few as one) to dispatch I/O event notifications to a greater number (often as many as several thousands) of I/O sessions or connections. Generally, one dispatch thread is maintained per CPU core.
snd_alive_secSender-side keep-alive seconds
snd_qlenSender queue length, which is infinite by default

Recommended values:

# HTTP Sender thread pool parameters

  • snd_t_core=200
  • snd_t_max=250
  • snd_alive_sec=5
  • snd_qlen=-1
  • snd_io_threads=16

# HTTP Listener thread pool parameters

  • lst_t_core=200
  • lst_t_max=250
  • lst_alive_sec=5
  • lst_qlen=-1
  • lst_io_threads=16

#timeout parameters

  • http.socket.timeout.receiver: Recommended socket timeout for listener is 120000 ms.

  • http.socket.timeout.sender: Recommended socket timeout for sender is 120000 ms.

PassThrough transport of API Gateway

Recommended values for the <API-M_HOME>/repository/conf/passthru-http.properties file are given below. Note that the commented out values in this file are the default values that will be applied if you do not change anything.

Property descriptions

worker_thread_keepalive_sec

Defines the keep-alive time for extra threads in the worker pool
worker_pool_queue_lengthDefines the length of the queue that is used to hold runnable tasks to be executed by the worker pool
io_threads_per_reactorDefines the number of IO dispatcher threads used per reactor

http.max.connection.per.host.port

Defines the maximum number of connections per host port
worker_pool_queue_lengthDetermines the length of the queue used by the PassThrough transport thread pool to store pending jobs.

Recommended values

  • worker_thread_keepalive_sec: Default value is 60s. This should be less than the socket timeout.

  • worker_pool_queue_length: Set to -1 to use an unbounded queue. If a bound queue is used and the queue gets filled to its capacity, any further attempts to submit jobs will fail, causing some messages to be dropped by Synapse. The thread pool starts queuing jobs when all the existing threads are busy and the pool has reached the maximum number of threads. So, the recommended queue length is -1.

  • io_threads_per_reactor: Value is based on the number of processor cores in the system. (Runtime.getRuntime().availableProcessors())

  • http.max.connection.per.host.port : Default value is 32767, which works for most systems but you can tune it based on your operating system (for example, Linux supports 65K connections).

  • worker_pool_size_core: 400
  • worker_pool_size_max: 500
  • io_buffer_size: 16384
  • http.socket.timeout: 60000
  • snd_t_core: 200  
  • snd_t_max: 250  
  • snd_io_threads: 16  
  • lst_t_core: 200  
  • lst_t_max: 250  
  • lst_io_threads: 16
Make the number of threads equal to the number of processor cores.
Timeout configurations

The API Gateway routes the requests from your client to an appropriate endpoint. The most common reason for your client getting a timeout is when the Gateway's timeout is larger than the client's timeout values. You can resolve this by either increasing the timeout on the client's side or by decreasing it on the API Gateway's side.

Here are a few parameters, in addition to the timeout parameters discussed in the previous sections.

synapse.global_timeout_interval

Defines the maximum time that a callback waits in the Gateway for a response from the backend. If no response is received within this time, the Gateway drops the message and clears out the callback. This is a global level parameter that affects all the endpoints configured in the Gateway.

Global timeout is defined in the <API-M_HOME>/repository/conf/synapse.properties file. Recommended value is 120000 ms.

Endpoint-level timeout

You can define timeouts per endpoint for different backend services, along with the action to be taken in case of a timeout.

The example below sets the endpoint to 50 seconds (5000 ms) and executes the fault handler in case of a timeout. We can set this in the synapse configuration of the API located in the syanpse-configs directory.

<timeout>
   <duration>50000</duration>
   <responseAction>fault</responseAction>
</timeout>

Alternatively, you can set this through the Publisher UI as well by following the steps below.

  1. Log in to the API Publisher (https://<HostName>:9443/publisher). Select your API and click Edit API.
  2. Click the Implement  tab and click the cogwheel icon next to the endpoint you want to re-configure.
  3. In the Advanced Settings dialog box that appears, increase the duration by modifying the default property set as 3000 ms.

    Note that when the endpoint is suspended, the default action is defined here as invoking the fault sequence.

  4. Click Save and re-publish the API.

The http.socket.timeout parameter needs to be adjusted based on the endpoint-level timeout so that it's value is equal to or higher than the highest endpoint-level timeout.

Key Manager nodes

Set the MySQL maximum connections:

mysql> show variables like "max_connections"; 
 max_connections was 151 
 set to global max_connections = 250; 

Set the open files limit to 200000 by editing the /etc/sysctl.conf file:

sudo sysctl -p

Set the following in the <API-M_HOME>/repository/conf/tomcat/ catalina-server.xml file.

If you use WSO2 Identity Server (WSO2 IS) as the Key Manager, then the root location of the above path and the subsequent path needs to change from <API-M_HOME> to <IS_HOME> .

maxThreads="750" 
minSpareThreads="150" 
disableUploadTimeout="false" 
enableLookups="false" 
connectionUploadTimeout="120000" 
maxKeepAliveRequests="600" 
acceptCount="600" 

Set the following connection pool elements in the <API-M_HOME>/repository/conf/datasources/master-datasources.xml file. Time values are defined in milliseconds.

<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>

Note that you set the <testOnBorrow> element to true and provide a validation query (e.g., in Oracle, SELECT 1 FROM DUAL), which is run to refresh any stale connections in the connection pool. Set a suitable value for the <validationInterval> element, which defaults to 30000 milliseconds. It determines the time period after which the next iteration of the validation query will be run on a particular connection. It avoids excess validations and ensures better performance.

Registry indexing configurations

The registry indexing process is only required to be run on the API Publisher and API Store nodes. To disable the indexing process from running on the other nodes (Gateways and Key Managers), you need to set the <wso2registry><indexingConfiguration><startIndexing> element to false in the <API-M_HOME>/repository/conf/registry.xml file of the relevant nodes.

Analytics-related settings

This section describes the parameters you need to configure to tune the performance of the Analytics profile when it is affected by high load, network traffic etc. You need to tune these parameters based on the deployment environment.

Tuning data-agent parameters

The following parameters are configured in the <APIM-ANALYTICS_HOME>/repository/conf/data-brige/data-agent-config.xml file.

ParameterDescriptionDefault ValueTuning Recommendation
QueueSizeThe number of messages that can be stored in WSO2 API-M at a given time before they are sent to be published in the Analytics Dashboard.32768

This value should be increased when the Analytics profile is busy due to a request overload or if there is high network traffic. This prevents the generation of the queue full, dropping message error.

When the Analytics profile is not very busy and when the network traffic is relatively low, the queue size can be reduced to avoid an overconsumption of memory.

The number specified for this parameter should be a power of 2.

BatchSizeThe WSO2 API-M statistical data sent to the Analytics profile to be published in the Analytics Dashboard are grouped into batches. This parameter specifies the number of requests to be included in a batch.200This value should be tuned in proportion to the volume of requests sent from WSO2 API-M to the Analytics profile. This value should be reduced if you want to reduce the system overhead of the Analytics profile. This value should be increased if WSO2 API-M is generating a high amount of statistics and if the QueueSize cannot be further increased without causing an overconsumption of memory.
CorePoolSizeThe number of threads allocated to publish WSO2 API-M statistical data to the Analytics Dashboard via Thrift at the time WSO2 API-M is started. This value increases when the throughput of statistics generated increases. However, the number of threads will not exceed the number specified for the MaxPoolSize parameter.1The number of available CPU cores should be taken into account when specifying this value. Increasing the core pool size may improve the throughput of statistical data published in the Analytics Dashboard, but latency will also be increased due to context switching.
MaxPoolSizeThe maximum number of threads that should be allocated at any given time to publish WSO2 API-M statistical data in the Analytics Dashboard.1The number of available CPU cores should be taken into account when specifying this value. Increasing the maximum core pool size may improve the throughput of statistical data published in the Analytics Dashboard, since more threads can be spawned to handle an increased number of events. However, latency will also increase since a higher number of threads would cause context switching to take place more frequently.
MaxTransportPoolSizeThe maximum number of transport threads that should be allocated at any given time to publish WSO2 API-M statistical data to the Analytics Server.250This value must be increased when there is an increase in the throughput of events handled by WSO2 API-M Analytics.

The value of the tcpMaxWorkerThreads parameter in the <APIM-ANALYTICS_HOME>/repository/conf/data-bridge/data-bridge-config.xml must change based on the value specified for this parameter and the number of data publishers publishing statistics. e.g., When the value for this parameter is 250 and the number of data publishers is 7, the value for the tcpMaxWorkerThreads parameter must be 1750 (i.e., 7 * 250). This is because you need to ensure that there are enough receiver threads to handle the number of messages published by the data publishers.
SecureMaxTransportPoolSizeThe maximum number of secure transport threads that should be allocated at any given time to publish WSO2 API-M statistical data to the Analytics Server.250

This value must be increased when there is an increase in the throughput of events handled by WSO2 API-M Analytics.

The value of the sslMaxWorkerThreads parameter in the <APIM-ANALYTICS_HOME>/repository/conf/data-bridge/data-bridge-config.xml must change based on the value specified for this parameter and the number of data publishers publishing statistics. e.g., When the value for this parameter is 250 and the number of data publishers is 7, the value for the sslMaxWorkerThreads parameter must be 1750 (i.e., 7 * 250). This is because you need to ensure that there are enough receiver threads to handle the number of messages published by the data publishers.

  • No labels