This documentation is for WSO2 Complex Event Processor 4.0.0. View documentation for the latest release.
WSO2 Complex Event Processor is succeeded by WSO2 Stream Processor. To view the latest documentation for WSO2 SP, see WSO2 Stream Processor Documentation.
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 10 Next »

This section describes some recommended performance tuning configurations to optimize WSO2 CEP. It assumes that you have set up WSO2 CEP on a server running Unix/Linux, which is recommended for a production deployment.

Important

  • Performance tuning requires you to modify important system files, which affect all programs running on the server. We recommend you to familiarize yourself with these files using Unix/Linux documentation before editing them.
  • The parameter values we discuss below are just examples. They might not be the optimal values for the specific hardware configurations in your environment. We recommend that you carry out load tests on your environment to tune the product accordingly.

OS-Level Settings

  1. To optimize network and OS performance, configure the following settings in /etc/sysctl.conf file of Linux. These settings specify a larger port range, a more effective TCP connection timeout value, and a number of other important parameters at the OS-level.

    net.ipv4.tcp_fin_timeout = 30
    fs.file-max = 2097152
    net.ipv4.tcp_tw_recycle = 1
    net.ipv4.tcp_tw_reuse = 1
    net.core.rmem_default = 524288
    net.core.wmem_default = 524288
    net.core.rmem_max = 67108864
    net.core.wmem_max = 67108864
    net.ipv4.tcp_rmem = 4096 87380 16777216
    net.ipv4.tcp_wmem = 4096 65536 16777216
    net.ipv4.ip_local_port_range = 1024 65535      

    When we have the localhost port range configuration lower bound to 1024, there is a possibility that some processes may pick the ports which are already used by WSO2 servers. Therefore, it's good to increase the lower bound as sufficient for production, e.g., 10,000.

  2. To alter the number of allowed open files for system users, configure the following settings in /etc/security/limits.conf file of Linux.

    * soft nofile 4096
    * hard nofile 65535

    Optimal values for these parameters depend on the environment.

  3. To alter the maximum number of processes your user is allowed to run at a given time, configure the following settings in  /etc/security/limits.conf file of Linux (be sure to include the leading * character). Each carbon server instance you run would require upto 1024 threads (with default thread pool configuration). Therefore, you need to increase the nproc value by 1024 per each carbon server (both hard and soft).

    * soft nproc 20000
    * hard nproc 20000

JDBC Pool Configuration

Within the WSO2 platform, we use Tomcat JDBC pooling as the default pooling framework due to its production ready stability and high performance. The table below indicates some recommendations on how to configure the JDBC pool using the <PRODUCT_HOME>/repository/conf/datasources/master-datasources.xml file. For more details about recommended JDBC configurations, see The Tomcat JDBC Connection Pool.

PropertyDescriptionRecommendation
maxActive

The maximum number of active connections that can be allocated from the connection pool at the same time. The default value is 100.

This value should match the maximum number of requests that can be expected at a time in your production environment. This is to ensure that, whenever there is a sudden increase in the number of requests to the server, all of them can be connected successfully without causing any delays. Note that this value should not exceed the maximum number of requests allowed for your database.
minIdleThe minimum number of connections that can remain idle in the pool, without extra ones being created. The connection pool can shrink below this number if validation queries fail. Default value is 0.This value should be similar or near to the average number of requests that will be received by the server at the same time. With this setting, you can avoid having to open and close new connections every time a request is received by the server.
testOnBorrow

The indication of whether connection objects will be validated before they are borrowed from the pool. If the object validation fails, it will be dropped from the pool, and we will attempt to borrow another connection.

Setting this property to 'true' is recommended as it will avoid connection requests from failing. The validationQuery property should be used if testOnBorrow is set to true. To increase the efficiency of connection validation and to improve performance, validationInterval property should also be used.

validationInterval

To avoid excess validation, run validation at most at this frequency (time in milliseconds). If a connection is due for validation, but has been validated previously within this interval, it will not be validated again. The default value is 30000 (30 seconds).

This time out can be as high as the time it takes for your DBMS to declare a connection as stale. For example, MySQL will keep a connection open for as long as 8 hours, which requires the validation interval to be within that range. However, note that having a low value for validation interval will not incur a big performance penalty, specially when database requests have a high throughput. For example, a single extra validation query run every 30 seconds is usually negligible.

validationQueryThe SQL query used to validate connections from this pool before returning them to the caller. If specified, this query does not have to return any data, it just can't throw an SQLException. The default value is null. Example values are SELECT 1(mysql), select 1 from dual(oracle), SELECT 1(MS Sql Server).Specify an SQL query, which will validate the availability of a connection in the pool. This query is necessary when testOnBorrow property is true.

When it comes to web applications, users are free to experiment and package their own pooling framework such BoneCP.

 

CEP-Level settings

Performance tuning can be tried out in the following areas at the CEP level. The performance is considered in terms of throughput per second (TPS) and latency.

Receiving events

It is recommended to use WSO2Event as the receiver type to receive events with increased speed and as a result, achieve a greater throughput within a given duration of time. These configurations are common for both thrift and binary protocols.   

The following parameters which affect the performance relating to receiving events are configured in the <CEP_HOME>/repository/conf/data-bridge/data-bridge-config.xml file.

PropertyDescriptionRecommendation
workerThreads

Number of threads in consumer to handle the load. Default value is 10.

Value should be higher when receiving throughput is high. Should to consider number of CPU cores.
eventBufferCapacitySize of the receiving event buffer. Default value is 10000.Needs to be higher value when receiving throughput is high. When increasing the value heap memory size also needs to be increased accordingly.

 

Publishing events

The following parameters which affect the performance relating to publishing events are configured in the <CEP_HOME>/repository/conf/data-bridge/data-agent-config.xml file. These configurations are common for both thrift and binary protocols.

PropertyDescriptionRecommendation
QueueSizeThis is the size of the event queue disruptor. The default value is 32768. Value should be high when higher throughput needs to be handled. If you need low latency this value should be result at cost of low throughput.
BatchSize

The maximum size of a batch. Default value is 200.

Should consider about throughput of events and value should be assigned proportionally.

CorePoolSizeReserved number of threads. Default value is 1.Should consider number of CPU cores and value should be less than MaxPoolSize.
MaxPoolSizeMaximum number of threads allowed. If load cannot be handled, threads will spin off up to MaxPoolSize. Default Value is 1.Should consider number of CPU cores and value should be greater than CorePoolSize.

For better througput you can configure as follows.

<QueueSize>32768</QueueSize>
<BatchSize>200</BatchSize>
<CorePoolSize>1</CorePoolSize>
<MaxPoolSize>1</MaxPoolSize>

If you want to achieve better latency at cost of reduced throughput you can configure as follows. 

<QueueSize>256</QueueSize>
<BatchSize>200</BatchSize>
<CorePoolSize>1</CorePoolSize>
<MaxPoolSize>1</MaxPoolSize>
  • No labels