This documentation is for WSO2 Stream Processor 4.3.0. View documentation for the latest release.

All docs This doc

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Within the WSO2 platform, we use Tomcat JDBC pooling as the default pooling framework due to its production ready stability and high performance. The table below indicates some recommendations on how to configure the JDBC pool using the  <PRODUCT_HOME>/repository/conf/datasources/master-datasources.xml file

PropertyDescriptionRecommendation
maxActive

The maximum number of active connections that can be allocated from the connection pool at the same time. The default value is  100.

This value should match the maximum number of requests that can be expected at a time in your production environment. This is to ensure that, whenever there is a sudden increase in the number of requests to the server, all of them can be connected successfully without causing any delays. Note that this value should not exceed the maximum number of requests allowed for your database.
testOnBorrow

The indication of whether connection objects will be validated before they are borrowed from the pool. If the object validation fails, it will be dropped from the pool, and we will attempt to borrow another connection.

Setting this property to 'true' is recommended as it will avoid connection requests from failing. The validationQuery property should be used if testOnBorrow is set to true. To increase the efficiency of connection validation and to improve performance, validationInterval property should also be used.

validationInterval

To avoid excess validation, run validation at most at this frequency (time in milliseconds). If a connection is due for validation, but has been validated previously within this interval, it will not be validated again. The default value is  30000  (30 seconds).

This time out can be as high as the time it takes for your DBMS to declare a connection as stale. For example, MySQL will keep a connection open for as long as 8 hours, which requires the validation interval to be within that range. However, note that having a low value for validation interval will not incur a big performance penalty, specially when database requests have a high throughput. For example, a single extra validation query run every 30 seconds is usually negligible.

validationQueryThe SQL query used to validate connections from this pool before returning them to the caller. If specified, this query does not have to return any data, it just can't throw an SQLException. The default value is null. Example values are SELECT 1(mysql), select 1 from dual(oracle), SELECT 1(MS Sql Server).Specify an SQL query, which will validate the availability of a connection in the pool. This query is necessary when testOnBorrow property is true.
Note

When it comes to web applications, users are free to experiment and package their own pooling framework such BoneCP.

...

The following parameters which affect the performance relating to the databridge communication are configured in the  <SP_HOME>/conf/editor/deployment.yaml under data-bridge-config property . These configurations are common for both thrift and binary protocols. 

PropertyDescriptionDefault ValueRecommendation
workerThreads The number of threads reserved to handle the load of events received.10This value should be increased if you want to increase the throughput by receiving a higher number of events at a given time. The number of available CPU cores should be considered when specifying this value. If the value specified exceeds the number of CPU cores, higher latency would occur as a result of context switching taking place more often.
maxEventBufferCapacityThe maximum size allowed for the event receiving buffer in
mega
bytes. The event receiving buffer temporarily stores the events received before they are forwarded to an event stream
10
10000000This value should be increased when there is an increase in the receiving throughput. When increasing the value heap memory size also needs to be increased accordingly.
eventBufferSizeThe number of messages that is allowed in the receiving queue at a given time.2000This value should be increased when there is an increase in the receiving throughput.
clientTimeoutMinSession timeout value in minutes.30Cache that contains all the agent sessions are expired after this value is reached. This value should be increased when there is an

Publishing events

The following parameters which affect the performance relating to the Data Agents - to publish events through databridge are configured in the <SP_HOME>/conf/editor/deployment.yaml under data.agent.config property. These configurations are common for both thrift and binary protocols. 

PropertyDescriptionDefault ValueRecommendation
queueSize The size of the queue event disruptor which handles events before they are published to an application/data store.32768

The value specified should always be the result of an exponent with 2 as the base. (e.g., 32768 is 215).

A higher value should be specified when a higher throughput needs to be handled. However, the increase in the load handled at a given time can reduce the speed at which the events are processed. Therefore, a lower value should be specified if you want to reduce the latency.

batchSize The maximum number of events in a batch sent to the queue event disruptor at a given time.200This value should be assigned proportionally to the throughput of events handled. Greater the batch size, higher will be the number of events sent to the queue event disruptor at a given time.
corePoolSize The number of threads that will be reserved to handle events at the time you start the CEP server. This value will increase as throughput   of events handled increases, but it will not exceed the value specified for the MaxPoolSize parameter.1The number of available CPU cores should be taken into account when specifying this value. Increasing the core pool size may improve the throughput, but latency  will also be increased due to context switching.
maxPoolSize The maximum number of threads that should be reserved at any given time to handle events.1The number of available CPU cores should be taken into account when specifying this value. Increasing the maximum core pool size may improve the throughput since more threads can be spawned to handle an increased number of events. However, latency  will also increase since a higher number of threads would cause context switching to take place more frequently.

For better througput you can configure the parameters as follows.

...