A Siddhi Application is a combination of multiple Siddhi executional elements. A Siddhi executional element can be a Siddhi Query or a Siddhi Partition. When defining a Siddhi application, you can specify a number of parallel instances to be created for each executional element, and how each executional element must be isolated for an SP instance. Based on this, the initial Siddhi application is divided into multiple Siddhi applications and deployed in different SP instances.
Any standalone Siddhi Application can be converted in to a distributed Siddhi Application by adding @dist
annotations. By adding these annotations, you ca divide the Siddhi Application to multiple execution groups and run them in parallel with multiple instances per group. The supported annotations are as follows:
Execution group annotation
Syntax | @dist(execGroup=’name of the group<string>’) |
---|---|
Description | This annotation specifies the execution groups. An execution group is a collection of queries that is executed as a single unit. You can add this annotation at the query level and specify a name for the group. Queries with same execution group name are considered as part of the same group. If you do not specify an execution group name, system generated name with the {Siddhi app name}-random UUID format is assigned. When grouping queries, the following scenarios result in validation exceptions and application deployment is halted as a result.
|
Applicable Level | Query |
Example | @info(name = ‘query-1') @dist(execGroup='group-1') from TempStream#window.time(2 min) select avg(temp) as avgTemp, roomNo, deviceID insert all events into AvgTempStream; |
Parallel annotation
Syntax | @dist(parallel=’no of parallel instances<int>’) |
---|---|
Description | This annotation specifies the execution parallelism of an execution group (i.e., the number of instances in which the executional elements of the execution group must be executed in parallel). Parallelism is always defined against an execution group. If no parallelism is specified,
|
Applicable Level | Query |
Example | @info(name = ‘query-3') @dist(parallel='2') from TempStream [(roomNo >= 100 and roomNo < 110) and temp > 40 ] select roomNo, temp insert into HighTempStream; |
Transport channel creation annotation
Syntax | @App:transportChannelCreationEnabled('true|false') |
---|---|
Description | This annotation specifies whether Stream Processor managers are allowed to create Kafka topics that are required for application deployment. By default, this is set to true and the topics are created when the application is deployed. If the annotation is set to false , you need to create the required intermediate topics before deploying the application. If the required topics do not exist when you deploy the application, the application deployment fails, and the deployment process is aborted. |
Applicable Level | Application |
Example | @App:name('wso2-app') @App:transportChannelCreationEnabled('false') |
Creating a distributed Siddhi application
This section explains how to write distributed Sidhi applications by assigning executional elements to different execution groups.
Annotations
The following annotations are used when writing a distributed Siddhi application.
Annotation | Description |
---|---|
@dist(execGroup='name of the group') | All the executional elements with the same execution group are executed in the same Siddhi application. When different execution groups are mentioned within the same distributed Siddhi application, WSO2 SP initiates a separate Siddhi Application per execution group. In each separated Siddhi application, only the executional elements assigned to the relevant execution group are executed. Executional elements that have no execution group assigned to them are executed in a separate SP instance. |
@dist (parallel='number of parallel instances’) | The number of instances in which the executional element must be executed in parallel. All the executional elements assigned to a specific execution group (i.e., via the When the number of parallel instances to be run is not given for the executional elements assigned to an execution group, only one Siddhi application is initiated for that execution group. |
Example
The following is a sample distributed Siddhi application.
@info(name = ‘query1') @dist(execGroup='group1') from TempStream#window.time(2 min) select avg(temp) as avgTemp, roomNo, deviceID insert all events into AvgTempStream; @info(name = ‘query2') @dist(execGroup='group1') from TempStream[temp > 30.0]#window.time(1 min) as T join RegulatorStream[isOn ==false]#window.length(1) as R on T.roomNo == R.roomNo select T.roomNo, R.deviceID, 'start' as action insert into RegulatorActionStream; @info(name = ‘query3') @dist(execGroup='group1') from every( e1=TempStream ) -> e2=TempStream[e1.roomNo==roomNo and (e1.temp + 5) <= temp ] within 10 min select e1.roomNo, e1.temp as initialTemp, e2.temp as finalTemp insert into AlertStream; @info(name = ‘query4') @dist(execGroup='group2' ,parallel ='2') from TempStream [(roomNo >= 100 and roomNo < 110) and temp > 40 ] select roomNo, temp insert into HighTempStream; @info(name = ‘query5') @dist(execGroup='group3' , parallel=’2’) partition with ( deviceID of TempStream ) begin from TempStream#window.time(1 min) select roomNo, deviceID, temp, avg(temp) as avgTemp insert into #AvgTempStream; from #AvgTempStream[avgTemp > 20]#window.length(10) select roomNo, deviceID, max(temp) as maxTemp insert into deviceTempStream; end;
When this Siddhi application is deployed, it is executed as shown in the table below.
Execution Group | Number of Siddhi Application Instances | Queries executed |
---|---|---|
| 1 |
|
| 2 |
|
| 2 | query5 |