Introduction
The first tutorial introduces you to WSO2 SP and explains some of its basic concepts.
For this tutorial, let's consider a scenario where Sam, the foreman of the Sweet Factory wants a name-wise total for all the categories of sweets produced until now. During a given run, each Sweet Bot in the factory produces a batch of sweets and generates an event that contains information about the name of the sweet it produced and the quantity produced.
Individual events generated by four different worker Sweet Bots are as follows:
Input Event | Sweet Bot No | Item | Quantity |
---|---|---|---|
1 | 1 | Toffee | 11 |
2 | 2 | Gateau | 2 |
3 | 3 | Gingerbread | 5 |
4 | 4 | Gateau | 8 |
The foreman expects to receive the following output from WSO2 SP if the above events are sent in the given order.
Output Event | Item | Quantity |
---|---|---|
1 | Toffee | 11 |
2 | Gateau | 2 |
3 | Gingerbread | 5 |
4 | Gateau | 10 |
This tutorial shows how this data is processed to generate the expected output if it is sent to WSO2 SP to be analyzed.
Before you begin:
- Install Oracle Java SE Development Kit (JDK) version 1.8* and set the
JAVA_HOME
environment variable. - Download the WSO2 SP.
Tutorial steps
The Siddhi application required for this scenario can be created in two methods:
- The Source View of WSO2 Sream Processor Studio where you can write the application in Siddhi Query Language.
- The Design View of WSO2 Stream Processor where you can design the Siddhi application in a graphical interface by dragging and dropping Siddhi components and linking them as required.
Click on the relevant tab to create the Siddhi application in the preferred interface.
Let's get started!
- Navigate to the
<SP_HOME>/bin
directory and issue the following command to start the WSO2 Stream Processor Studio.- For Windows:
editor.bat
- For Linux:
sh editor.sh
- For Windows:
Access the Stream Processor Studio via the
http://localhost:<EDITOR_PORT>/editor
URL. The Stream Processor Studio opens as shown below.The default URL is
http://localhost:9390/editor
.- Click New to start defining a new Siddhi application. A new file opens as shown below.
First, you need to specify the stream from which the data to be analyzed must be taken. For this tutorial, let's assume that all data to be analyzed is sent to a stream named
SweetProductionStream
. Let's add a query as follows to specify that information to be processed must be taken from theSweetProductionStream
stream.from SweetProductionStream
To specify what information needs to be extracted from the events generated by the Sweet Bots, let's add a query as follows.
select name, sum(amount) as totalProduction
Here, the value for the
name
attribute of the incoming event is extracted with the same name. The sum of the values for theamount
attribute of incoming events is extracted ashourlyTotal
.The amounts need to be grouped according to the sweet category name. To do this, add a
group by
clause as follows.group by name
The data taken from the input stream must be sent to an output stream before it is published. In this tutorial, let's name the output stream as
SweetTotalStream
. This can be specified as shown below.insert into SweetTotalStream;
The Siddhi application should now look as follows.
from SweetProductionStream select name, sum(amount) as totalProduction group by name insert into SweetTotalStream;
Before saving these queries as a Siddhi application, you must name your appliction. In this scenario, let's name it as
SweetTotalApp
as shown below.@App:name('SweetTotalApp')
Lets also specify a name for the queries you created as shown below to allow someone checking the Siddhi application to understand their purpose.
@info(name='SweetTotalQuery')
- The streams that you referred to in you queries need to be defined as folows.
Let's define the input stream as follows.
define stream SweetProductionStream (name string, amount long);
There are two attributes included in the schema to capture the name of the sweet category as a string value, and the amount produced as a long value.
Let's define the output stream as follows.
@sink(type='log', prefix='Sweet Totals:') define stream SweetTotalStream(name string, totalProduction long);
This output stream definition includes the
name
attribute to present the name of the sweet category, and thetotalProduction
attribute to present the total production of each sweet category. In addition, there is an@sink
annotation. This represents a logger sink that outputs all the data in theSweetTotalStream
output stream to which it is connected. The output is presented as logs that are printed in the foreman's console.Event sinks are covered in detail in Publishing Processed Events.
Now the Siddhi application is ready to be saved. It should look as follows.
@App:name('SweetTotalApp') define stream SweetProductionStream (name string, amount long); @sink(type='log', prefix='Sweet Totals:') define stream SweetTotalStream(name string, totalProduction long); @info(name='SweetTotalQuery') from SweetProductionStream select name, sum(amount) as totalProduction group by name insert into SweetTotalStream;
To save this Siddhi file, click File, and then click Save. The following dialog box appears. Click Save.
As a result, this file is saved in the
<SP_HOME>/wso2/editor/deployment/workspace
directory.
Let's get started!
- Navigate to the
<SP_HOME>/bin
directory and issue the following command to start the WSO2 Stream Processor Studio.- For Windows:
editor.bat
- For Linux:
sh editor.sh
- For Windows:
Access the Stream Processor Studio via the
http://localhost:<EDITOR_PORT>/editor
URL. The Stream Processor Studio opens as shown below.The default URL is
http://localhost:9390/editor
.- Click New to start defining a new Siddhi application. A new file opens as shown below.
- Click Design View to open the design view of the Stream Processor Studio.
- First, you need to specify the stream from which the data to be analyzed must be taken. For this tutorial, let's assume that all data to be analyzed is sent to a stream named
SweetProductionStream
. Let's follow the substeps below.
- Drag and drop the following icon from the left pane to the grid. As a result, the Stream Configuration form appears below the grid.
- Let's enter information as follows in the Stream Configuration form.
- In the name field, let's enter
SweetProductionStream
. The sweet bots publish the name of the sweet and the number produced in a given batch. To capture this information, let's add two attributes to the stream as follows.
Attribute Name Attribute Tpe name string
amount
long
- Click Submit to save the stream configuration.
- In the name field, let's enter
- Drag and drop the following icon from the left pane to the grid. As a result, the Stream Configuration form appears below the grid.
Once the information published by the sweet bots is analyzed, the result needs to be presented as the name of the sweet and the total produced. To achieve this, you need an output stream with a schema that captures this information. Let's define this stream as follows.
- Drag and drop the stream icon to the grid again.
In the Stream Configuration form, enter
SweetToatlStream
as the name of the stream. Then enter two attributes as follows.Attribute Name Attribute Tpe name
string
totalProduction
long
- Click Submit to save the stream configuration.
Now the streams to capture the input and the output are ready. Now the Siddhi application you are creating needs to contain the logic based on which the input is processed to generate the required output. In this scenario, the processing to be carried out involved calculating the total sweet production for each sweet with each new batch. This can be achieved via a simple query. To add this query, follow the substeps below.
- The query to be added needs to project the output. Therefore, drag and drop the icon for projection queries from the left panel to the grid.
- Before defining the processing logic, you need to specify that the query needs to consider the
SweetProductionStream
stream as the input stream, and theSweetTotalsStream
stream as the output stream. To show this connection, draw an arrow from theSweetProductionStream
stream to the projection query, and another arrow from the projection query to theSweetTotalsStream
stream as shown below.
- To open the Query Configuration form and define the processing logic, click the settings icon on the projection query object as shown below.
- The name of the sweet needs to be projected in the output with the
name
label (same as the input stream attribute). The total on the other hand needs to be derived by calculating the sum for the amount of sweets produced in all the batches, and output with thetotalProduction
label. To do this, let's enter information in the Query Configuration form as follows.
The output is projected with the attributes used in the output stream (i.e., the
SweetTotalStream
). Therefore, in the Select section, the attributes in theSweetTotalStream
stream are already listed under As. To map the attributes in the input stream (i.e.,SweetProductionStream
) with the same attributes, you need to enter expressions as explained below.
Output Attribute Required Expression name
The values for the name attribute in the SweetProductionStream
stream can be inserted into theSweetTotalStream
stream with the same attribute name. To indicate this, entername
in the Expression field.totalProduction
To derive the value for the totalProduction
attribute, the sum of the total number produced of each sweet must be calculated with each new batch produced by sweet bots> To do this, let's specify the expression assum(amount)
.- Each output event indicating the total sweet production at the given time needs to be directed to the
SweetTotalsStream
output stream in order to be published. Therefore, make sure that all events is selected in the For field under Output.
- Click Save to save the query configuration.
- The query to be added needs to project the output. Therefore, drag and drop the icon for projection queries from the left panel to the grid.
- Click the Edit menu option, and then click Auto Allign. The Siddhi application you created now looks as follows in the design view.
- Click Source View. The application is displayed as follows.
Before saving these queries as a Siddhi application, you must name your appliction. In this scenario, let's name it as
SweetTotalApp
as shown below.@App:name('SweetTotalApp')
Lets also specify a name for the queries you created as shown below to allow someone checking the Siddhi application to understand their purpose.
@info(name='SweetTotalQuery')
- Click the File menu optIon, and then click Save to save the application you created. The following dialog box appears. Click Save.
As a result, this file is saved in the
<SP_HOME>/wso2/editor/deployment/workspace
directory.
Generating the output
Let's see how the Siddhi application you created functions when it is deployed and run in WSO2 SP. For the purpose of learning, the following four events generated by the Sweet Bots are simulated in WSO2 SP via the Event Simulation tool.
To check whether your Siddhi application generates this output, follow the steps below:
- In the Stream Processor Studio, click the following icon for event simulation. This opens the left panel for event simulation.
- In the Single Simulation tab, do the following:
- In the Siddhi App Name field, select SweetTotalApp.
- In the Stream Name field, select SweetProductionStream.
- In the name field, enter
Toffee
. - In the amount field, enter
11
. - Click Start and Send. This will start the siddhi app in run mode and send the event. The following is logged in the console.
Repeat these substeps to send three more events. Change the value for the name and amount fields as follows.
name amount Gateau
2
Gingerbread
5
Gateau
8
The resulting output events should be logged as follows.