This documentation is for WSO2 IoT Server 3.1.0. View the documentation for the latest release.
Analyzing Data - IoT Server 3.1.0 - WSO2 Documentation
Skip to end of metadata
Go to start of metadata

Data can be processed as batch or real-time or combination of both.

Events can be processed in real-time using WSO2's Siddhi Complex Event Processor (CEP) engine. Before we begin let's understand  the basic components which are used it real time processing. A stream is defined as a sequence of events having the same data type. It can be processed using the Siddhi Query Language (SiddhiQL). Siddhi works as an upside down database by storing all the queries in memory, matching incoming events against the stored queries at runtime and producing events that matched as streams. In order to process events, the execution logic should be written as a script using SiddhiQL and deployed in the server. This execution plan will act as an isolated execution environment and there will be an instance of the Siddhi Execution Runtime created for each execution plan. An execution plan can import one or more streams from the server for processing and push zero or more output streams back to the server.  To import and export streams, the script needs to contain import and export annotations that binding the streams defined in the execution plan to the server.

On the other hand, in order to perform batch analytics data needs to be  retrieved from the data stores and various analysis operations needs to be performed according to the defined analytic queries.

The following sections describe how to work with these components to analyze your stored data:

  • Interactive Analytics

    Follow the WSO2 DAS documentation on how you can perform interactive analytics using the Apache Lucene Query Language.

  • Batch Analytics Using Spark SQL

    Apache Spark is a powerful open-source processing engine built around speed, ease of use, and sophisticated analytics. WSO2 DAS employs Apache Spark as its analytics engine. Follow the WSO2 DAS documentation on how you can perform batch analytics using Apache Spark SQL.  

  • Real-time Analytics Using Siddhi

    Follow the WSO2 DAS documentation on how you can perform real-time analytics using Siddhi in WSO2 DAS.

  • Predictive Analytics Using WSO2 ML

    Deploy datasets and generates predictive models via the Machine Learner Wizard. Follow WSO2 DAS and WSO2 Machine Learner (ML) documentation for more information on this section.

  • No labels