The latest version for DAS is WSO2 Data Analytics Server 3.1.0. View documentation for the latest release.
WSO2 Data Analytics Server is succeeded by WSO2 Stream Processor. To view the latest documentation for WSO2 SP, see WSO2 Stream Processor Documentation.

All docs This doc
Skip to end of metadata
Go to start of metadata

The main purpose of the Compressed Event Analytics Relational Provider is to split the aggregated events published to DAS into single events and load them to Apache Spark. In order to use the Compressed Event Analytics Relational Provider, the Data Access Layer (DAL) table should have the following two columns.

  • A meta field named meta_compressed indicating whether the data in the other column is compressed or not.
  • The flowData column that contains the aggregated events. Data in this column may be compressed (i.e. zipped, and followed by the Base64 encoding).

The aggregated data in the  flowData column should be structured as shown below.

	'events': [{
		'componentType': 'Proxy Service',
		'componentId': 'TestProxy',
		'children': [1],
		'entryPoint': 'TestProxy'
		. . .
	}, {
		'componentType': 'Sequence',
		'componentId': 'PROXY_INSEQ',
		'children': [2],
		'entryPoint': 'TestProxy'
		. . .
	'messageFlowId': 123456,
	'payloads': [{
		'payload': '?',
		'events': [{
			'eventIndex': 16,
			'attribute': 'beforePayload'
		. . .
	}, {
		'payload': '?',
		'events': [{
			'eventIndex': 15,
			'attribute': 'beforePayload'
			. . .

Use the following query syntax to create a table in the Spark environment (i.e. if it does not already exist) using data from Compressed Events Analytics.

CREATE TEMPORARY TABLE <temp_table_name> 
USING CompressedEventAnalytics 
OPTIONS (tableName <table_name>,  schema "componentType STRING, componentId STRING, children STRING, entryPoint STRING -sp, messageFlowId FACET 
Compressed Event Analytics relation provider options

The options that can be used with the Compressed Event Analytics Relation Provider are described in the table below.

Specify the options in key value pairs separated by commas, and enter the values within quotation marks.

tableName/streamNameThe name of the table in the DAL.

tableName "plug_usage"


streamName "plug.usage"


The schema of the temporary table. It is required to specify this.

Schema names are column name and column type value pairs with indexing options. These fields should be comma separated.

Schema indexing options are as follows.

  • -i denotes an indexed column. All indexed columns should be of the numeric type.
  • -sp denotes an indexed column with a score parameter.

The following fields are special fields in an analytics table:-

  • _timestamp denotes the timestamp of the record when it is persisted
  • _tenantId denotes the tenant id of the record
schema "componentType STRING, componentId STRING, children STRING, entryPoint STRING -sp, messageFlowId FACET -i"
mergeSchemaThis is a boolean flag used for schema merging. If this option is set to true, the given schema is merged with the corresponding table schema in the DAL (if a schema exists). The default value is "false".mergeSchema "false"
  • No labels