The latest version for DAS is WSO2 Data Analytics Server 3.1.0. View documentation for the latest release.
WSO2 Data Analytics Server is succeeded by WSO2 Stream Processor. To view the latest documentation for WSO2 SP, see WSO2 Stream Processor Documentation.

All docs This doc
||
Skip to end of metadata
Go to start of metadata

This section describes the additional configurations that can be done to data collected by DAS before they are processed.

Setting a custom timestamp value for incoming events

When an event is received by WSO2 DAS, the timestamp of that event is set as a parameter value. This value often needs to be overridden in many situations including the following examples.

In order to override the timestamp value assigned at the time an event is received, the client that publishes events to WSO2 DAS should set an attribute named _timestamp of the Long type as a payload attribute in each event.

Querying for the timestamp in a Spark script

You can query for events by the timestamp in an Analytics script. In order to do this, you need to add a _timestamp long option in the table schema.

e.g., If there is a table named TestTable1 in WSO2 DAS with the name STRING, b INT schema, the timestamps of the events in the table can be queried as follows.

CREATE TEMPORARY TABLE table1 USING CarbonAnalytics OPTIONS (tableName “TestTable1”, schema “a1 STRING, b1 INT, _timestamp LONG”) ;
SELECT a1, _timestamp FROM table1;

Additionally, you can use the _timestamp option to insert a custom value for the timestamp parameter of an event. 

e.g., If you need to insert a set of values from the table1 table to another table named table2, you can write a query as follows.

CREATE TEMPORARY TABLE table2 USING CarbonAnalytics OPTIONS (tableName “TestTable2”, schema “a2 STRING, _timestamp LONG”) ;

The following query inserts the results of the SELECT  a1, _timestamp FROM table1 query into the TestTable2 table.

INSERT INTO TABLE table2 SELECT  a1, _timestamp FROM table1;

You can override the behavior of the _timestamp field to use the current timestamp of the event, by setting -1 as shown below.

INSERT INTO TABLE table2 SELECT  a1, -1 as _timestamp FROM table1;
  • No labels