This documentation is for WSO2 Stream Processor 4.3.0. View documentation for the latest release.

All docs This doc
||
Skip to end of metadata
Go to start of metadata

 This section explains how to prevent the loss of data that can result from a system failure by persisting the state of WSO2 SP periodically either into a database system or into the file system.

Prerequisites

Before configuring database persistence, the following prerequisites must be completed.

  • One or more Siddhi Applications must be running in the WSO2 SP server.
  • A working RDBMS instance that can be used for data persistence must exist.
  • The requirements of the datasource must be already defined.
  • Database persistence involves updating the databases connected to WSO2 Stream Processor with the latest information relating to the events that are being processed by WSO2 SP at a given time.

Configuring database system persistence

The supported databases are H2, MySQL, Postgres, MSSQL and Oracle. The relevant jdbc driver jar should be downloaded and added to the <SP_HOME>/lib directory to prior to using database system persistence.

To configure periodic data persistence, update the <SP_HOME>/conf/worker/deployment.yaml file under state.persistence  as follows:

ParameterPurposeRequired Value
enabledThis enables data persistence.true
intervalInMin
The time interval in minutes that defines the interval in which state of Siddhi applications should be persisted1
revisionsToKeepThe number of revisions to keep in the system. When a new persist takes place, the old revisions are removed.3
persistenceStore
The persistence store.org.wso2.carbon.stream.processor.core.persistence.DBPersistenceStore
config > datasourceThe datasource to be used in persisting the state. The provided datasource should be properly defined in the deployment.yaml. For detailed instructions of how to configure a datasource, see Configuring Datasources.
WSO2_PERSISTENCE_DB (Datasource with this name should be defined in wso2.datasources)
config > tableThe table that should be created and used for the persisting of the state.
PERSISTENCE_TABLE

The following is a sample segment of the required configurations in the <SP_HOME>/conf/worker/deployment.yaml file to configure file system persistence.

Sample deployment.yaml segment
state.persistence:
  enabled: true
  intervalInMin: 1
  revisionsToKeep: 3
  persistenceStore: org.wso2.carbon.stream.processor.core.persistence.DBPersistenceStore
  config:
    datasource: <DATASOURCE NAME>   # A datasource with this name should be defined in wso2.datasources namespace
    table: <TABLE NAME>


Configuring file system persistence

This section explains how to persist the states of Siddhi applications during a required time interval in the file system in order to maintain back-ups. To configure state persistence, update the <SP_HOME>/conf/worker/deployment.yaml file under state.persistence  as follows:

ParameterPurposeRequired Value
enabledThis enables data persistence.true
intervalInMin The time interval in minutes that defines the interval in which state of Siddhi applications should be persisted
1
revisionsToKeepThe number of revisions to keep in the system. When a new persist takes place, the old revisions are removed.3
persistenceStore
The persistence store.
org.wso2.carbon.stream.processor.core.persistence.FileSystemPersistenceStore
config > locationA fully qualified folder location to where the revision files should be persisted.
siddhi-app-persistence


The following is a sample segment of the required configurations in the <SP_HOME>/conf/worker/deployment.yaml file to configure file system persistence.

Sample deployment.yaml segment
state.persistence:
  enabled: true
  intervalInMin: 1
  revisionsToKeep: 2
  persistenceStore: org.wso2.carbon.stream.processor.core.persistence.FileSystemPersistenceStore
  config:
    location: siddhi-app-persistence
  • No labels