This documentation is for WSO2 Enterprise Integrator version 6.1.1 . View documentation for the latest release in the 6.x.x family and the latest release in the 7.x.x family.

All docs This doc

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

ESB Analytics stores data and performs various analysis operations on them according to defned defined analytic queries. Thereby, as the volume of the data stored grows over time, the analysis and summarization jobs will also consume more time. Hence, purging data allows you to increase the performance of ESB Analytics by reducing the time taken to execute the analytics scripts as well as optimising optimizing the disk usage.

Warning

Decide on a retention period based on the number of days you need to retain data for analysis and purge data that are older than that in the below tables.

  • ORG_WSO2_ESB_ANALYTICS_STREAM_EVENT

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECOND

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECONDALL

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTE

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTEALL

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERHOUR

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMONTH

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERDAY

  • ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERSECOND

  • ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMINUTE

  • ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERHOUR

  • ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERDAY

  • ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMONTH

There are two three ways to purge data in ESB Analytics:

Table of Contents
maxLevel3
minLevel3

Using the Carbon App

For EI versions 6.1.0-6.3.0, purging analytics data can be managed via the org.wso2.analytics.ei.realtime Carbon application. This application is deployed by default if you get the WUM update timestamp: 1565079455315.

For more information about WUM-updating WSO2 products, see Getting Continuous Updates.

Using the per-table method

...

  1. Open the <EI_HOME>/wso2/analytics/conf/analytics/analytics-config.xml file.
  2. Change the content under the <analytics-data-purging> property as shown below:

    Note

    Change the value of the <data-retention-days> property as per your requirement.

    Code Block
    languagexml
    <analytics-data-purging>
        <purging-enable>true</purging-enable>
        <cron-expression>0 0 12 * * ?</cron-expression>
        <purge-include-tables>
             <table>ORG_WSO2_ESB_ANALYTICS_STREAM_EVENT</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECOND</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECONDALL</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTE</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTEALL</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERHOUR</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMONTH</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERDAY</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERSECOND</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMINUTE</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERHOUR</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERDAY</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMONTH</table>
          </purge-include-tables>
        <data-retention-days>2</data-retention-days>
      </analytics-data-purging>

    The properties of the above configuration file are shown below.

    PropertyDescription
    <purging-enable>
    Change the value to true if you want to enable data purging.
    <cron-expression>
    The cron expression to define how you want to schedule the data purging operation. For example, the following cron expression will configure the archive job to run at  12:00 PM (noon) every day 0 0 12 * * ?  For more information on cron expressions, go to Oracle Documentation.
    <purge-include-table-patterns>
    Specify the tables of which you want to purge data. By default, it is configured to perform data purging on all tables as follows:

    <table>.*</table> However, you can specify the required tables by defining a regular expression or a table name within the <table> property. Define one tag per each regular expression if you want to specify multiple tables.

    <data-retention-days>

    Define the value as to keep data of only the last 'n' no of days back in the selected table.   For example, the default value 365 will purge all data stored before 1 year.

    Info

    You can purge all records by setting a minus value (e.g. -1) for <data-retention-days>.

  3. Save your changes.