This documentation is for WSO2 Enterprise Integrator version 6.0.0 . View documentation for the latest release.
Skip to end of metadata
Go to start of metadata

ESB Analytics stores data and performs various analysis operations on them according to defned analytic queries. Thereby, as the volume of the data stored grows over time, the analysis and summarization jobs will also consume more time. Hence, purging data allows you to increase the performance of ESB Analytics by reducing the time taken to execute the analytics scripts as well as optimising the disk usage.

Decide on a retention period based on the number of days you need to retain data for analysis and purge data that are older than that in the below tables.

  • ORG_WSO2_ESB_ANALYTICS_STREAM_EVENT

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECOND

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECONDALL

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTE

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTEALL

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERHOUR

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMONTH

  • ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERDAY

  • ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERSECOND

  • ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMINUTE

  • ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERHOUR

  • ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERDAY

  • ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMONTH

There are two ways to purge data in ESB Analytics:

Using the per-table method

 Follow the steps below to schedule data purging via the per-table method.

  1. Log in to the Management Console of the Analytics profile.
  2. In the Main tab, click Data Explorer to open the Data Explorer page.
  3. Select the required table in the Table Name field, and click Schedule Data Purging as shown below. 

    The Schedule Data Purging option is displayed only for users of whom the assigned role has the Delete permission under Record enabled.


    This opens the Schedule Data Purging dialog box.

  4. In the Schedule Data Purging dialog box, set the time and days within which you want to purge data as shown below, and click Save.

    Do not purge data that is less than 2 days old as it can result in a data loss.

    The fields of the above screen are described below.

    FieldDescription
    Enable Data PurgingWhether you want to enable data purging or not.
    Schedule Time (Either cron string or HH:MM)Enter the time on which you want to purge data via cron expression or by defining the time in the following format: HH:MM. For example, the following cron expression will configure the data purging job to run at  12:00 PM (noon) every day 0 0 12 * * ?  For more information on cron expressions, go to Oracle Documentation.
    Purge Record Older Than (Days)Define the value as to keep data of only the last 'n' no of days back in the selected table. For example, if you give 1 as the value, the system will purge all data stored before yesterday.
  5. Repeat these steps for all the required tables to schedule them to be cleared regularly.

Removing scheduled data purging operations

Follow the steps below to remove a data purging operation that you already scheduled.

  1. Log in to the Management Console as an admin user, if you are not already logged in.
  2. Click Main, and then click Data Explorer.
  3. Select the required table in the Table Name field, and then click Schedule Data Purging to open the Schedule Data Purging dialog box.
  4. Clear the Enable Data Purging check box as shown below.
    disable data purging
  5. Click Save, and close the dialog box.

Using the global method

This action affects all tenants.

  1. Open the <EI_HOME>/wso2/analytics/conf/analytics/analytics-config.xml file.
  2. Change the content under the <analytics-data-purging> property as shown below:

    Change the value of the <data-retention-days> property as per your requirement. However, do not purge data that is less than 2 days old as it can result in a data loss.

    <analytics-data-purging>
        <purging-enable>true</purging-enable>
        <cron-expression>0 0 12 * * ?</cron-expression>
        <purge-include-table-patterns>
          <table>
             <table>ORG_WSO2_ESB_ANALYTICS_STREAM_EVENT</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECOND</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECONDALL</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTE</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTEALL</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERHOUR</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMONTH</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERDAY</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERSECOND</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMINUTE</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERHOUR</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERDAY</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMONTH</table>
          </table>
          </purge-include-table-patterns>
        <data-retention-days>2</data-retention-days>
      </analytics-data-purging>

    The properties of the above configuration file are shown below.

    PropertyDescription
    <purging-enable>
    Change the value to true if you want to enable data purging.
    <cron-expression>
    The cron expression to define how you want to schedule the data purging operation. For example, the following cron expression will configure the archive job to run at  12:00 PM (noon) every day 0 0 12 * * ?  For more information on cron expressions, go to Oracle Documentation.
    <purge-include-table-patterns>
    Specify the tables of which you want to purge data. By default, it is configured to perform data purging on all tables as follows:

    <table>.*</table> However, you can specify the required tables by defining a regular expression or a table name within the <table> property. Define one tag per each regular expression if you want to specify multiple tables.

    <data-retention-days>

    Define the value as to keep data of only the last 'n' no of days back in the selected table.   For example, the default value 365 will purge all data stored before 1 year.

    You can purge all records by setting a minus value (e.g. -1) for <data-retention-days>.

  3. Save your changes.
  • No labels