This documentation is for WSO2 Enterprise Integrator version 6.1.1 . View documentation for the latest release in the 6.x.x family and the latest release in the 7.x.x family.

All docs This doc

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Using the per-table method

Tip

This is the recommended method to purge data because it allows you to manage the data retention in an optimum way by defining different retention periods for tables of different time scales.


 Follow the steps below to schedule data purging via the per-table method.

  1. Log in to the Management Console of the Analytics profile.
  2. In the Main tab, click Data Explorer to open the Data Explorer page.
  3. Select the required table in the Table Name field, and click Schedule Data Purging as shown below. 

    Note

    The Schedule Data Purging option is displayed only for users of whom the assigned role has the Delete permission under Record enabled.


    This opens the Schedule Data Purging dialog box.

  4. In the Schedule Data Purging dialog box, set the time and days within which you want to purge data as shown below, and click Save.

    Note

    Do not purge data that is less than 2 days old as it can result in a data loss.

    The fields of the above screen are described below.

    FieldDescription
    Enable Data PurgingWhether you want to enable data purging or not.
    Schedule Time (Either cron string or HH:MM)Enter the time on which you want to purge data via cron expression or by defining the time in the following format: HH:MM. For example, the following cron expression will configure the data purging job to run at  12:00 PM (noon) every day 0 0 12 * * ?  For more information on cron expressions, go to Oracle Documentation.
    Purge Record Older Than (Days)Define the value as to keep data of only the last 'n' no of days back in the selected table. For example, if you give 1 as the value, the system will purge all data stored before yesterday.
  5. Repeat these steps for all the required tables to schedule them to be cleared regularly.


    Note

    Change the value of the <data-retention-days> property as per your requirement. However, the following values are recommended to retain and purge data in an optimum way.

    • You do not need to enable data purging for table defined for months and years.

    • In the <data-retention-days> property of tables for days, you must specify 45 days in order to retain data analyzed based on days for an acceptable time period.

    • For other tables created for shorter time granularities (e.g., seconds, minutes, and hours), you can spoecify a retentionb period of two days.

Removing scheduled data purging operations

...

  1. Open the <EI_HOME>/wso2/analytics/conf/analytics/analytics-config.xml file.
  2. Change the content under the <analytics-data-purging> property as shown below:

    Note

    Change the value of the <data-retention-days> property as per your requirement. However, do not purge data that is less than 2 days old from tables for seconds, minutes and hours because it can result in a data loss. You can specify 45 daya as the value for the <data-retention-days> property when you define a table for days.

    Code Block
    languagexml
    <analytics-data-purging>
        <purging-enable>true</purging-enable>
        <cron-expression>0 0 12 * * ?</cron-expression>
        <purge-include-tables>
             <table>ORG_WSO2_ESB_ANALYTICS_STREAM_EVENT</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECOND</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECONDALL</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTE</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTEALL</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERHOUR</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMONTH</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERDAY</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERSECOND</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMINUTE</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERHOUR</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERDAY</table>
        	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMONTH</table>
          </purge-include-tables>
        <data-retention-days>2</data-retention-days>
      </analytics-data-purging>

    The properties of the above configuration file are shown below.

    PropertyDescription
    <purging-enable>
    Change the value to true if you want to enable data purging.
    <cron-expression>
    The cron expression to define how you want to schedule the data purging operation. For example, the following cron expression will configure the archive job to run at  12:00 PM (noon) every day 0 0 12 * * ?  For more information on cron expressions, go to Oracle Documentation.
    <purge-include-table-patterns>
    Specify the tables of which you want to purge data. By default, it is configured to perform data purging on all tables as follows:

    <table>.*</table> However, you can specify the required tables by defining a regular expression or a table name within the <table> property. Define one tag per each regular expression if you want to specify multiple tables.

    <data-retention-days>

    Define the value as to keep data of only the last 'n' no of days back in the selected table.   For example, the default value 365 will purge all data stored before 1 year.

    Info

    You can purge all records by setting a minus value (e.g. -1) for <data-retention-days>.

  3. Save your changes.