Purging Analytics Data

Data purging is an option to remove historical data in ESB Analytics. This is important since you cannot delete tables or table data in ESB Analytics. By purging data, you can achieve high performance in data analysis without removing analyzed summary data. You can purge only stream data that are generated by the ESB profile that are contained in the following tables:

• ORG_WSO2_ESB_ANALYTICS_STREAM_EVENT

• ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECOND

• ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECONDALL

• ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTE

• ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTEALL

• ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERHOUR

• ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMONTH

• ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERDAY

• ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERSECOND

• ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMINUTE

• ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERHOUR

• ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERDAY

• ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMONTH

There are two ways to purge data in ESB Analytics:

Using the Management Console

To schedule data purging via the Management Console, follow the steps below:

1. Log in to the Management Console of the Analytics profile.
2. In the Main tab, click Data Explorer to open the Data Explorer page.
3. Select the required table in the Table Name field, and click Schedule Data Purging as shown below. 
   
   This opens the Schedule Data Purging dialog box.

4. In the Schedule Data Purging dialog box, set the time and days within which you want to purge data as shown below, and click Save.

   Do not purge data that is less than 2 days old as it can result in a data loss.

   

5. Repeat these steps for all the required tables to schedule them to be cleared regularly.

Using the global method

1. Open the <EI_HOME>/wso2/analytics/conf/analytics/analytics-config.xml file.

2. Change the content under the <analytics-data-purging> property as shown below:

   Change the value of the <data-retention-days> property as per your requirement. However, do not purge data that is less than 2 days old as it can result in a data loss.

   <analytics-data-purging>
       <purging-enable>true</purging-enable>
       <cron-expression>0 0 12 * * ?</cron-expression>
       <purge-include-table-patterns>
         <table>
            <table>ORG_WSO2_ESB_ANALYTICS_STREAM_EVENT</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECOND</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECONDALL</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTE</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTEALL</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERHOUR</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMONTH</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERDAY</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERSECOND</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMINUTE</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERHOUR</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERDAY</table>
       	 <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMONTH</table>
         </table>
         </purge-include-table-patterns>
       <data-retention-days>2</data-retention-days>
     </analytics-data-purging>

3. Save your changes.