All docs This doc
||
Skip to end of metadata
Go to start of metadata

Given below are the configurations that should be in place for your API Manager server in order to use the metrics feature. You need to first enable metrics for your server and then enable the required storage types (reporters) that are used for storing the metrics data. See the following topics for instructions:

Enabling metrics

To enable metrics for your product, set the Enabled parameter under the Metrics element to true in the <APIM_HOME>/repository/conf/metrics.xml file. Alternatively, you can enable metrics at the time of starting the API Manager server by using the following command:

-Dmetrics.enabled=true

Configuring the storage of metrics

WSO2 API Manager is configured by default to store the information from metrics in the following reporters: JMX, CSV and JDBC. These reporters are configured in the metrics.xml file (stored in the <APIM_HOME>/repository/conf directory). You can disable metrics for individual reporters by setting the Enabled parameter to false

If you set the  the Enabled parameter under the metrics element to false in the metrics.xml file, metrics is disabled for all the reporters and it is not possible to enable metrics for individual reporters. 

See the following topics for information on configuring each of the available storage types. 

JMX

The following parameters in the metrics.xml file can be used to configure a JMX storage for metrics data.

Element NameDescriptionTypeDefault ValueMandatory/Optional
EnabledThis parameter specifies whether metics monitoring is enabled for JMX or not.BooleantrueMandatory

CSV

The following parameters in the metrics.xml file can be used to configure a CSV storage for metrics data.

Element NameDescriptionTypeDefault ValueMandatory/Optional
EnabledThis parameter specifies whether metrics monitoring is enabled for CSV or not.BooleanfalseMandatory
LocationThe location where the CSV files are stored.String${carbon.home}/repository/logs/metrics/
PollingPeriodThe time interval between polling activities that are carried out to update the metrics dashboard based on latest information. For example, if the polling period is 60, polling would be carried out every 60 milliseconds.Integer60

JDBC

The following parameters in the metrics.xml file can be used to configure a JDBC storage for metrics data.

Element NameDescriptionTypeDefault ValueMandatory/OptionalNotes
Enabled

This parameter specifies whether metrics monitoring is enabled for JDBC or not.

If you need to use metrics in WSO2 API-M, change this value to true. For more information, see Creating the datasource connection for the Metrics database.

BooleanfalseMandatory
DataSourceNameThe name of the datasource used.Stringjdbc/WSO2MetricsDB

PollingPeriodThe time interval between polling activities that are carried out to update the metrics dashboard based on latest information. For example, if the polling period is 60 , polling would be carried out every 60 seconds.Integer60
This value is specified in seconds.
ScheduledCleanupThis element contains parameters relating to scheduled cleanup. The possible values are Enabled, ScheduledCleanupPeriod and DaysToKeep. Scheduled cleanup involves scheduling a task to clear metric data in the database after a specified time interval. This is done to avoid excessive memory usage.



ScheduledCleanup/EnabledThis parameter specifies whether scheduled cleanup is enabled or not.Booleantrue

ScheduledCleanup/ScheduledCleanupPeriodThe number of seconds that should elapse after a cleanup task before the next clean-up task is carried out.Integer86400

ScheduledCleanup/DaysToKeepThe number of days during which the scheduled clean-up task should be carried out.Integer7

If you have enabled JDBC, then you also need to specify a datasource configuration to be used to create the connection between WSO2 API Manager and the JDBC data storage system. The metrics-datasources.xml file is used for configuring this datasource for metrics. 

Parameters that can be configured for a datasource are as follows:

XML elementAttributeDescriptionData typeDefault valueMandatory/Optional
<datasources-configuration>xmlns

The root element. The namespace is specified as: xmlns:svns="http://org.wso2.securevault/configuration"



Mandatory
<providers>
The container element for the datasource providers.

Mandatory

<provider>


The datasource provider, which should implement org.wso2.carbon.ndatasource.common
.spi.DataSourceReader
. The datasources follow a pluggable model in providing datasource type implementations using this approach.
Fully qualified Java class
Optional
<datasources>
The container element for the datasources.

Mandatory
<datasource>
The root element of a datasource.

Mandatory
<name>
Name of the datasource.String
Mandatory
<description>
Description of the datasource.String
Optional
<jndiConfig>
The container element that allows you to expose this datasource as a JNDI datasource.

Optional
<name>
The JNDI resource name to which this datasource should be bound.String
Mandatory if specifying JNDI configuration
<environment>

The container element in which you specify the following JNDI properties:

  • java.naming.factory.initial:  Selects the registry service provider as the initial context.
  • java.naming.provider.url: Specifies the location of the registry when the registry is being used as the initial context.
Fully qualified Java class
Optional
<definition>typeThe container element for the data source definition. Set the type attribute to RDBMS, or to custom if you're creating a custom type. The "RDBMS" data source reader expects a "configuration" element with the sub-elements listed below.String
Mandatory
<configuration>
The container element for the RDBMS properties.

Mandatory if definition type is RDBMS
<url>
The connection URL to pass to the JDBC driver to establish the connection.URL
Mandatory
<username>
The connection user name to pass to the JDBC driver to establish the connection.String
Optional
<password>
The connection password to pass to the JDBC driver to establish the connection.String
Optional
<driverClassName>
The class name of the JDBC driver to use.Fully qualified Java class
Mandatory
<maxActive>
The maximum number of active connections that can be allocated from this pool at the same time.Integer100Optional
<maxWait>
Maximum number of milliseconds that the pool waits (when there are no available connections) for a connection to be returned before throwing an exception.Integer30000 (30 seconds)Optional
<testOnBorrow>
Specifies whether objects are validated before being borrowed from the pool. If the object fails to validate, it is dropped from the pool, and we will attempt to borrow another.
When set to true, the validationQuery parameter must be set to a non-null string.
BooleanfalseOptional
<validationQuery>
The SQL query used to validate connections from this pool before returning them to the caller. If specified, this query does not have to return any data, it just can't throw a SQLException. The default value is null. Example values are SELECT 1(mysql), select 1 from dual(oracle), SELECT 1(MS Sql Server).StringnullMandatory when testOnBorrow is set to true
<validationInterval>
To avoid excess validation, only run validation at most at this frequency (interval time in milliseconds). If a connection is due for validation, but has been validated previously within this interval, it will not be validated again. The default value is 30000 (30 seconds).Long30000 (30 seconds)Optional

Sample configuration

Shown below is a sample metrics.xml file with the default configurations specifying the types of storages enabled for metrics data. See the above topics for instructions.

 The default configurations in the metrics.xml file
-->

<!--
    This is the main configuration file for metrics
-->

<Metrics xmlns="http://wso2.org/projects/carbon/metrics.xml">

    <!--
        Enable Metrics
    -->
    <Enabled>false</Enabled>
    <!--
        Metrics reporting configurations
    -->

    <Reporting>
        <JMX>
            <Enabled>true</Enabled>
        </JMX>
        <CSV>
            <Enabled>false</Enabled>
            <Location>${carbon.home}/repository/logs/metrics/</Location>
            <!-- Polling Period in seconds -->
            <PollingPeriod>60</PollingPeriod>
        </CSV>
        <JDBC>
            <Enabled>true</Enabled>
            <!-- Source of Metrics, which will be used to
                identify each metric in database -->
            <!-- Commented to use the hostname
                <Source>Carbon</Source>
            -->
            <!--
                JNDI name of the data source to be used by the JDBC Reporter.
                This data source should be defined in a *-datasources.xml
                file in conf/datasources directory.
            -->
            <DataSourceName>jdbc/WSO2MetricsDB</DataSourceName>
            <!-- Polling Period in seconds -->
            <PollingPeriod>60</PollingPeriod>
            <ScheduledCleanup>
                <!--
                    Schedule regular deletion of metrics data older than a set number of days.
                    It is strongly recommended that you enable this job to ensure your metrics tables do not get extremely
                    large. Deleting data older than seven days should be sufficient.
                -->
                <Enabled>true</Enabled>
                <!-- This is the period for each cleanup operation in seconds -->
                <ScheduledCleanupPeriod>86400</ScheduledCleanupPeriod>
                <!-- The scheduled job will cleanup all data older than the specified days -->
                <DaysToKeep>7</DaysToKeep>
            </ScheduledCleanup>
        </JDBC>
    </Reporting>
</Metrics>

Changing the default metrics database

Follow the instructions below to change the deault metrics database.

Step 1 - Set up the database

You can set up the following database types for the API-M-specific databases:

Note that we recommend to use Fail Over configuration over Load Balanced configuration with the MySQL clusters.

Step 2 - Create the datasource connection for the Metrics database

A datasource is used to establish the connection to a database. By default, the datasource connection for the Metrics database is configured in the metrics-datasources.xml file. This datasource configuration points to the default  H2 databases, which is shipped with the product. After setting up new databases to replace the default H2 databases, you can either change the default configurations in the above-mentioned files or configure new datasources.

Follow the steps below to create the datasource connection for the Metrics database:

  1. Open the <API-M _HOME>/repository/conf/datasources/metrics -datasources.xml file and locate the <datasource> configuration element.

  2. Update the URL pointing to you database, the username and password required to access the database, and the driver details as shown below. 

     Optionally, you can update the other elements for your database connection.
    ElementDescription
    maxActive The maximum number of active connections that can be allocated at the same time from this pool. Enter any negative value to denote an unlimited number of active connections.
    maxWait The maximum number of milliseconds that the pool waits (when there are no available connections) for a connection to be returned before throwing an exception. You can enter zero or a negative value to wait indefinitely.
    minIdle The minimum number of active connections that can remain idle in the pool without extra ones being created. You can enter zero to create none.

    testOnBorrow

    The indication of whether objects are validated before being borrowed from the pool. If the object fails to validate, it is dropped from the pool, and another attempt is made to borrow another.
    validationQuery The SQL query that is used to validate connections from this pool before returning them to the caller.
    validationInterval The indication to avoid excess validation, and only run validation at the most, at this frequency (time in milliseconds). If a connection is due for validation but has been validated previously within this interval, it is not validated again.

    defaultAutoCommit

    This property is only applicable to the MB Store database of WSO2 APIM, where this property should be explicitly set to false. In all other database connections explained above, auto committing is enabled or disabled at the code level as required for that database, i.e., the default auto commit configuration specified for the RDBMS driver will be effective instead of this property element. Note that auto committing is typically enabled for an RDBMS by default.

    When auto committing is enabled, each SQL statement will be committed to the database as an individual transaction, as opposed to committing multiple statements as a single transaction.

    For more information on other parameters that can be defined in the <API-M_HOME>/conf/datasources/ master-datasources.xml file, see Tomcat JDBC Connection Pool.

<datasource>
      <name>WSO2_METRICS_DB</name>
      <description>The MySQL datasource used for WSO2 Carbon Metrics</description>
      <jndiConfig>
            <name>jdbc/WSO2MetricsDB</name>
      </jndiConfig>
      <definition type="RDBMS">
             <configuration>
                 <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                 <url>jdbc:mysql://localhost:3306/WSO2METRICS_DB</url>
                 <username>root</username>
                 <password>root</password>
                 <maxActive>50</maxActive>
                 <maxWait>60000</maxWait>
                 <minIdle>5</minIdle>
                 <testOnBorrow>true</testOnBorrow>
                 <validationQuery>SELECT 1</validationQuery>
                 <validationInterval>30000</validationInterval>
                 <defaultAutoCommit>true</defaultAutoCommit>
            </configuration>
      </definition>
</datasource>

Setting the connection URL for MS SQL

If you are using the MS SQL configuration as shown below, note that the SendStringParametersAsUnicode parameter should be set to ‘false’ in the database connection URL. This is necessary in order to overcome a limitation in the MS SQL client driver. Without this parameter, the database driver will erroneously convert VARCHAR data into NVARCHAR, and thereby lower the database performance.

<datasource>
      <name>WSO2_METRICS_DB</name>
      <description>The MSSQL datasource used for WSO2 Carbon Metrics</description>
      <jndiConfig>
            <name>jdbc/WSO2MetricsDB</name>
      </jndiConfig>
      <definition type="RDBMS">
            <configuration>
                <driverClassName>com.microsoft.sqlserver.jdbc.SQLServerDriver</driverClassName>
                <url>jdbc:sqlserver://localhost:1433;databaseName=WSO2METRICS_DB;SendStringParametersAsUnicode=false</url>
                <username>sa</username>
                <password>sa</password>
                <maxActive>200</maxActive>
                <maxWait>60000</maxWait>
                <minIdle>5</minIdle>
                <testOnBorrow>true</testOnBorrow>
                <validationQuery>SELECT 1</validationQuery>
                <validationInterval>30000</validationInterval>
                <defaultAutoCommit>true</defaultAutoCommit>
            </configuration>
      </definition>
</datasource>       
<datasource>
      <name>WSO2_METRICS_DB</name>
      <description>The Oracle datasource used for WSO2 Carbon Metrics</description>
      <jndiConfig>
            <name>jdbc/WSO2MetricsDB</name>
      </jndiConfig>
      <definition type="RDBMS">
            <configuration>
                <driverClassName>oracle.jdbc.OracleDriver</driverClassName>
                <url>jdbc:oracle:thin:@localhost:1521/wso2_metrics</url>
                <username>scott</username>
                <password>tiger</password>
                <maxActive>100</maxActive>
                <maxWait>60000</maxWait>
                <minIdle>5</minIdle>
                <testOnBorrow>true</testOnBorrow>
                <validationQuery>SELECT 1 FROM DUAL</validationQuery>
                <validationInterval>30000</validationInterval>
                <defaultAutoCommit>true</defaultAutoCommit>
                <databaseProps>
                    <property name="SetFloatAndDoubleUseBinary">true</property>
                </databaseProps>
            </configuration>
      </definition>
</datasource>
<datasource>
      <name>WSO2_METRICS_DB</name>
      <description>The MSSQL datasource used for WSO2 Carbon Metrics</description>
      <jndiConfig>
            <name>jdbc/WSO2MBStoreDB</name>
      </jndiConfig>
      <definition type="RDBMS">
            <configuration>
                <url>jdbc:postgresql://localhost:5432/wso2_metrics</url>
                <username></username>
                <password></password>
                <driverClassName>org.postgresql.Driver</driverClassName>
                <maxActive>80</maxActive>
                <maxWait>60000</maxWait>
                <minIdle>5</minIdle>
                <testOnBorrow>true</testOnBorrow>
				<validationQuery>SELECT 1; COMMIT</validationQuery>
                <validationInterval>30000</validationInterval>
                <defaultAutoCommit>false</defaultAutoCommit>
            </configuration>
       </definition>
</datasource>

Step 3 - Create database tables in the Metrics database 

To create the database tables, connect to the databases that you created earlier and run the scripts provided in the product pack. The DB scripts corresponding to the database type are provided in the <API-M_HOME>/dbscripts/metrics directory.

To create the necessary database tables:

  1. Connect to the database and run the relevant script. 
    For example, run the following command to create the MB tables in a MySQL database.

    mysql -u root -p -DWSO2_METRICS_DB < '<API-M_HOME>/dbscripts/metrics/mysql.sql';

    <API-M_HOME>/dbscripts/metrics/mysql.sql is the script that should be used for MySQL 5.6 and prior versions. If you database is MySQL 5.7 or later version, use <API-M_HOME>/dbscripts/metrics/mysql5.7.sql script file.

  2. Restart the WSO2 API-M server. 
  • No labels