This documentation is for WSO2 API Manager 2.0.0 View documentation for the latest release.
Due to a known issue do not use JDK1.8.0_151 with WSO2 products. Use JDK 1.8.0_144 until JDK 1.8.0_162-ea is released.
Configuring APIM Analytics - API Manager 2.0.0 - WSO2 Documentation
||
Skip to end of metadata
Go to start of metadata

This section explains how to configure analytics for WSO2 API Manager. The API Manager integrates with the WSO2 Analytics platform to provide reports, statistics and graphs on the APIs deployed in WSO2 API Manager. You can then configure alerts to monitor these APIs and detect unusual activity, manage locations via geo location statistics and carry out detailed analysis of the logs.

By default, WSO2 API Manager has a port offset of 0 (no port offset) and WSO2 API Manager Analytics has an offset of 1. Therefore, this guide assumes that you do not have any other carbon servers running on the same machine with port offsets of 0 or 1.

Click on the Quick Setup tab to set up analytics for quick demos and try-out scenarios, or click on the Standard Setup tab to set up analytics for a production environment.

Some documentation links are pointed to WSO2 DAS documentation intentionally, since the API Manager Analytics server is built on top of WSO2 DAS.

This mode of setup is only recommended for non-critical demos and quick try-out scenarios. Once you run the servers you will notice that there are database files being created in the directory where you installed the two servers. These are H2 databases that hold the summarized data of your API Analytics. In a more standard deployment, this data is recorded in database servers such as MySQL, Oracle, etc. Therefore, this mode of operation is not recommended for production grade or other critical deployments.

  1. Download the WSO2 API Manager and the WSO2 APIM Analytics distributions (zip files), and extract both files to the same directory (preferably an empty directory).
  2. Take the following steps to install WSO2 APIM Analytics. Because this procedure is identical to installing WSO2 Data Analytics Server (DAS), these steps take you to the DAS documentation for details.
    1. Ensure that you have met the Installation Prerequisites.
    2. Go to the installation instructions relevant to your operating system:
  3. To enable Analytics, open the <API-M_HOME>/repository/conf/api-manager.xml file and set the Enabled property under Analytics to true as shown below. Save this change.

    <Enabled>true</Enabled>

    If you are working on a distributed (clustered) setup of API Manager, do the configurations instructed to be done in API Manager in Publisher, Store and Gateway nodes.

  4. Share the WSO2AM_STATS_DB datasource between WSO2 API-M and WSO2 APIM Analytics as follows.
    1. Open the <API-M_HOME>/repository/conf/datasources/master-datasources.xml file and make sure that a configuration for the WSO2AM_STATS_DB datasource is included with your datasource configurations. The default configuration which is already available in master-datasources.xml file is as follows. It is configured for the in-built h2 database by default. You can change the datasource according to the database you use refering to change statistics datasource.

      <datasource>    
          <name>WSO2AM_STATS_DB</name>
          <description>The datasource used for getting statistics to API Manager</description>
          <jndiConfig>
              <name>jdbc/WSO2AM_STATS_DB</name>
          </jndiConfig>
          <definition type="RDBMS">
              <configuration>
                  <url>jdbc:h2:../tmpStatDB/WSO2AM_STATS_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000;AUTO_SERVER=TRUE</url>
                  <username>wso2carbon</username>
                  <password>wso2carbon</password>
                  <defaultAutoCommit>false</defaultAutoCommit>
                  <driverClassName>org.h2.Driver</driverClassName>
                  <maxActive>50</maxActive>
                  <maxWait>60000</maxWait>
                  <testOnBorrow>true</testOnBorrow>
                  <validationQuery>SELECT 1</validationQuery>
                  <validationInterval>30000</validationInterval>
              </configuration>
          </definition>
      </datasource>
    2. Open the <API-M_ANALYTICS_HOME>/repository/conf/datasources/stats-datasources.xml file and make sure that the same configuration of

      WSO2AM_STATS_DB in the <API-M_HOME>/repository/conf/datasources/master-datasources.xml file (mentioned in the previous sub step) is added in it.

  5. Open the <API-M_HOME>/repository/conf/log4j.properties file. Add the DAS_AGENT to the end of the root logger as shown below.

    This configuration is required only if you want to analyze WSO2 API-M logs using the Log Analyzer. If you are working on a distributed (clustered) setup of API Manager, follow these steps in Gateway Node of the setup.

    log4j.rootLogger=<other loggers>, DAS_AGENT
  6. Start the WSO2 API-M Analytics server, and then start the API Manager server. To start a WSO2 product server, navigate to the <PRODUCT_HOME>/bin directory in your console and run one of the following scripts as relevant.
    1. On Windows:  wso2server.bat --run
    2. On Linux/Mac OS:  sh wso2server.sh

You can now start using the API Manager for its usual operations and the required Analytics functionality is enabled.

Downloading WSO2 API-M Analytics

Follow the instructions below to download the binary distribution of WSO2 API-M Analytics.

The binary distribution contains the binary files for both MS Windows, and Linux-based operating systems. You can also download and build the source code.

  1. Go to the WSO2 API Manager page.
  2. Click DOWNLOAD ANALYTICS to download the WSO2 API-M Analytics product pack.

Installing WSO2 API-M Analytics

Take the following steps to install WSO2 API-M Analytics. Because this procedure is identical to installing WSO2 Data Analytics Server (DAS), these steps take you to the DAS documentation for details.

  1. Ensure that you have met the Installation Prerequisites.
  2. Go to the installation instructions relevant to your operating system:

Configuring WSO2 API Manager to publish statistics

Follow the instructions below to do the required configurations for WSO2 API-M to publish statistics in the WSO2 API-M Analytics server.

If you are working on a distributed (clustered) setup of API Manager, do the configurations instructed to be done in API Manager in Publisher, Store and Gateway nodes.


  1. Open the <API-M_HOME>/repository/conf/api-manager.xml file. 
  2. Under the <Analytics> sub element, set the Enabled parameter to true.
  3. Configure the following parameters if required.

    ParameterValueDescription
    <DASServerURL><protocol>://<hostname>:<port>/ The server URL of the remote DAS/CEP server used to collect statistics from WSO2 API Manager. An event can also be published to multiple receiver groups, where each group has one or more receivers. Receiver groups are delimited by curly braces, whereas receivers are delimited by commas.

    e.g., Three receivers within a single group can be specified as follows.
    tcp://localhost:7612,tcp://localhost:7613,tcp://localhost:7614

    Two receiver groups with two receivers each can be specified as follows.
    {tcp://localhost:7612,tcp://localhost:7613},{tcp://localhost:7712,tcp://localhost:7713}

    <DASUsername>A valid administrator username

    The administrator username to log into the remote DAS/CEP server that collects statistics from WSO2 API Manager.

    It is required to change the values DASUsername and DASPassword parameters if the user of WSO2 API-M is different to the user of WSO2 DAS/CEP. The default value ${admin.username} retrieves the admin user of the current API-M node as the DASUsername.

    <DASPassword>The password of the username specified.

    The administrator password to log into the remote DAS/CEP server that collects statistics from WSO2 API Manager.

    It is required to change the values DASUsername and DASPassword parameters if the user of WSO2 API-M is different to the user of WSO2 DAS/CEP. The default value ${admin.password} retrieves the admin user of the current API-M node as the DASUsername

    <DASRestApiURL>https:/<host>:<port>The WSO2 DAS (WSO2 API-M Analytics) REST API URL. The WSO2 DAS REST API connection information, which are included under the REST API-M connection information, are defined as global properties, as they are common to all the WSO2 API-M analytics.
    <DASRestApiUsername>A valid administrator usernameThe administrator username to log into the remote WSO2 DAS (WSO2 API-M Analytics) server.
    <DASRestApiPassword>The password of the username specified.The administrator password to log into the remote WSO2 DAS (WSO2 API-M Analytics) server.
  4. Save the changes.

Configuring the Log Analyzer

This configuration is required only if you want to analyze WSO2 API-M logs using the Log Analyzer. If you are working on a distributed (clustered) setup of API Manager, follow these steps in Gateway Node of the setup.

Follow the steps below to configure the Log Analyzer.

  1. Add DAS_AGENT to the end of the root logger in the <API-M_HOME>/repository/conf/log4j.properties file as shown below.

    log4j.rootLogger=<other loggers>, DAS_AGENT


    Then check and make sure that the following configuration is available in the same file. Modify the values for userNamepassword and/or url if required.

    The values given below are the default values of the configuration.

    If you have multiple analytics nodes configured, see Setting up Multi Receiver and Load Balancing Data Agent to understand how to configure DAS_AGENT.url

    # DAS_AGENT is set to be a Custom Log Appender.
    log4j.appender.DAS_AGENT=org.wso2.carbon.analytics.shared.data.agents.log4j.appender.LogEventAppender
    # DAS_AGENT uses PatternLayout.
    log4j.appender.DAS_AGENT.layout=org.wso2.carbon.analytics.shared.data.agents.log4j.util.TenantAwarePatternLayout
    log4j.appender.DAS_AGENT.columnList=%D,%S,%A,%d,%c,%p,%m,%H,%I,%Stacktrace
    log4j.appender.DAS_AGENT.userName=admin
    log4j.appender.DAS_AGENT.password=admin
    log4j.appender.DAS_AGENT.url=tcp://localhost:7612
    log4j.appender.DAS_AGENT.maxTolerableConsecutiveFailure=5
    log4j.appender.DAS_AGENT.streamDef=loganalyzer:1.0.0
  2. If the API-M server and the DAS server run on two different hosts with their default certificates, set the HostnameVerifier to AllowAll in the Pass Through HTTP SSL Sender section of the <API-M_HOME>/repository/conf/axis2/axis2.xml file as follows. If this setting is not changed, the javax.net.ssl.SSLException: Host name verification failed for host exception may occur for the API-M instance.

        <transportSender name="https"  
            class="org.apache.synapse.transport.passthru.PassThroughHttpSSLSender">
            <parameter name="non-blocking" locked="false">true</parameter>
            <parameter name="keystore" locked="false">
                <KeyStore>
                    <Location>repository/resources/security/wso2carbon.jks</Location>
                    <Type>JKS</Type>
                    <Password>wso2carbon</Password>
                    <KeyPassword>wso2carbon</KeyPassword>
                </KeyStore>
            </parameter>
            <parameter name="truststore" locked="false">
                <TrustStore>
                    <Location>repository/resources/security/client-truststore.jks</Location>
                    <Type>JKS</Type>
                    <Password>wso2carbon</Password>
                </TrustStore>
            </parameter>
            <parameter name="HostnameVerifier">AllowAll</parameter>
        </transportSender>
  3. If the WSO2 API Manager was started before these configurations were done, restart it in order to apply the changes.

Securing log4j properties file with a secure vault

Using secure vaults allows you to avoid exposing passwords by having them in plain text in the log4j properties file. Follow the procedure below if you want to secure the log4j properties file with a secure vault.

  1. If you have not already generated the secret-conf.properties file with default values, navigate to the <API-M_HOME>/bin directory and execute the following command to generate it. This generates the secret-conf.properties file in the <PRODUCT_HOME>/repository/conf/security directory. If you have already generated this file, proceed to Step 2.
    sh ciphertool.sh -Dconfigure

    Enter wso2carbon as the keystore password when the following appears in the console.
    [Please Enter Primary KeyStore Password of Carbon Server : ] 
  2. Execute the following command from the <API-M_HOME>/bin directory. This generates the encrypted value for the clear text password.
    sh ciphertool.sh
    1. Enter wso2carbon when the following appears in the console.
      [Please Enter Primary KeyStore Password of Carbon Server : ]
    2. Enter admin as the input value when the following appears in the log. (This value is entered based on the secret-conf.properties file you generated in step 1.)
      [Enter Plain text value :]
      [Please Enter value Again :]
    The following output is displayed in the console.

    Encryption is done Successfully

    The encrypted value is: 
    MpfXhKP+iJSImA/KNa+DoOXCPQAyF3JLhlFNAdG6F3naWK+N1/WEWOJkFx4kK34i1VtkywNN9SiC
    MRQGTFw+nqzK5/INgcFFdoxv49lM/FJw8CyXKQ0JWdxw5QJPtrjJzvGp6Rj6xt4ysb6HdG5uNG+a
    1E0lqdmzGUYQ6oejIlk=
  3. Open the <API-M_HOME>/repository/conf/security/ cipher-text.properties file and add the following entry.

    log4j.appender.DAS_AGENT.password=MpfXhKP+iJSImA/KNa+DoOXCPQAyF3JLhlFNAdG6F3naWK+N1/WEWOJkFx4kK34i1VtkywNN9SiC
    MRQGTFw+nqzK5/INgcFFdoxv49lM/FJw8CyXKQ0JWdxw5QJPtrjJzvGp6Rj6xt4ysb6HdG5uNG+a
    1E0lqdmzGUYQ6oejIlk=
  4. Open the <API-M_HOME>/repository/conf/log4j.properties file and configure the password as log4j.appender.DAS_AGENT.password=secretAlias:log4j.appender.DAS_AGENT.password (as shown below).

    log4j.appender.DAS_AGENTt=org.wso2.carbon.analytics.shared.data.agents.log4j.appender.LogEventAppender
    log4j.appender.DAS_AGENT.layout=org.wso2.carbon.analytics.shared.data.agents.log4j.util.TenantAwarePatternLayout
    log4j.appender.DAS_AGENT.columnList=%D,%S,%A,%d,%c,%p,%m,%H,%I,%Stacktrace
    log4j.appender.DAS_AGENT.userName=admin
    log4j.appender.DAS_AGENT.password=secretAlias:log4j.appender.DAS_AGENT.password
    log4j.appender.DAS_AGENT.url=tcp://localhost:7612
    log4j.appender.DAS_AGENT.maxTolerableConsecutiveFailure=5
    log4j.appender.DAS_AGENT.streamDef=loganalyzer:1.0.0
  5. Start the WSO2 API Manager server by running one of the following commands from the <API-M_ANALYTICS_HOME>/bin directory. 
    • On Windows:  wso2server.bat --run
    • On Linux/Mac OS:  sh wso2server.sh

Enter wso2carbon when the following appears in the console.
[Enter KeyStore and Private Key Password :] 

If you want to start the server as a background process, carry out the following steps before starting the server.

  1. Create a file named password-tmp.txt in the <API-M_HOME> directory. Add wso2carbon (the primary keystore password) to this file and save.

    By default, the password provider assumes that both the private key and the keystore passwords are the same. If you want them to be different, the private key password should be entered in the second line of the file.

  2. The keystore password is picked from the password-tmp.txt file. This file is automatically deleted from the file system when you start the server. Make sure to add this temporary file back whenever you start the server as a background process.

    If you name the password file password-persist.txt instead of password-tmp.txt, then the file is not deleted once the server is started. Therefore, it is not required to provide the password in subsequent startups.


Configuring databases

Configuring databases allow you to persist data relating to APIs, process them and analyze. Follow the procedure below to configure databases.

The following is a list of database versions that are compatible with WSO2 APIM Analytics.

  • Postgres 9.5 and later
  • MySQL 5.6
    MySQL 5.7 
  • Oracle 12c
  • MS SQL Server 2012
  • DB2


  1. Open the <APIM-ANALYTICS_HOME>/repository/conf/datasources/analytics-datasources.xml file. Note that two datasources named as WSO2_ANALYTICS_EVENT_STORE_DB and WSO2_ANALYTICS_PROCESSED_DATA_STORE_DB are configured by default to point to the H2 databases.
  2. Create two database schemas in your database server (MySQL, Oracle, etc) for the two datasources, and change the configurations of those datasources to point to the relevant schemas. A sample configuration is given below. 

    The database user you provide here requires permissions to create tables. 


    <datasource>
        <name>WSO2_ANALYTICS_EVENT_STORE_DB</name>
        <description>The datasource used for analytics record store</description>
        <definition type="RDBMS">
            <configuration>
                <url>jdbc:mysql://localhost:3306/stats_200?autoReconnect=true&amp;relaxAutoCommit=true</url>
                <username>root</username>
                <password>root</password>
                <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                <maxActive>50</maxActive>
                <maxWait>60000</maxWait>
                <testOnBorrow>true</testOnBorrow>
                <validationQuery>SELECT 1</validationQuery>
                <validationInterval>30000</validationInterval>
                <defaultAutoCommit>false</defaultAutoCommit>
            </configuration>
        </definition>
    </datasource>
    • If you are using DB2, run this script before you start the WSO2 API-M Analytics server.

    • If you are using MySQL 5.7, open <wso2am-analytics-2.0.0>/repository/conf/analytics/spark/spark-jdbc-config.xml and configure the stringType property under the typeMapping element as follows.
      <stringType>VARCHAR(100)</stringType> 
  3. Share the WSO2AM_STATS_DB datasource between WSO2 API-M and WSO2 API-M Analytics as follows.

    1. Open the <API-M_HOME>/repository/conf/datasources/master-datasources.xml file and make sure that a configuration for the WSO2AM_STATS_DB datasource is included. The default configuration is as follows.

      <datasource>
         <name>WSO2AM_STATS_DB</name>
         <description>The datasource used for setting statistics to API Manager</description>
         <jndiConfig>
            <name>jdbc/WSO2AM_STATS_DB</name>
         </jndiConfig>
         <definition type="RDBMS">
            <configuration>
               <url>jdbc:mysql://localhost:3306/WSO2AM_STATS_DB?autoReconnect=true&amp;relaxAutoCommit=true</url>
               <username>root</username>
               <password>root</password>
               <driverClassName>com.mysql.jdbc.Driver</driverClassName>
               <maxActive>50</maxActive>
               <maxWait>60000</maxWait>
               <testOnBorrow>true</testOnBorrow>
               <validationQuery>SELECT 1</validationQuery>
               <validationInterval>30000</validationInterval>
               <defaultAutoCommit>false</defaultAutoCommit>
            </configuration>
         </definition>
      </datasource>
    2. Open the <API-M_ANALYTICS_HOME>/repository/conf/datasources/stats-datasources.xml file and make sure that the same configuration in the <API-M_HOME>/repository/conf/datasources/master-datasources.xml file (mentioned in the previous sub step) is added in it.
  4. In the <API-M_ANALYTICS_HOME>/repository/conf/datasources/stats-datasources.xml file, a datasource named WSO2AM_STATS_DB is configured by default to point to H2 databases.

    Create a schema in your database server similar to the WSO2AM_STATS_DB datasource. Make sure that this datasource points to the relevant schema. 

     The database user you provide here requires permissions to create tables.
  5. Download and copy the relevant database driver JAR file to the <API-M_ANALYTICS_HOME>/repository/components/lib directory.
  6. Start the WSO2 API-M Analytics server.


  • No labels