This documentation is for WSO2 API Manager 1.7.0 View documentation for the latest release.
Publishing API Runtime Statistics - API Manager 1.7.0 - WSO2 Documentation

All docs This doc
||
Skip to end of metadata
Go to start of metadata

You can set up WSO2 Business Activity Monitor (version 2.4.1 used here) to collect and analyze runtime statistics from the API Manager. To publish data from the API Manager to BAM, the Thrift protocol is used. Information processed in BAM is stored in a database from which the API Publisher retrieves information before displaying in the corresponding UI screens.

By default, org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataPublisher is configured to collect data events from WSO2 BAM. If you use a system other than WSO2 BAM to collect and analyze runtime statistics, you write a new data publishing agent by extending APIMgtUsageDataPublisher . Find the API templates inside <APIM_HOME>/repository/resources/api_templates. When writing a new data publishing agent, make sure the data publishing logic you implement has a minimal impact to API invocation.

The data source and database names used in this guide are just examples. They may vary depending on your configurations.

Prerequisites

  • JDK 1.6.* or 1.7

    If you install JDK in Program Files in the Windows environment, avoid the space by using PROGRA~1 when specifying environment variables for JAVA_HOME and PATH. Else, the server throws an exception.

     

  • Cygwin ( http://www.cygwin.com ) : Required only if you are using Windows. WSO2 BAM analytics framework depends on Apache Hadoop, which requires Cygwin in order to run on Windows. Install at least the basic net (OpenSSH,tcp_wrapper packages) and security related Cygwin packages. After Cygwin installation, update the PATH variable with C:/cygwin/bin and restart BAM.

Configuring WSO2 API Manager

If you are running a distributed API Manager setup and the registry is not shared, do the same configurations in all API Gateway nodes, API Publisher node and API Store nodes.

  1. Do the following changes in <APIM_HOME>/repository/conf/api-manager.xml file:

    • Enable API usage tracking by setting the <APIUsageTracking> element to true.
    • Because you will apply an offset to the default BAM port later in this guide, you need to apply the same offset to the default Thrift port. To do that, change the port value to 7614 in the <ThriftPort> element of this file. The API Manager will then push the data to BAM through port 7614, using the Thrift protocol.
    • Uncomment the <DataSourceName> element. It sets the datasource used to get statistics from BAM
    • Set <BAMServerURL> to tcp://<BAM host IP>:7614/ where <BAM host IP> is the machine IP address. Do not use localhost unless you're in a disconnected mode.

      <BAMServerURL> refers to the endpoint to which events will be published from the API Gateway. This endpoint is also known as the event receiver. You can define multiple event receiver groups, each with one or more receivers. A receiver group is defined within curly braces and receiver URLs are delimited by commas.

      For example, <BAMServerURL>{tcp://localhost:7612/,tcp://localhost:7613/},{tcp://localhost:7712/,tcp://localhost:7713/}</BAMServerURL>. This example has two receiver groups defined with two receivers in each group. When a request passes through the API Gateway, an event will be published to one selected receiver in each group.

    <APIUsageTracking>
        <!-- Enable/Disable the API usage tracker. -->
        <Enabled>true</Enabled>   
        <PublisherClass>org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher</PublisherClass>
        <ThriftPort>7614</ThriftPort> 
        <BAMServerURL>tcp://<BAM host IP>:7614/</BAMServerURL>
        <BAMUsername>admin</BAMUsername>
        <BAMPassword>admin</BAMPassword>
        <!-- JNDI name of the data source to be used for getting BAM statistics. This data source should
            be defined in the master-datasources.xml file in conf/datasources directory. -->
        <DataSourceName>jdbc/WSO2AM_STATS_DB</DataSourceName>
    </APIUsageTracking>
  2. Specify the datasource definition under the <datasource> element in the <APIM_HOME>/repository/conf/datasources/master-datasources.xml file. The tables are created automatically when the Hive script runs. You just need to create the schema. The example below connects to a MySQL instance:

    The WSO2AM_STATS_DB database is not available in <BAM_HOME>/repository/database directory at this point. It is created only after BAM starts up.

    <datasource>
        <name>WSO2AM_STATS_DB</name>
        <description>The datasource used for getting statistics to API Manager</description>
        <jndiConfig>
            <name>jdbc/WSO2AM_STATS_DB</name>
        </jndiConfig>
        <definition type="RDBMS">
            <configuration>
                <url>jdbc:mysql://localhost:3306/stats_db?autoReconnect=true&amp;</url>
                <username>db_username</username>
                <password>db_password</password>
                <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                <maxActive>50</maxActive>
                <maxWait>60000</maxWait>
                <testOnBorrow>true</testOnBorrow>
                <validationQuery>SELECT 1</validationQuery>
                <validationInterval>30000</validationInterval>
             </configuration>
        </definition>
     </datasource>
  3. Save the database driver JAR inside both <AM_HOME>/repository/components/lib and <BAM_HOME>/repository/components/lib folders.

Next, prepare BAM to collect and analyze statistics from the API Manager.

Configuring WSO2 BAM

  1. Download WSO2 BAM 2.4.1 from location: http://wso2.com/products/business-activity-monitor.
  2. Apply an offset of 3 to the default BAM port by editing the <BAM_HOME>/repository/conf/carbon.xml file. This step is done when you run both products on the same server.

    <Offset>3</Offset>

    This increments all ports used by the server by 3, which means the BAM server will run on port 9446. Port offset is used to increment the default port by a given value. It avoids possible port conflicts when multiple WSO2 products run in same host.

  3. Do the following changes in <BAM_HOME>/repository/conf/datasources/bam_datasources.xml file:
    • Copy/paste WSO2AM_STATS_DB definition from API Manager's master-datasources.xml file. You edited it in step 2. WSO2AM_STATS_DB is used to fetch analytical data from the database.
    • Replace the port of WSO2BAM_CASSANDRA_DATASOURCE in URL ( jdbc:cassandra://localhost:9163/EVENT_KS).
      Note that localhost is used here; not the machine IP. Cassandra is bound by default on localhost, unless you change the data-bridge/data-bridge-config.xml file. Also, if you are running BAM on a different server, the port will be different.

      Do not edit the WSO2BAM_UTIL_DATASOURCE with an offset value as it is using the offset.

      Manually updating the port of WSO2BAM_CASSANDRA_DATASOURCE is not needed if you are using WSO2 BAM 2.5.0.

  4. Copy the file <APIM_HOME>/statistics/API_Manager_Analytics.tbox to directory <BAM_HOME>/repository/deployment/server/bam-toolbox.
    If this folder is not in the BAM installation directory by default, create it. The toolbox describes the information collected, how to analyze the data, as well as the location of the database where the analyzed data is stored.
  5. Open <BAM_HOME>/repository/conf/etc/hector-config.xml file and change the port to localhost:9163. You must add the other nodes too when configuring a clustered setup.

    <Nodes>localhost:9163</Nodes>

    Step 5 is not needed if you are using WSO2 BAM 2.5.0.

  6. Restart BAM server by running <BAM_HOME>/bin/wso2server.[sh/bat].

  7. Optional: If you want to host the BAM server on a different machine or change the running port, you must edit the <APIUsageTracking> node in <APIM_HOME>/repository/conf/api-manager.xml file as follows:

    <!--API usage tracker configuration used by the BAM data publisher in API gateway.-->
        <APIUsageTracking>
            <!-- Enable/Disable the API usage tracker.-->
            <Enabled>true</Enabled>
    
            <!-- API Usage Data Publisher.-->
            <PublisherClass>org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher</PublisherClass>
    
            <!--Thrift port of the remote BAM server.-->
            <ThriftPort>7612</ThriftPort>
    
            <!-- Server URL of the remote BAM server used to collect statistics. Must be specified in protocol://hostname:port/ format.-->
            <BAMServerURL>tcp://localhost:7614</BAMServerURL>
    
            <!--Administrator username to login to the remote BAM server.-->
            <BAMUsername>admin</BAMUsername>
    
            <!--Administrator password to login to the remote BAM server.-->
            <BAMPassword>admin</BAMPassword>
    
            <!--JNDI name of the data source to be used for getting BAM statistics.This data source should be defined in the master    
                datasources.xml file in conf/datasources directory.-->
            <DataSourceName>jdbc/WSO2AM_STATS_DB</DataSourceName>
        </APIUsageTracking>

Troubleshooting common issues

Given below is how to do troubleshoot some common issues users come across:

  1. Do you get an exception as unable to connect to server Cassandra?
    Check if you changed the Cassandra port according to the port offset applied to the default BAM port. See Step 3 under configuring BAM section.
  2. Do you get a connection refused exception on the BAM console?
    This happens when you  execute Hive scripts prior to changing the default port. Add the following line at the beginning of the Hive scripts and rerun: drop table <hive_cassandra_table_name>; You can find the Hive scripts deployed with the toolbox file, which is inside <BAM_HOME>/repository/deployment/server/bam-toolbox folder. For information, see Editing an Analytic Script in WSO2 BAM documentation.

Changing the statistics database

To use a different database than the default H2 for statistical publishing, you must change the properties of the datasource element, and additionally delete some metadata tables created by previous executions of the Hive script, if there are any.

To delete the metadata tables,

  1. Log in to BAM management console and select Add in Analytics menu.
  2. Go to the Script Editor in the window that opens.
  3. Execute the  following script.

    drop TABLE APIRequestData; 
    drop TABLE APIRequestSummaryData; 
    drop TABLE APIVersionUsageSummaryData; 
    drop TABLE APIResourcePathUsageSummaryData; 
    drop TABLE APIResponseData; 
    drop TABLE APIResponseSummaryData; 
    drop TABLE APIFaultData; 
    drop TABLE APIFaultSummaryData; 
    drop TABLE APIDestinationData; 
    drop TABLE APIDestinationDataSummaryData; 

After configuring WSO2 BAM to render and produce statistics of APIs hosted and managed in the API Manager, you can view them through various statistical dashboards in the API Publisher, depending on your permission levels. For information, refer to section Viewing API Statistics .

  • No labels