This documentation is for WSO2 API Manager 2.1.0 View documentation for the latest release.
Introducing the WSO2 API Manager Statistics Model - API Manager 2.1.0 - WSO2 Documentation

All docs This doc
||
Skip to end of metadata
Go to start of metadata

Introduction

This section describes and illustrates the API Manager statistic publishing and generating model. It describes the internal components of API Manager, external analyzer information and other data retrieval components. API Manager generates events based on the API Manager invocation pattern and publishes them to all the listening event analyzers. The analyzer is responsible for the accumulation of all events and generates summary data based on the defined summarisation logic. After the summarized data is generated, the API Manager Dashboard can retrieve statistics from the analyzer data-source to the UI via the API Manager analytics client.

API Manager usage publisher

The internal API Manager component listens to the API Manager invocations and its behavior. Based on the request and responses, the event is generated and published to all the event receivers. This publisher publishes the following event streams,

  • org.wso2.apimgt.statistics.request
  • org.wso2.apimgt.statistics.response

  • org.wso2.apimgt.statistics.fault

  • org.wso2.apimgt.statistics.throttle

Workflow executor

This component publishes the org.wso2.apimgt.statistics.workflow event stream containing work-flow related event data.

Data analyzer

The data analyzer is a summary data generator based on the received event. WSO2 API-M uses WSO2 APIM Analytics to perform analytics tasks. APIM Analytics uses Apache Spark as a processing language. The <APIM-ANALYITCS_HOME>/repository/deployment/server/carbonapps/org_wso2_carbon_analytics_apim-1.0.0.car CAR file (which is deployed in the APIM Analytics server by default) contains details of all summarized data and their destination.

A Carbon Application (C-App) or a CAR file is a collection of artifacts deployable on a WSO2 product instance. These artifacts are usually JAVA-based or XML configurations, designed differently for each product in the WSO2 Carbon platform. You can deploy these artifacts to generate services.

A single WSO2 product can have numerous artifacts such as Axis2 services, dataservices, synapse configurations, endpoints, proxy services, mediators, registry resources, BPEL workflows etc. Usually, these artifacts are created in a development environment and then moved one by one into staging/production environments. Manually configuring artifacts to build up the entire solution this way is a time-consuming task. Instead, you can bundle configuration files and artifacts in a C-App and port Web service based solutions across environments more easily. C-Apps allow you to export your entire solution as a single archive file.


API Manager event streams

API-M 2.1.0 provides six types of event streams as listed below.

org.wso2.apimgt.statistics.request

This stream tracks information for the API request.

org.wso2.apimgt.statistics.request
meta_clienType  :       STRING          :           Meta information of Client type
consumerKey     :       STRING			: 			Consumer key of API invoked client application	
context         :       STRING			: 			API context depending on the user's request
api_version     :       STRING			: 			API synapse artifact contained name [API Provider +"--"+API Name]
api             :       STRING			: 			API Name
resourcePath    :       STRING			: 			API resource URL pattern of API request
method          :       STRING			: 			HTTP Verb of API request [e.g.:GET/POST]
version         :       STRING			: 			API version
request         :       INT				: 			Request count (e.g. 1)
requestTime     :       LONG			: 			API request hit time in APIM
userId          :       STRING			: 			API invoked end user name
tenantDomain    :       STRING			: 			Tenant domain of API provider
hostName        :       STRING			: 			API Manager server host
apiPublisher    :       STRING			: 			API provider
applicationName :       STRING			: 			Name of the client application
applicationId   :       STRING			: 			ID of the client application
userAgent       :       STRING			: 			User agent of the user
tier            :       STRING			: 			Name of the throttling policy assigned to the request	
throttledOut    :       BOOL			: 			Describes whether this is a throttled request or not
clientip        :       STRING          :            IP Adress of the Client
applicationOwner:       STRING          :            Name of the Owner of the Application


org.wso2.apimgt.statistics.response

This stream tracks information for the API response. It includes the time taken for the response to get back, the received time etc.

org.wso2.apimgt.statistics.response
meta_clientType :       STRING          :           Meta information of Client type
consumerKey     :       STRING			: 			Consumer key of the client application invoking the API
context         :       STRING			: 			API context depending on the user's request
api_version     :       STRING			: 			API synapse artifact contained name [API Provider +"--"+API Name]	
api             :       STRING			: 			API Name
resourcePath    :       STRING			: 			API resource URL pattern of API request
resourceTemplate:       STRING          :           API resource URL template of the API request
method          :       STRING			: 			HTTP Verb of API request [e.g.:GET/POST]
version         :       STRING			: 			API version
response        :       INT				: 			Response count (e.g. 1)
responseTime    :       LONG			: 			Total time taken for request/response flow[serviceTime+backendTime]
serviceTime     :       LONG			: 			Time taken to serve the API request in APIM side
backendTime     :       LONG			: 			Time taken process the request at the backend	
username        :       STRING			: 			API invoked end user name	
eventTime       :       LONG			: 			Timestamp of response event published
tenantDomain    :       STRING			: 			Tenant domain of API provider
hostname        :       STRING			: 			API Manager server hostname
apiPublisher    :       STRING			: 			API provider
applicationName :       STRING			: 			Name of the client application
applicationId   :       STRING			: 			ID of the client application
cacheHit        :       BOOL			: 			Describes if response caching is enabled or not
responseSize    :       LONG			: 			Response message size in bytes	
protocol        :       STRING 			: 			Protocol used to send the response (HTTP/HTTPS) and the port
responseCode    :       STRING          :           HTTP Response Code
destination     :       STRING          :           URL of the Desination IP

org.wso2.apimgt.statistics.fault

This stream contains the fault API invocations. It includes the API with back end errors, timeout etc.

org.wso2.apimgt.statistics.fault
meta_clientType :       STRING          :           Meta information of Client Type
consumerKey     :       STRING			: 			Consumer key of the client application invoking the API	
context         :       STRING			: 			API context depending on the user's request	
api_version     :       STRING			: 			API version
api             :       STRING			: 			API Name
resourcePath    :       STRING			: 			API resource url pattern of API request
method          :       STRING			: 			HTTP Verb of API request [e.g.:GET/POST]
version         :       STRING			: 			API version
errorCode       :       STRING			: 			HTTP error code
errorMessage    :       STRING			: 			Description of error message
requestTime     :       LONG			: 			API request time in millisecond
userId          :       STRING			: 			API invoked end user name
tenantDomain    :       STRING			: 			Tenant domain of API provider
hostName        :       STRING			: 			API Manager server host
apiPublisher    :       STRING     		: 			API provider
applicationName :       STRING			: 			Name of the client application
applicationId   :       STRING			: 			ID of the client application 
protocol        :       STRING			: 			Protocol used to send the response (HTTP/HTTPS) and the port

org.wso2.apimgt.statistics.throttle

This stream contains the API invocation with throttle information. Throttling can happen due to any of the following reasons:

  • The application limit has exceeded.

  • The resource limit has exceeded.

  • The API limit has exceeded.

  • The hard level limit has exceeded.

org.wso2.apimgt.statistics.throttle
meta_clientType     :       STRING          :           meta information of Client Type
accessToken         :       STRING			: 			Access token of the request
userId              :       STRING			: 			API invoked end user name
tenantDomain        :       STRING			: 			Tenant domain of API provider
api                 :       STRING			: 			API Name
api_version         :       STRING			: 			API synapse artifact contained name [API Provider +"--"+API Name]
context             :       STRING			: 			API context depending on the user's request
apiPublisher        :       STRING			: 			API provider
throttledTime       :       LONG			: 			The timestamp which throttle out event triggers
applicationName     :       STRING			: 			Name of the client application
applicationId       :       STRING			: 			ID of the client application
subscriber          :       STRING          :           Name of the subscriber of the Application
throttledOutReason  :       STRING 			: 			The reason describing why the request has been throttled out 

org.wso2.apimgt.statistics.workflow

This event stream creates events based on the API Manager workflow and publishes them to the analyzer.

org.wso2.apimgt.statistics.workflow
meta_clientType     :       STRING          :           meta information of Client Type
workflowReference   :       STRING			: 			Holds the workflow reference ID	
workflowStatus      :       STRING			: 			Status of the workflow e.g.: CREATED, APPROVED, REJECTED, REGISTERED
tenantDomain        :       STRING			: 			Tenant domain of subscriber who triggers the workflow in APIStore
workflowType        :       STRING			: 			Type of the workflow e.g.:AM_APPLICATION_CREATION,
createdTime         :       LONG			: 			The workflow was creation time in milliseconds
updatedTime         :       LONG			: 			The last updated time of the workflow in milliseconds

org.wso2.apimgt.statistics.execution.time

This stream contains information relating to API invocation including time stamps and the time taken by the API at different stages of invocation (e.g., time taken to backend, time taken to mediation flow, response time etc.).

org.wso2.apimgt.statistics.execution.time
meta_clientType         :       STRING      :           meta information of Client Type
api						:		STRING		:			API Name	
api_version				:		STRING		:			API Version
tenantDomain			:		STRING		:			Tenant domain of subscriber who triggers the workflow in APIStore
apiPublisher			:		STRING		:			API Provider
apiResponseTime			:		LONG		:			Total time taken for request/response flow
context					:		STRING		:			API context depending on the user's request
securityLatency			:		LONG		:			Time taken for authentication
throttlingLatency 		:		LONG		:			Time taken for throttling the request/response
requestMediationLatency	:		LONG		:			Time taken to mediate the request
responseMediationLatency:		LONG		:			Time taken to mediate the response
backendLatency			:		LONG		:			Time taken by the backend to return the response
otherLatency			:		LONG		:			Time taken to process tasks other than mentioned above
eventTime				:		LONG		:			Timestamp of the sent event


API Manager summarized tables (APIM Analytics internal storage)

Note that these summarized tables are stored in the APIM Analytics internal storage. Both the C-Apps that API Manager deploy on APIM Analytics first stores the summary data into these tables. There are additional columns in some of these tables containing the composition column of other columns. Those columns types are of facet type, which is used to support the aggregation function on the APIM Analytics REST API. All the columns are indexed in order to search using Apache Lucene and supports the APIM Analytics REST API. When API Manager is configured with the RDBMS client, all these tables are replicated in the external RDBMS except for the facet columns.

  • API_REQUEST_SUMMARY

    This table contains the summary data of the request event stream.

    API_REQUEST_SUMMARY table schema
    CREATE TEMPORARY TABLE API_REQUEST_SUMMARY_FINAL USING CarbonAnalytics OPTIONS (tableName "API_REQUEST_SUMMARY",
      schema "api string -i,
      api_version string -i,
      version string -i,
      apiPublisher string -i,
      consumerKey string -i,
      userId string -i,
      context string -i,
      max_request_time long -i,
      total_request_count int -i,
      hostName string -i,
      year int -i,
      month int -i,
      day int -i,
      time string -i,
      key_api_facet facet -i,
      key_userId_facet facet -i,
      api_version_userId_facet facet -i,
      api_version_userId_apiPublisher_facet facet -i,
      api_version_userId_context_facet facet -i",
      primaryKeys "api,api_version,version,apiPublisher,consumerKey,userId,context,hostName,year,month,day"
      );
     Expand to find the table of descriptions for each column
    api				: API Name
    api_version		: API synapse artifact contained name [API Provider +"--"+API Name]
    version			: API version
    apiPublisher	: API provider
    context			: API context depending on the user's request	
    consumerKey		: Consumer key of the client application invoking the API
    userId			: End user name invoked by the API
    max_request_time: Time of the latest API request occurrence
    total_request_count: Total request count for the requests coming for same API
    hostname		: APIM server hostname
    year			: The year of initial API request occurred of the batch of API requests
    month			: The month of initial API request occurred of the batch of API requests
    day				: The date of API initial request occurred of the batch of API requests
    time			: The time of API initial request occurred of the batch of API requests
  • API_VERSION_USAGE_SUMMARY

    This table contains the summary data for API Manager usage. It is also derived from the request event table.

    VERSION_USAGE_SUMMARY table schema
    CREATE TEMPORARY TABLE API_VERSION_USAGE_SUMMARY_FINAL USING CarbonAnalytics OPTIONS (tableName "API_VERSION_USAGE_SUMMARY",
      schema "api string -i,
      version string -i,
      apiPublisher string -i,
      context string -i,
      total_request_count int -i,
      hostName string -i,
      year int -i,
      month int -i,
      day int -i,
      time string -i,
      max_request_time long -i,
      api_version_context_facet facet -i",
      primaryKeys "api,version,apiPublisher,context,hostName,year,month,day"
      );
     Expand to find the table of descriptions for each column
    api				: API Name
    version			: API version
    apiPublisher	: API provider
    context			: API context depending on the user's request	
    total_request_count: Total request count of an API version
    hostname		: APIM server hostname
    year			: The year of initial API request occurred of the batch of API requests
    month			: The month of initial API request occurred of the batch of API requests
    day				: The date of API initial request occurred of the batch of API requests
    time			: The time of API initial request occurred of the batch of API requests
  • API_Resource_USAGE_SUMMARY

    This table contains the summarized data for API Manager usage by resources and it is also derived from request event table.

    API_Resource_USAGE_SUMMARY table schema
      CREATE TEMPORARY TABLE API_Resource_USAGE_SUMMARY_FINAL USING CarbonAnalytics OPTIONS (tableName "API_Resource_USAGE_SUMMARY",
      schema "api string -i,
      version string -i,
      apiPublisher string -i,
      consumerKey string -i,
      resourcePath string -i,
      context string -i,
      method string -i,
      total_request_count int -i,
      hostName string -i,
      year int -i,
      month int -i,
      day int -i,
      time string -i,
      max_request_time long -i,
      key_api_method_path_facet facet -i,
      api_version_context_method_facet facet -i",
      primaryKeys "api,version,apiPublisher,consumerKey,context,resourcePath,method,hostName,year,month,day"
      );
     Expand to find the table of descriptions for each column
    api				: API Name
    version			: API version
    apiPublisher	: API provider
    context			: API context depending on the user's request
    consumerKey		: Consumer key of the client application invoking the API	
    resourcePath	: API resource url pattern of API request
    method			: HTTP Verb of API request [eg:GET/POST]
    total_request_count :Total request count for a particular API resource pattern
    hostname		: API Manager server hostname
    year			: The year of initial API request occurred of the batch of API requests
    month			: The month of initial API request occurred of the batch of API requests
    day				: The date of API initial request occurred of the batch of API requests
    time			: The time of API initial request occurred of the batch of API requests
  • API_RESPONSE_SUMMARY

    This table contains the summarized data from API responses. It is derived from the response event table.

    API_RESPONSE_SUMMARY table schema
      CREATE TEMPORARY TABLE API_RESPONSE_SUMMARY_FINAL USING CarbonAnalytics OPTIONS (tableName "API_RESPONSE_SUMMARY",
      schema "api_version string -i,
      apiPublisher string -i,
      context string -i,
      serviceTime int -i,
      total_response_count int -i,
      hostName string -i,
      year int -i,
      month int -i,
      day int -i,
      time string -i,
      max_request_time long -i,
      api_version_context_facet facet -i",
      primaryKeys "api,version,apiPublisher,context,hostName,year,month,day"
      );
     Expand to find the table of descriptions for each column
    api_version			: API synapse artifact contained name [API Provider +"--"+API Name]
    apiPublisher		: API provider
    context				: API context depending on the user's request
    consumerKey			: Consumer key of the client application invoking the API	
    serviceTime			: Total time taken to serve the batch of API requests in APIM side
    total_response_count: Total response count for the API requests for a specific API 
    hostname			: API Manager server hostname
    year				: The year of initial API request occurred of the batch of API requests
    month				: The month of initial API request occurred of the batch of API requests
    day					: The date of API initial request occurred of the batch of API requests
    time				: The time of API initial request occurred of the batch of API requests
  • API_FAULT_SUMMARY

    This table contains the summarized data of faulty API invocations and is derived from the fault event stream.

    API_FAULT_SUMMARY table schema
      CREATE TEMPORARY TABLE API_FAULT_SUMMARY_FINAL USING CarbonAnalytics OPTIONS (tableName "API_FAULT_SUMMARY",
      schema "api string -i,
      version string -i,
      apiPublisher string -i,
      consumerKey string -i,
      context string -i,
      total_fault_count int -i,
      hostName string -i,
      year int -i,
      month int -i,
      day int -i,
      time string -i,
      max_request_time long -i,
      consumerKey_api_facet facet -i,
      api_version_apiPublisher_context_facet facet -i",
      primaryKeys "api,version,apiPublisher,consumerKey,context,hostName,year,month,day"
      );
     Expand to find the table of descriptions for each column
    api				: API Name
    version			: API version
    apiPublisher	: API provider
    consumerKey		: Consumer key of the client application invoking the API	
    context			: API context depending on the user's request
    total_fault_count: Total faulty API request count for a specific API 
    hostname		: APIM server hostname
    year			: The year of initial API request occurred of the batch of API requests
    month			: The month of initial API request occurred of the batch of API requests
    day				: The date of API initial request occurred of the batch of API requests
    time			: The time of API initial request occurred of the batch of API requests
  • API_DESTINATION_SUMMARY

    This table contains the summarized data of the API destinations and is derived from the destination event stream.

    API_DESTINATION_SUMMARY table schema
      CREATE TEMPORARY TABLE API_DESTINATION_SUMMARY_FINAL USING CarbonAnalytics OPTIONS (tableName "API_DESTINATION_SUMMARY",
      schema "api string -i,
      version string -i,
      apiPublisher string -i,
      context string -i,
      destination string -i,
      total_request_count int -i,
      hostName string -i,
      year int -i,
      month int -i,
      day int -i,
      time string -i,
      max_request_time long -i,
      api_version_context_dest_facet facet -i",
      primaryKeys "api,version,apiPublisher,context,destination,hostName,year,month,day"
      );
     Expand to find the table of descriptions for each column
    api					: API Name
    version				: API version
    apiPublisher		: API provider
    context				: API context depending on the user's request
    destination			: API endpoint hostname
    total_request_count	: Total request count for the requests coming for same API to same destination
    hostname			: APIM server hostname
    year				: The year of initial API request occurred of the batch of API requests
    month				: The month of initial API request occurred of the batch of API requests
    day					: The date of API initial request occurred of the batch of API requests
    time				: The time of API initial request occurred of the batch of API requests
  • API_LAST_ACCESS_TIME_SUMMARY

    This table contains the summary data of the last access times of the API and is derived from the request event stream.

    API_LAST_ACCESS_TIME_SUMMARY table schema
      CREATE TEMPORARY TABLE API_LAST_ACCESS_TIME_SUMMARY_FINAL USING CarbonAnalytics OPTIONS (tableName "API_LAST_ACCESS_TIME_SUMMARY",
      schema "tenantDomain string -i,
      apiPublisher string -i,
      api string -i,
      version string -i,
      userId string -i,
      context string -i,
      max_request_time long -i",
      primaryKeys "tenantDomain,apiPublisher,api"
      );
     Expand to find the table of descriptions for each column
    api				: API Name
    version			: API version
    apiPublisher	: API provider
    context			: API context depending on the user's request
    userId			: API invoked end user name
    tenantDomain	: Tenant domain of API provider
    max_request_time: Time of the latest API request occurrence
  • API_THROTTLED_OUT_SUMMARY

    This table contains the summary of the throttle out API invocation data. It is derived from the throttle out event table and request table.

    API_THROTTLED_OUT_SUMMARY table schema
    CREATE TEMPORARY TABLE THROTTLED_OUT_FINAL_SUMMARY USING CarbonAnalytics OPTIONS (tableName "API_THROTTLED_OUT_SUMMARY",
      schema "api string -i,
      api_version string -i,
      context string -i,
      apiPublisher string -i,
      applicationName string -i,
      tenantDomain string -i,
      year int -i,
      month int -i,
      day int -i,
      week int -i,
      time string -i,
      success_request_count int -i,
      throttleout_count int -i,
      max_request_time long -i,
      api_year_month_week_day_facet facet -i,
      applicationName_facet facet -i",
      primaryKeys "api,api_version,context,apiPublisher,applicationName,tenantDomain,year,month,day"
    );
     Expand to find the table of descriptions for each column
    api					: API Name
    api_version			: API synapse artifact contained name [API Provider +"--"+API Name]
    apiPublisher		: API provider
    context				: API context depending on the user's request
    applicationName		: Name of the client application
    tenantDomain		: Tenant domain of API provider
    throttleout_count	: Total throttled out API request count for the particular API
    success_request_count: Total successful API request count for the particular API
    year				: The year of initial API request occurred of the batch of API requests
    month				: The month of initial API request occurred of the batch of API requests
    day					: The date of API initial request occurred of the batch of API requests
    time				: The time of API initial request occurred of the batch of API requests

API Manager statistics

API statistics are provided in both the API Publisher and the API Store. Apart from the number of subscriptions per API, all other statistical dashboards require an instance of WSO2 Data Analytics Server installed. For information on the available statistics and how to view them, see Viewing API Statistics.

  • No labels