Complex Event Processor 3.0.0
Expand all   Collapse all
Child pages
  • Input Kafka Event Adapter

This documentation is for WSO2 CEP 3.0.0. View the home page of the latest release.

                                                                                                                                                                                                                                                                                                                                                                                                       
||
Skip to end of metadata
Go to start of metadata

The Apache Kafka input adapter reads the tail of a given file and inputs that to the WSO2 product engine. This feature is donated by Andres Gomez Ferrer. For more information, go to Apache Kafka documentation.

Prerequisites

Set up the below prerequisites to start configuring an input Apache Kafka event adapter.

  1. Download Apache Kafka server. 

  2. Copy the following client JAR files from <KAFKA_HOME>/lib/ directory  to <CEP_HOME>/repository/components/lib/ directory.
    • kafka_2.10-0.8.1.jar
    • zkclient-0.3.jar
    • scala-library-2.10.1.jar
    • zookeeper-3.3.4.jar
  3. Copy all the JAR files, which are located in <KAFKA_HOME>/lib/ directory to <CEP_HOME>/samples/lib/ directory.

Configuring input Kafka event adapter

You can configure input Kafka event adapter using the management console or using a configuration file.

Configure using the management console

Follow the instructions below to configure input Kafka event adapter using the management console. This deploys the event adapter in the <PRODUCT_HOME>/

repository/deployment/server/inputeventadaptors/ directory.

  1. Log in to product's management console, and click Configure.
  2. Click Input Event Adaptors in the Event Processor Configs menu, and then click Add Input Event Adaptor.

  3. Enter details as shown in the below example. 
    create input Kafka event adapter

  4.  Click Add Event Adaptor.

Configure using a configuration file

Follow the instructions below to configure input Kafka event adapter using a configuration file.

  1. Create an XML file with the following Kafka input event adapter configurations. Input event adapter implementation must start with <inputEventAdaptor> as the root element.

    <inputEventAdaptor name="kafkaInputAdaptor" statistics="disable"
  trace="disable" type="kafka" xmlns="http://wso2.org/carbon/eventadaptormanager">
  <property name="optional.configuration">zk.connectiontimeout.ms:6000,zk.sessiontimeout.ms:7000</property>
  <property name="group.id">test-group</property>
  <property name="zookeeper.id">127.0.0.1:2181</property>
  <property name="threads">5</property>
</inputEventAdaptor>

  2. Add the XML file to the  <PRODUCT_HOME>/repository/deployment/server/inputeventadaptors/ directory. Since hot deployment is enabled, you can simply add/remove files to deploy/undeploy to/from the server.  
     

After an adapter is successfully added, it gets added to the list of adapters displayed under Event Processor Configs in the Configure menu of the product's management console. Click Edit to change its configuration and redeploy it. This opens an XML-based editor allowing you to edit the event adapter configurations from the UI.  Do your modifications and click Update. You can also delete it, enable/disable statistics or enable/disable tracing on it using the provided options in the UI.

Configuring input Kafka event adapter message properties

The configured input event adapters are used in Event Builders. With input event adapter message properties, event builders are able to extract data from the incoming events to build the event streams. Message properties specific to input event adapters of the kafka type can be configured as follows.

  1. Log into the CEP Management Console. For detailed instructions, see Running the Product.
  2. Click the Configure tab and then click Event Builders to open the Available Event Builders page.

  3. Click the name of the event builder to which the relevant input event adapter is connected. This opens the Event Builder Details page.

    input Kafka event adapter message propertiesThe message properties of a kafka input event adapter are described below.

    PropertyDescription
    Topic KafkaName of the Kafka topic to which, input messages are published
    Input Mapping TypeThe incoming event type to be mapped
  • No labels
||

 
Copyright © WSO2 Inc. 2005-2015