All docs This doc

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Publishing the messages to the Kafka brokers

The kafkaproduce-operation The publishMessages operation sends the messages to brokers using the topic.

Code Block
languagexml
titlekafkaproduce-operationpublishMessages
<kafkaTransport.kafkaproduce-operation>publishMessages>
	<topic>{$ctx:topic}</topic>
</kafkaTransport.kafkaproduce-operation>publishMessages>

Sample configuration

  • Download and install Apache Kafka. For more information, see Apache Kafka documentation
  • Copy the following client libraries from the <KAFKA_HOME>/lib directory to the <ESB_HOME>/repository/components/lib directory.

    • kafka_2.9.2-0.8.1.1.jar
    • scala-library-2.9.2.jar
    • zkclient-0.3.jar
    • zookeeper-3.3.4.jar
    • metrics-core-2.2.0.jar
  • Run the following command to start the ZooKeeper server:

    bin/zookeeper-server-start.sh config/zookeeper.properties

    You will see the following log:

  • Run the following command to start the Kafka server
    bin/kafka-server-start.sh config/server.properties
    You will see the following log:


  • Following is a sample proxy service that illustrates how to connect to Kakfa brokers with the init operation and use the kafkaproduce-operation the publishMessages operation to send publish the messages with the topic.

Code Block
languagexml
titleSample Proxy
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="KafkaTransport"
       transports="https http"
       startOnLoad="true"
       trace="disable">
   <description/>
   <target>
      <inSequence>
         <kafkaTransport.init>
            <brokerList>localhost:9092</brokerList>
         </kafkaTransport.init>
         <kafkaTransport.kafkaproduce-operation>publishMessages>
            <topic>test</topic>
         </kafkaTransport.kafkaproduce-operation>publishMessages>
      </inSequence>
   </target>
</proxy>            
  • The following image shows how to send the messages using a sample client(JMeter).

Analyzing the output

  • Run the following command to verify the messages
    bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning

...