Unknown macro: {search-box}
Child pages
  • Start In-Built Cassandra and ZooKeeper Servers for all Broker Nodes

This documentation is for WSO2 Message Broker version 2.1.0. View documentation for the latest release.

Skip to end of metadata
Go to start of metadata

The deployment model of this scenario is as follows.

cluster scenario 04

Figure: In-Built Cassandra and ZooKeeper Servers for all Broker Nodes.

MB ServerIP Address
MB Server 01192.168.0.100
MB Server 02192.168.0.101
MB Server 03192.168.0.102

Configure Cassandra Server

1. First, configure the HostName in $CARBON_HOME/repository/conf/carbon.xml file by specifying the ip of the host as follows. Configure this in all three servers.

<!-- Host name or IP address of the machine hosting this server e.g. www.wso2.org, 192.168.1.10.
This is will become part of the End Point Reference of the services deployed on this server instance.
-->

<HostName>192.168.0.100</HostName>

2. Configure Cassandra servers in these MB servers to form a cassandra ring, by changing the following entries in $CARBON_HOME/repository/conf/etc/Cassandra.yaml.

 # seeds is actually a comma-delimited list of addresses.
              # Ex: "<ip1>,<ip2>,<ip3>"
              - seeds: "192.168.0.101,192.168.0.102"
  • Change listen address of the server, which is "localhost" by default. Change it to the ip address of the host where you have the cassandra server. You need to change the listen address of each server to it's ip address. In this example, the server 192.168.0.102 is used as follows.
listen_address: 192.168.0.102
  • Change the RPC address of the server, which is "localhost" by default. Change it to the ip address of the host where you have the cassandra server. You need to change the RPC address of each server to it's ip address. For example,
rpc_address: 192.168.0.103

Configure Zookeeper Cluster

3. Next, configure the ZooKeeper Servers to create a ring among all broker nodes, by adding client port address to $CARBON_HOME/repository/conf/etc/zoo.cfg. Note that we are disabling starting the Zookeeper server here by setting 'start_zk_server=false' as the server will be started internally by Message Broker at the broker start up.

After modifying, the file looks as follows.

tickTime=2000
dataDir=repository/data/zookeeper
clientPort=2181
start_zk_server=false
clientPortAddress=192.168.0.101
server.1=192.168.0.100:2888:3888
server.2=192.168.0.101:2888:3888
server.3=192.168.0.102:2888:3888

4. The above zoo.cfg file is for server 192.168.0.101. Copy the same zoo.cfg file to other two servers and change the entry "clientPortAddress" to the ip of that particular server.

5. Create file "myid" in the data directory (repository/data/zookeeper) of each server and specify the id of the server in that file. For example, if the Id of the the server 192.168.0.100 is "1", have number 1 in that myid file.

    For more information, refer to the zookeeper clustering setup guide: http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_zkMulitServerSetup

6. You also need to setup "Zookeeper Server To Accept SASL Connections" for each Zookeeper server using the guide here 

7. Repeat the above configuration in each message broker server appropriately.

Configure Broker Nodes

Do the following configuration changes in each Message Broker node.

7. Modify $CARBON_HOME/repository/conf/advanced/qpid-config.xml file as follows.

  • Enable clustering.
  • Change the server of the Zookeeper.
<clustering>

          <enabled>true</enabled>
          <OnceInOrderSupportEnabled>false</OnceInOrderSupportEnabled>
          <externalCassandraServerRequired>false</externalCassandraServerRequired>

          <coordination>
              <!-- Apache Zookeeper Address -->
              <ZooKeeperConnection>192.168.0.100:2181,192.168.0.101:2181,192.168.0.102:2181</ZooKeeperConnection>
              <!-- Format yyyy-MM-dd HH:mm:ss -->
              <ReferenceTime>2012-02-29 08:08:08</ReferenceTime>
          </coordination>
...

8.Then modify $CARBON_HOME/repository/conf/advanced/qpid-virtualhosts.xml file as follows.

  • Change the Cassandra storage address with IP of the particular node.
<store>
      <class>org.wso2.andes.server.store.CassandraMessageStore</class>
      <username>admin</username>
      <password>admin</password>
      <cluster>ClusterOne</cluster>
      <idGenerator>org.wso2.andes.server.cluster.coordination.TimeStampBasedMessageIdGenerator</idGenerator>
      <connectionString>192.168.0.100:9160</connectionString>

9. Start all three Message Broker nodes. 

When starting the Message Broker use the following command. This will disable ZooKeeper from registering Log4j MBeans to manage log4j through JMX.

     sh wso2server.sh  -Dzookeeper.jmx.log4j.disable=true


  • No labels