This documentation is for WSO2 Message Broker 3.0.0. View documentation for the latest release.
Due to a known issue do not use JDK1.8.0_151 with WSO2 products. Use JDK 1.8.0_144 until JDK 1.8.0_162-ea is released.
Skip to end of metadata
Go to start of metadata

The instructions on this page takes you through the steps for upgrading from MB 2.2.0 to MB 3.0.0. Note that you cannot rollback the upgrade process. However, it is possible to restore a backup of the previous database and restart the upgrade progress.

Preparing to upgrade

The following prerequisites must be completed before upgrading:

  • Make a backup of the databases used for MB 2.2.0.

  • Also, copy the <MB_HOME_2.2.0> directory in order to backup the product configurations.

  • Download WSO2 Message Broker 3.0.0 from http://wso2.com/products/message-broker/.

NOTE: The downtime is limited to the time taken for switching databases when in the production environment.

Step 1: Setting up new databases

In a typical setup for WSO2 MB, you will have at least two DBMSs configured: One for user management and registry data (Carbon database) and another for MB-specific data. Follow the steps given below.

  1. First create a new database for user management and registry data in MB 3.0.0. See the section on setting up the Carbon database for more information.
  2. Restore the old user management and registry data in the new database. 
  3. Create another new database for MB-specific data in MB 3.0.0. See the section on setting up the MB-specific database for more information.
  4. Restore the old MB-specific data in the new database.

You will now have two new databases for MB 3.0.0 with the old data that existed for MB 2.2.0.

If there are multiple databases set up for MB 2.2.0, such as separate databases for multiple user stores etc., you need to create new databases for each of them.

Step 2: Updating the configurations

Once you have updated the databases as explained in step 1, you must update the configuration files in MB 3.0.0 to point to the new databases. The following are the updates that need to be done to the configuration files:

Note that configuration files should not be copied directly between servers.

  1. To connect MB 3.0.0 to the new database for user management and registry data, update the following configuration files:
    1. Configure the <MB_HOME_3.0.0>/repository/conf/datasources/master­datasources.xml file. This configuration should point to the new database that you set up in the previous section. See the following example:

      <datasource>
      user manager</description>
      <name>WSO2_CARBON_DB</name>
      <description>The datasource used for registry and
      <jndiConfig>
      <name>jdbc/WSO2CarbonDB</name>
      </jndiConfig>
      <definition type="RDBMS">
      <configuration>
      <url>jdbc:mysql://localhost:3306/<new_database></url>
      <username>username</username>
      <password>password</password>
      <driverClassName>com.mysql.jdbc.Driver</driverClassName>
      <maxActive>80</maxActive>
      <maxWait>60000</maxWait>
      <minIdle>5</minIdle>
      <testOnBorrow>true</testOnBorrow>
      <validationQuery>SELECT1</validationQuery>
      <validationInterval>30000</validationInterval>
      </configuration>
      </definition>
      </datasource>
    2. Go to the <MB_HOME_3.0.0>/repository/conf/ directory and update the datasource references in the user-­mgt.xml and registry.xml files to match the updated configurations in the master­datasources.xml file. The following are sample configurations with the datasource set to “jdbc/WSO2CarbonDB”:

      registry.xml 

      <dbConfig name="wso2registry">
      <dataSource>jdbc/WSO2CarbonDB</dataSource>
      </dbConfig>

      user­mgt.xml

      <UserManager>
      <Realm>
      <Configuration>
      ...
      <Property
      name="dataSource">jdbc/WSO2CarbonDB</Property>
      </Configuration>
      ...
      </Realm>
      </UserManager>
  2. To connect MB 3.0.0 to the new database for MB-specific data, configure the following files:
    1. Configure the <MB_HOME_3.0.0>/repository/conf/datasources/master­datasources.xml file. This configuration should point to the MB-specific database that you set up in the previous section. See the following example:

      <datasource>
        <name>WSO2_MB_STORE_DB</name>
        <jndiConfig>
           <name>WSO2MBStoreDB</name>
        </jndiConfig>
           <definition type="RDBMS">
               <configuration>
                  <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                  <url>jdbc:mysql://localhost/wso2_mb</url>
                  <username>root</username>
                  <password>root</password>
                  <maxActive>50</maxActive>
                  <maxWait>60000</maxWait>
                  <minIdle>5</minIdle>
                  <testOnBorrow>true</testOnBorrow>
                  <validationQuery>SELECT 1</validationQuery>
                  <validationInterval>30000</validationInterval>
                  <defaultAutoCommit>false</defaultAutoCommit>
               </configuration>
            </definition>
      </datasource>
    2. Go to the broker.xml file (stored in the <MB_HOME_3.0.0>/repository/conf folder) and update the persistence information related to MB-specific data. See the following example:

      <messageStore class="org.wso2.andes.store.rdbms.RDBMSMessageStoreImpl">
          <property name="dataSource">WSO2MBStoreDB</property>
          <property name="storeUnavailableSQLStateClasses">08</property>
          <property name="integrityViolationSQLStateClasses">23,27,44</property>
          <property name="dataErrorSQLStateClasses">21,22</property>
          <property name="transactionRollbackSQLStateClasses">40</property>
      </messageStore>
      <contextStore class="org.wso2.andes.store.rdbms.RDBMSAndesContextStoreImpl">
          <property name="dataSource">WSO2MBStoreDB</property>
          <property name="storeUnavailableSQLStateClasses">08</property>
          <property name="integrityViolationSQLStateClasses">23,27,44</property>
          <property name="dataErrorSQLStateClasses">21,22</property>
          <property name="transactionRollbackSQLStateClasses">40</property>
      </contextStore>
  3. Check for any other configurations that were done for MB 2.2.0 (based on your solutions), and update the configuration files in MB 3.0.0 accordingly. For example, external user stores, caching, mounting, etc. 
    • The following configuration is introduced with the Carbon 4.4.2 version and is applicable to all WSO2 products that are based on Carbon 4.4.2:

      Configuration File 
      The user-mgt.xml file stored in the <PRODUCT_HOME>/repository/conf directory.

      The <isCascadeDeleteEnabled> property enables cascade delete for records entered in the UM_USER_PERMISSION and UM_ROLE_PERMISSION tables of the database used for storing user permissions.

    • The following configuration files are new in MB 3.0.0:
      • broker.xml file (stored in the <MB_HOME_3.0.0>/repository/conf folder)
      • metrics.xml file (stored in the <MB_HOME_3.0.0>/repository/conf folder)
      • metrics.properties file (stored in the <MB_HOME_3.0.0>/repository/conf folder)
      • metrics-datasources.xml file (stored in the <MB_HOME_3.0.0>/repository/conf/datasources folder)

Step 3: Start the new server and create new Topics and Queues

You now have your new MB 3.0.0 set up and configured. 

  1. Proceed to start the server.
  2. Log in to the Management Console of MB 3.0.0.
  3. Manually create all the Topics and Queues that were there in MB 2.2.0. See the sections on managing Queues and managing Topics for detailed instructions on how to add Topics and Queues.

Step 4: Migrating the messages from Topics and Queues

Once you have configured the new MB 3.0.0 server as explained in the previous steps, you have to move all the Topics and Queues stored in the MB 2.2.0 instance to the new MB 3.0.0 instance. This data can be migrated by using a separate JMS client. This new JMS client will subscribe to the Queues and Topics in MB 2.2.0 and publish all the messages from those Queues and Topics to MB 3.0.0. The following diagram illustrates this process:
JMS client for data migration 

Given below is a sample JMS client that can be used for the above purpose. Click on the relevant tab to see how the subscriber and publisher classes are implemented in this sample JMS client. When you develop your own JMS client, you will have to modify the implementation given in this sample as applicable to your system. For example,

The Subscriber.java class should be modified as follows:

  • The connection details of your MB 2.2.0 installation should be specified.
  • Details of the Queues and Topics you have in MB 2.2.0 should be specified.

The Publisher.java class should be modified as follows:

  • The connection details of your MB 3.0.0 installation should be specified.
  • Details of the Queues and Topics to which messages will be published should be appropriately specified.

This is the sample implementation of a Subscriber client, which will subscribe to Queues and Topics in an MB 2.2.0 installation.

import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.lmax.disruptor.BlockingWaitStrategy;
import com.lmax.disruptor.RingBuffer;
import com.lmax.disruptor.dsl.Disruptor;
import com.lmax.disruptor.dsl.ProducerType;
import com.lmax.disruptor.util.DaemonThreadFactory;
import org.wso2.andes.thread.DefaultThreadFactory;
import javax.jms.*;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import java.util.Properties;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadFactory;

public class Subscriber extends Thread{
    public static final String QPID_ICF = "org.wso2.andes.jndi.PropertiesFileInitialContextFactory";
    private static final String CF_NAME_PREFIX = "connectionfactory.";
    private static final String CF_NAME = "qpidConnectionfactory";
    String userName = "admin";
    String password = "admin";
    private static String CARBON_CLIENT_ID = "carbon";
    private static String CARBON_VIRTUAL_HOST_NAME = "carbon";
    private static String CARBON_DEFAULT_HOSTNAME = "localhost";
    private static String CARBON_DEFAULT_PORT = "5672";
    String queueName = "testQueue";
    private QueueConnection queueConnection;
    private QueueSession queueSession;
    private MessageConsumer consumer;
    private Disruptor<MessagingEvent> disruptor;
    ExecutorService executorPool;
    private static final int EXECUTOR_POOL_SHUTDOWN_WAIT_TIME = 10;
    public Subscriber() throws JMSException, NamingException {
        ThreadFactory namedThreadFactory = new ThreadFactoryBuilder()
                .setNameFormat("DisruptorMessagingEventThread-%d").build();
        executorPool = Executors.newCachedThreadPool(namedThreadFactory);
        int bufferSize = 4096;
        disruptor = new Disruptor<>(MessagingEvent.getFactory(), bufferSize, executorPool, ProducerType.SINGLE,
                                    new BlockingWaitStrategy());
        disruptor.handleEventsWith(new Publisher());
        disruptor.start();
        registerSubscriber();
    }
    public void shutDown() throws JMSException {
        // Housekeeping
        consumer.close();
        queueSession.close();
        queueConnection.stop();
        queueConnection.close();
    }
    public void registerSubscriber() throws NamingException, JMSException
    {
        Properties properties = new Properties();
        properties.put(Context.INITIAL_CONTEXT_FACTORY, QPID_ICF);
        properties.put(CF_NAME_PREFIX + CF_NAME, getTCPConnectionURL(userName, password));
        properties.put("queue."+ queueName,queueName);
        InitialContext ctx = new InitialContext(properties);
        // Lookup connection factory
        QueueConnectionFactory connFactory = (QueueConnectionFactory) ctx.lookup(CF_NAME);
        queueConnection = connFactory.createQueueConnection();
        queueConnection.start();
        queueSession =
                queueConnection.createQueueSession(false, QueueSession.AUTO_ACKNOWLEDGE);
        //Receive message
        Queue queue =  (Queue) ctx.lookup(queueName);
        consumer = queueSession.createConsumer(queue);
    }
    public void receiveMessages() throws NamingException, JMSException {
        TextMessage message = (TextMessage) consumer.receive();
        System.out.println("Got message from queue receiver==>" + message.getText());
        publishToDisrutptor(message);
    }
    public void publishToDisrutptor(TextMessage message){
        RingBuffer<MessagingEvent> ringBuffer = disruptor.getRingBuffer();
        long sequence = ringBuffer.next();
        MessagingEvent evt = ringBuffer.get(sequence);
        evt.setMessage(message);
        ringBuffer.publish(sequence);
        System.out.println("Published message to ring buffer");
    }
    public void run(){
        while (true){
            try {
                receiveMessages();
            } catch (NamingException e) {
                e.printStackTrace();
                break;
            } catch (JMSException e) {
                e.printStackTrace();
                break;
            }
        }
    }
    private String getTCPConnectionURL(String username, String password) {
        // amqp://{username}:{password}@carbon/carbon?brokerlist='tcp://{hostname}:{port}'
        return new StringBuffer()
                .append("amqp://").append(username).append(":").append(password)
                .append("@").append(CARBON_CLIENT_ID)
                .append("/").append(CARBON_VIRTUAL_HOST_NAME)
                .append("?brokerlist='tcp://").append(CARBON_DEFAULT_HOSTNAME).append(":").append(CARBON_DEFAULT_PORT).append("'")
                .toString();
    }
}

This is the sample implementation of a publisher client, which will publish to the Queues and Topics in an MB 3.0.0 installation.

import com.lmax.disruptor.EventHandler;
import javax.jms.*;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import java.util.Properties;

public class Publisher implements EventHandler<MessagingEvent>{
    @Override
    public void onEvent(MessagingEvent messagingEvent, long l, boolean b) throws Exception {
        System.out.println("Handling message receipt for the disruptor ");
        sendMessages(messagingEvent.message);
    }
    public static final String QPID_ICF = "org.wso2.andes.jndi.PropertiesFileInitialContextFactory";
    private static final String CF_NAME_PREFIX = "connectionfactory.";
    private static final String QUEUE_NAME_PREFIX = "queue.";
    private static final String CF_NAME = "qpidConnectionfactory";
    String userName = "admin";
    String password = "admin";
    private static String CARBON_CLIENT_ID = "carbon";
    private static String CARBON_VIRTUAL_HOST_NAME = "carbon";
    private static String CARBON_DEFAULT_HOSTNAME = "localhost";
    private static String CARBON_DEFAULT_PORT = "5673";
    String queueName = "testQueue";
    private QueueConnection queueConnection;
    private QueueSession queueSession;
    public void sendMessages(TextMessage message) throws NamingException, JMSException {
        Properties properties = new Properties();
        properties.put(Context.INITIAL_CONTEXT_FACTORY, QPID_ICF);
        properties.put(CF_NAME_PREFIX + CF_NAME, getTCPConnectionURL(userName, password));
        properties.put(QUEUE_NAME_PREFIX + queueName, queueName);
        InitialContext ctx = new InitialContext(properties);
        // Lookup connection factory
        QueueConnectionFactory connFactory = (QueueConnectionFactory) ctx.lookup(CF_NAME);
        queueConnection = connFactory.createQueueConnection();
        queueConnection.start();
        queueSession = queueConnection.createQueueSession(false, QueueSession.AUTO_ACKNOWLEDGE);
        // Send message
        Queue queue = (Queue)ctx.lookup(queueName);
        // create the message to send
        javax.jms.QueueSender queueSender = queueSession.createSender(queue);
        queueSender.send(message);
        queueSender.close();
        queueSession.close();
        queueConnection.close();
    }
    private String getTCPConnectionURL(String username, String password) {
        // amqp://{username}:{password}@carbon/carbon?brokerlist='tcp://{hostname}:{port}'
        return new StringBuffer()
                .append("amqp://").append(username).append(":").append(password)
                .append("@").append(CARBON_CLIENT_ID)
                .append("/").append(CARBON_VIRTUAL_HOST_NAME)
                .append("?brokerlist='tcp://").append(CARBON_DEFAULT_HOSTNAME).append(":").append(CARBON_DEFAULT_PORT).append("'")
                .toString();
    }
}
import com.lmax.disruptor.EventFactory;
import javax.jms.TextMessage;

public class MessagingEvent {
    TextMessage message;
    public void setMessage(TextMessage aTextMessage){
        message = aTextMessage;
    }
    public static class PublishEventFactory implements EventFactory<MessagingEvent> {
        @Override
        public MessagingEvent newInstance() {
            return new MessagingEvent();
        }
    }
    public static EventFactory<MessagingEvent> getFactory() {
        return new PublishEventFactory();
    }
}
import javax.jms.JMSException;
import javax.naming.NamingException;

public class Main {
    public static void main(String[] args) throws NamingException, JMSException {
        Thread subscriber = new Thread(new Subscriber());
        subscriber.start();
    }
}

Follow the steps given below.

Before you begin:

  • Make sure that the publishers to MB 2.2.0 are stopped before data migration starts.
  • Make sure that all the configuration files are updated in MB 3.0.0.
  • Data migration should only take place after all the users, roles, queues, topics etc. are created in MB 3.0.0.
  1. Both MB 2.2.0 and MB 3.0.0 should be started.

    Be sure to use a port offset for one of the two product installations using the carbon.xml file (stored in the <MB_HOME>/repository/conf folder).

  2. Start the JMS Client. The following events will be executed:
    • The JMS Subscriber will receive all the messages from the Queues and Topics in MB 2.2.0.
    • The JMS Publisher will publish the messages (that were received from MB 2.2.0) to MB 3.0.0.

This concludes the upgrade from MB 2.2.0 to MB 3.0.0.

Step 5: Testing the upgrade

  1. Verify that all the required scenarios are working as expected as shown below. This confirms that the upgrade is successful.
    1. Make sure that the server starts up fine without any errors.

    2. Verify that the Users and Roles are picked up:

      1. Navigate to Configure -> Accounts & Credentials -> Users and Roles

      2. Verify that the list of users and roles are shown correctly.

      3. View the permissions of a chosen role, and make sure that the permissions are correct.
  2. Verify that all the messages to the Queues and Topics have been successfully published in the MB 3.0.0 instance.
  3. Run some of the samples to see that the system is working fine.
  • No labels