This documentation is for WSO2 Enterprise Integrator version 6.2.0 . View documentation for the latest release in the 6.x.x family and the latest release in the 7.x.x family.

All docs This doc

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

All profiles of WSO2 EI uses a database to store information such as user management details and registry data. All nodes in the cluster must use one central database for config and governance registry mounts. You can create the following databases and associated datasources.

Database NameDescription
WSO2_USER_DB
JDBC user store and authorization manager.

REGISTRY_DB

Shared database for config and governance registry mounts in the product's nodes.
REGISTRY_LOCAL1Local registry space in Node 1.
REGISTRY_LOCAL2Local registry space in Node 2.

In addition to the databases listed above, the MB profile of WSO2 EI requires the following broker-specific data:

Database NameDescription
MB_DB
Stores instance data that are specific to the message broker profile.
Warning

It is recommended to use an industry-standard RDBMS such as Oracle, PostgreSQL, MySQL, MS SQL, etc. for most enterprise testing and production environments. However, you can also use the embedded H2 database only for the REGISTRY_LOCAL1 and REGISTRY_LOCAL2.

...

  1. Download and install the MySQL Server.

  2. Download the MySQL JDBC driver.

  3. Download and unzip the WSO2 EI binary distribution. Be sure to select the correct product version. 

    Tip

    Throughout this guide, <EI_HOME> refers to the extracted directory of the WSO2 EI product distribution.

  4. Unzip the downloaded MySQL driver, and copy the MySQL JDBC driver JAR (mysql-connector-java-x.x.xx-bin.jar) to the <EI_HOME>/lib/ directory of both the WSO2 EI nodes.
  5. Add the following line to the /etc/hosts file to define the hostname for configuring permissions for the new database: <MYSQL-DB-SERVER-IP> carbondb.mysql-wso2.com

    Note

    Do this step only if your database is not on your local machine and on a separate server.

  6. Execute the following command in a terminal/command window, where username is the username you want to use to access the databases: mysql -u username -p
  7. When prompted, specify the password to access the databases.

  8. Create the databases using the following commands:

    Code Block
    languagesql
    mysql> create database WSO2_USER_DB;
    mysql> use WSO2_USER_DB;
    mysql> source <EI_HOME>/dbscripts/mysql.sql;
    mysql> grant all on WSO2_USER_DB.* TO [email protected]"carbondb.mysql-wso2.com" identified by "regadmin";
    
    mysql> create database REGISTRY_DB;
    mysql> use REGISTRY_DB;
    mysql> source <EI_HOME>/dbscripts/mysql.sql;
    mysql> grant all on REGISTRY_DB.* TO [email protected]"carbondb.mysql-wso2.com" identified by "regadmin";
    
    mysql> create database REGISTRY_LOCAL1;
    mysql> use REGISTRY_LOCAL1;
    mysql> source <EI_HOME>/dbscripts/mysql.sql;
    mysql> grant all on REGISTRY_LOCAL1.* TO [email protected]"carbondb.mysql-wso2.com" identified by "regadmin";
     
    mysql> create database REGISTRY_LOCAL2;
    mysql> use REGISTRY_LOCAL2;
    mysql> source <EI_HOME>/dbscripts/mysql.sql;
    mysql> grant all on REGISTRY_LOCAL2.* TO [email protected]"carbondb.mysql-wso2.com" identified 
    by "regadmin";
    Info
    titleAbout using MySQL in different operating systems

    For users of Microsoft Windows, when creating the database in MySQL, it is important to specify the character set as latin1. Failure to do this may result in an error (error code: 1709) when starting your cluster. This error occurs in certain versions of MySQL (5.6.x) and is related to the UTF-8 encoding. MySQL originally used the latin1 character set by default, which stored characters in a 2-byte sequence. However, in recent versions, MySQL defaults to UTF-8 to be friendlier to international users. Hence, you must use latin1 as the character set as indicated below in the database creation commands to avoid this problem. Note that this may result in issues with non-latin characters (like Hebrew, Japanese, etc.). The following is how your database creation command should look.

    mysql> create database <DATABASE_NAME> character set latin1;

    For users of other operating systems, the standard database creation commands will suffice. For these operating systems, the following is how your database creation command should look.

    mysql> create database <DATABASE_NAME>;
  9. Create and configure the MB_DB database, which is specific to the MB profile:

    Tip

    <EI_HOME> , username, and password are the same as those you specified in the previous steps.

    Code Block
    languagesql
    mysql> create database wso2_mb;
    mysql> use wso2_mb;
    mysql> source <EI_HOME>/wso2/broker/dbscripts/mb-store/mysql-mb.sql;
    mysql> grant all on wso2_mb.* TO [email protected]"carbondb.mysql-wso2.com" identified by "regadmin";

...

  1. Configure the datasources to point to the REGISTRY_LOCAL1WSO2_REGISTRY_DB, and WSO2_USER_DB databases as follows in the <EI_HOME>/wso2/broker/conf/datasources/master-datasources.xml file as follows:

    Tip

    Replace the username, password, and database URL of your MySQL environment accordingly.

    Localtab Group
    Localtab
    activetrue
    titleMB Node 1

    For MB node 1, configure the datasources to point to the REGISTRY_LOCAL1WSO2_REGISTRY_DB, and WSO2_USER_DB databases as follows:

    Code Block
    languagexml
    <datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration"> 
         <providers> 
            <provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider> 
        </providers> 
        <datasources> 
            <datasource> 
                <name>REGISTRY_LOCAL1</name> 
                <description>The datasource used for registry- local</description> 
                <jndiConfig> 
                    <name>jdbc/WSO2CarbonDB</name> 
                </jndiConfig> 
                <definition type="RDBMS"> 
                    <configuration> 
                        <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_LOCAL1?autoReconnect=true</url> 
                        <username>regadmin</username> 
                        <password>regadmin</password> 
                        <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                        <maxActive>50</maxActive> 
                        <maxWait>60000</maxWait> 
                        <testOnBorrow>true</testOnBorrow> 
                        <validationQuery>SELECT 1</validationQuery> 
                        <validationInterval>30000</validationInterval> 
                    </configuration> 
                </definition> 
            </datasource> 
            <datasource> 
                <name>REGISTRY_DB</name> 
                <description>The datasource used for registry- config/governance</description> 
                <jndiConfig> 
                    <name>jdbc/WSO2RegistryDB</name> 
                </jndiConfig> 
                <definition type="RDBMS"> 
                    <configuration> 
                        <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_DB?autoReconnect=true</url> 
                        <username>regadmin</username> 
                        <password>regadmin</password> 
                        <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                        <maxActive>50</maxActive> 
                        <maxWait>60000</maxWait> 
                        <testOnBorrow>true</testOnBorrow> 
                        <validationQuery>SELECT 1</validationQuery> 
                        <validationInterval>30000</validationInterval> 
                    </configuration> 
                </definition> 
            </datasource> 
             <datasource> 
                <name>WSO2_USER_DB</name> 
                <description>The datasource used for registry and user manager</description> 
                <jndiConfig> 
                    <name>jdbc/WSO2UMDB</name> 
                </jndiConfig> 
                <definition type="RDBMS"> 
                    <configuration> 
                        <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/WSO2_USER_DB</url> 
                        <username>regadmin</username> 
                        <password>regadmin</password> 
                        <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                        <maxActive>50</maxActive> 
                        <maxWait>60000</maxWait> 
                        <testOnBorrow>true</testOnBorrow> 
                        <validationQuery>SELECT 1</validationQuery> 
                        <validationInterval>30000</validationInterval> 
                    </configuration> 
                </definition> 
            </datasource> 
       </datasources> 
    </datasources-configuration>
    Localtab
    titleMB Node 2

    For node 2, configure the datasources to point to the REGISTRY_LOCAL2WSO2_REGISTRY_DB, and WSO2_USER_DB  databases as shown below. Change the username, password, and database URL as needed for your environment.

    Code Block
    languagexml
    <datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration"> 
         <providers> 
            <provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider> 
        </providers> 
        <datasources> 
            <datasource> 
                <name>REGISTRY_LOCAL2</name> 
                <description>The datasource used for registry- local</description> 
                <jndiConfig> 
                    <name>jdbc/WSO2CarbonDB</name> 
                </jndiConfig> 
                <definition type="RDBMS"> 
                    <configuration> 
                        <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_LOCAL1?autoReconnect=true</url> 
                        <username>regadmin</username> 
                        <password>regadmin</password> 
                        <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                        <maxActive>50</maxActive> 
                        <maxWait>60000</maxWait> 
                        <testOnBorrow>true</testOnBorrow> 
                        <validationQuery>SELECT 1</validationQuery> 
                        <validationInterval>30000</validationInterval> 
                    </configuration> 
                </definition> 
            </datasource> 
            <datasource> 
                <name>REGISTRY_DB</name> 
                <description>The datasource used for registry- config/governance</description> 
                <jndiConfig> 
                    <name>jdbc/WSO2RegistryDB</name> 
                </jndiConfig> 
                <definition type="RDBMS"> 
                    <configuration> 
                        <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_DB?autoReconnect=true</url> 
                        <username>regadmin</username> 
                        <password>regadmin</password> 
                        <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                        <maxActive>50</maxActive> 
                        <maxWait>60000</maxWait> 
                        <testOnBorrow>true</testOnBorrow> 
                        <validationQuery>SELECT 1</validationQuery> 
                        <validationInterval>30000</validationInterval> 
                    </configuration> 
                </definition> 
            </datasource> 
             <datasource> 
                <name>WSO2_USER_DB</name> 
                <description>The datasource used for registry and user manager</description> 
                <jndiConfig> 
                    <name>jdbc/WSO2UMDB</name> 
                </jndiConfig> 
                <definition type="RDBMS"> 
                    <configuration> 
                        <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/WSO2_USER_DB</url> 
                        <username>regadmin</username> 
                        <password>regadmin</password> 
                        <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                        <maxActive>50</maxActive> 
                        <maxWait>60000</maxWait> 
                        <testOnBorrow>true</testOnBorrow> 
                        <validationQuery>SELECT 1</validationQuery> 
                        <validationInterval>30000</validationInterval> 
                    </configuration> 
                </definition> 
            </datasource> 
       </datasources> 
    </datasources-configuration>
  2. Remove or comment out the default H2-based WSO2_MB_STORE_DB and WSO2_CARBON_DB datasource configurations from the <EI_HOME>/wso2/broker/conf/datasources/master-datasources.xml file. 

  3. Uncomment or add the following WSO2_MB_STORE_DB configuration in the <EI_HOME>/wso2/broker/conf/datasources/master-datasources.xml file based on your DBMS type.

    Tip

    Update the JDBC URL to correctly point to your database and enter the username and password for your database user with the proper permissions.

    Localtab Group
    Localtab
    titleConfiguring MSSQL
    Code Block
    languagexml
    <datasource>
        <name>WSO2_MB_STORE_DB</name>
        <jndiConfig>
            <name>jdbc/WSO2MBStoreDB</name>
        </jndiConfig>
        <definition type="RDBMS">
            <configuration>
                <defaultAutoCommit>false</defaultAutoCommit>
                <dataSourceClassName>com.microsoft.sqlserver.jdbc.SQLServerXADataSource</dataSourceClassName>
                <dataSourceProps>
                    <property name="URL">jdbc:sqlserver://127.0.0.1\SQLExpress</property>
                    <property name="databaseName">wso2mb</property>
                    <property name="user">sa</property>
                    <property name="password">mssql</property>
                </dataSourceProps>
            </configuration>
        </definition>
    </datasource>
    Localtab
    titleConfiguring MySQL
    Note

    If your MySQL username or password is not root, make sure to update the <username> and <password> properties.

    Code Block
    languagexml
    <datasource>
        <name>WSO2_MB_STORE_DB</name>
        <jndiConfig>
            <name>WSO2MBStoreDB</name>
        </jndiConfig>
        <definition type="RDBMS">
            <configuration>
                <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                <url>jdbc:mysql://localhost/wso2_mb</url>
                <username>regadmin</username>
                <password>regadmin</password>
                <maxActive>50</maxActive>
                <maxWait>60000</maxWait>
                <minIdle>5</minIdle>
                <testOnBorrow>true</testOnBorrow>
                <validationQuery>SELECT 1</validationQuery>
                <validationInterval>30000</validationInterval>
                <defaultAutoCommit>false</defaultAutoCommit>
            </configuration>
        </definition>
    </datasource>
    Localtab
    titleConfiguring Oracle
    Code Block
    languagexml
     <datasource>
        <name>WSO2_MB_STORE_DB</name>
        <jndiConfig>
            <name>WSO2MBStoreDB</name>
        </jndiConfig>
        <definition type="RDBMS">
            <configuration>
                <defaultAutoCommit>false</defaultAutoCommit>
                <driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
                <url>jdbc:oracle:thin:@localhost:1521/orcl</url>
                <maxActive>100</maxActive>
                <maxWait>60000</maxWait>
                <minIdle>5</minIdle>
                <testOnBorrow>true</testOnBorrow>
                <validationQuery>SELECT 1</validationQuery>
                <validationInterval>30000</validationInterval>
                <username>scott</username>
                <password>tiger</password>
            </configuration>
        </definition>
    </datasource>
  4. Add the following configuration in the <EI_HOME>/wso2/broker/conf/user-mgt.xml file of both nodes to configure the user stores. 

    Tip

    Enter the datasource information for the user store that you configured in the <EI_HOME>/wso2/broker/conf/datasources/master-datasources.xml file. You can change the admin username and password as well. However, you should do this before starting the server.

    Code Block
    languagexml
    <Configuration>
        <AddAdmin>true</AddAdmin>
        <AdminRole>admin</AdminRole>
        <AdminUser>
            <UserName>admin</UserName>
            <Password>admin</Password>
        </AdminUser>
        <EveryOneRoleName>everyone</EveryOneRoleName>
        <Property name="dataSource">jdbc/WSO2UMDB</Property>
    </Configuration>
  5. Uncomment or add the following configuration in the <EI_HOME>/wso2/broker/conf/broker.xml file.  Enter the message store based on your database type and Andes context store.

    Localtab Group
    Localtab
    titleConfiguring MSSQL
    Code Block
    ...
    <persistence>
      <messageStore class="org.wso2.andes.store.rdbms.RDBMSMessageStoreImpl">
            <propertyname="dataSource">WSO2MBStoreDB</property>
      </messageStore>
      <contextStore class="org.wso2.andes.store.rdbms.RDBMSAndesContextStoreImpl">
            <propertyname="dataSource">WSO2MBStoreDB</property>
      </contextStore>
    ...
    </persistence>
    Localtab
    titleConfiguring MySQL
    Code Block
    ...
    <persistence>
      <messageStore class="org.wso2.andes.store.rdbms.RDBMSMessageStoreImpl">
            <propertyname="dataSource">WSO2MBStoreDB</property>
      </messageStore>
      <contextStore class="org.wso2.andes.store.rdbms.RDBMSAndesContextStoreImpl">
            <propertyname="dataSource">WSO2MBStoreDB</property>
      </contextStore>
    ...
    </persistence>
    Localtab
    titleConfiguring Oracle
    Code Block
    ...
    <persistence>
      <messageStore class="org.wso2.andes.store.rdbms.RDBMSMessageStoreImpl">
            <propertyname="dataSource">WSO2MBStoreDB</property>
      </messageStore>
      <contextStore class="org.wso2.andes.store.rdbms.RDBMSAndesContextStoreImpl">
            <propertyname="dataSource">WSO2MBStoreDB</property>
      </contextStore>
    ...
    </persistence>
    
    
  6. Add the following thrift-related configurations in the <EI_HOME>/wso2/broker/conf/broker.xml file.

    Tip

    This file is the root configuration file of brokering. Do the changes you do to this file in all the broker profile nodes. Configure the thriftServerHost value to point to the IP address of the MB profile node.

    Code Block
    languagexml
    <coordination>
        <nodeID>default</nodeID>
        <thriftServerHost>192.168.0.103</thriftServerHost>
        <thriftServerPort>7611</thriftServerPort>
    	<thriftSOTimeout>0</thriftSOTimeout>
    </coordination>
    ConfigurationDescription
    coordination

    This configuration is related to brokering thrift communications.

    nodeIDIn a clustered deployment, an ID is assigned to each brokering node via the cluster node identifier. This element can be used to override the cluster node identifier for this brokering node. If the value for this element is left as default, the default node ID is generated using the IP and a universally unique identifier (UUID). The node ID of each member in a cluster must be unique.
    thriftServerHost

    This is a sub-element of the <coordination> tag. The MB profile uses Apache Thrift for communications relating to message delivery. Therefore, an Apache Thrift server is started in each brokering node in a clustered deployment. This element should point to the IP address of the Apache Thrift server. This should point to the IP address of the brokering node that hosts the thrift server. The default value for this is localhost. For example, if you are configuring a brokering node hosted in 192.168.0.102 as the thrift server, this value should be 192.168.0.102.

    thriftServerPort

    This is another sub-element of the <coordination> tag. This should point to the port of the thrift server in the MB profile.

    Info
    titleWhat is the effective thrift server port ?
    The broker.xml file indicates 7611 as the default thrift server port. However, the default thrift server port that is effective in the MB profile of WSO2 EI is 7614 (7611 + 3). This is because, by default, the server port of the MB profile of WSO2 EI is offset by 3 in the <EI_HOME>/wso2/broker/conf/carbon. xml file. It is important to note that the effective thrift server port depends upon the default server port offset in the carbon.xml file as well as the thriftServerPort value in the broker.xml file. Be sure to keep the correct thrift server port open in your system to avoid errors.

    It is recommended to use the same thrift server port for all broker nodes in your cluster.

    thriftSOTimeout

    This is used to handle half-open TCP connections between the broker nodes in a cluster. In such situations, the socket may need to have a timeout value to invalidate the connection (in milliseconds). A timeout of zero is interpreted as an infinite timeout. Be sure to set this value to 180000 milliseconds as shown below.

    Code Block
    <thriftSOTimeout>180000</thriftSOTimeout>
  7. Anchor
    enable_hazelcast
    enable_hazelcast
    Edit the <EI_HOME>/wso2/broker/conf/axis2/axis2.xml file as follows to set up the cluster configurations.

    1. Enable clustering for this node as follows: <clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
    2. Set the membership scheme to "wka" to enable the well-known address registration method. (This node sends cluster initiation messages to the WKA members): 
      <parameter name="membershipScheme">wka</parameter>
    3. Specify the name of the cluster this node will join as follows: 
      <parameter name="domain">wso2.ei.domain</parameter>
    4. Specify the hosts to communicate cluster messages as follows: 
      <parameter name="localMemberHost">xxx.xxx.xxx.xx2</parameter>

    5. Specify the port to communicate cluster messages as follows: 
      <parameter name="localMemberPort">4100</parameter>

      Info

      This port number is not affected by the port offset value specified in the <EI_HOME>/wso2/broker/conf/carbon. xml file. If this port number is already assigned to another server, the clustering framework automatically increments this port number. However, if there are two servers running on the same machine, ensure that a unique port is set for each server.

    6. Specify the well-known members as shown below. The port value for the WKA node must be the same value as it's localMemberPort (in this case it is 4100.

      Info

      You can also use IP address ranges for the hostname (e.g., 192.168.1.2-10). However, you can define a range only for the last portion of the IP address. Smaller the range, the faster the time it takes to discover members since each node has to scan a lesser number of potential members. The best practice is to add all the members (including itself) in all the nodes to avoid any conflicts in configurations.

      Code Block
      languagexml
      <members>    
          <member>
              <hostName>xxx.xxx.xxx.xx1</hostName>
              <port>4100</port>
          </member>
          <member>
              <hostName>xxx.xxx.xxx.xx2</hostName>
              <port>4200</port>
          </member>
      </members>
  8. Enable heartbeat messaging for each of the broker nodes by following the steps given below. This is required for handling TCP connections between the broker nodes and client systems (publishers and subscribers).

    Info
    titleWhat is heartbeat messaging?
    Expand
    titleClick here for more information.

    When a client is connected to the broker, both the broker and the client should be able to detect problem situations where the TCP connection is half open or where the connecting client/broker is unresponsive. This can be achieved by enabling heartbeat messaging between the broker and the client.

    The heartbeat messaging configuration allows both the broker and the client to verify whether the connection is inactive or whether the connecting system (broker or client) is inactive by periodically sending messages to each other.

    For example, you can set the heartbeat delay time to 30 seconds. This means that if the broker (or the client) does not receive the expected response from the other system within 30 seconds, it will send a heartbeat message to check if the connection is inactive or if the other system is inactive. If the connection is inactive or is half-open, there will be a clear indication of connection failure. However, if the connection is active but the other system is unresponsive, the broker or the client will continue to send heartbeat messages (every 30 seconds). By default, the broker and clients are configured to terminate (timeout) the connection after sending two heartbeat messages; however, this time out configuration can be changed.

    Note: In a scenario where the subscriber client is sending the heartbeat message to the broker node, if the connection is found to be broken or if the broker node is inactive, the connection will failover to another broker node in the cluster.

    1. Open the  qpid-config.xml file that is stored in the  <EI_HOME>/wso2/broker/conf/advanced directory.
    2. Set the heartbeat delay as shown below. The recommended heartbeat delay is 30 seconds.

      Code Block
      <heartbeat>
          <delay>30</delay>
          <timeoutFactor>2.0</timeoutFactor>
      </heartbeat>

      The elements in the above configuration are explained below.

      delayThe <delay> element specifies the time interval between heartbeat messages. This is the time that the broker or client will wait to receive a response from the other party. If the response is not received within this time, a heartbeat message is triggered.
      timeoutFactorThe number of heartbeat messages the broker will send to the client before terminating the connection. That is, if the timeoutFactor is 2, the broker will send heartbeat messages every 30 seconds twice, and if a response is not received from the client, the connection will be terminated.

Optional: Configuring Hazelcast properties

...

  • Cluster coordination parameters:

    ParameterDescription
    enabled

    This property is set to "true" by default, which indicates that RDBMS-based cluster coordination is enabled:

    Code Block
    <rdbmsBasedCoordination enabled="true">
    It is not recommended to disable this setting. If you disable this setting, the hazelcast engine will handle cluster coordinatin without using the RDBMS network, which can cause network partitioning. Find out more about how to handle network partitioning in hazelcast cluster coordination.
    heartbeatIntervalThe time interval for heartbeat messaging, which is used to check the availability of the nodes in the cluster. Declared in milliseconds.
    coordinatorEntryCreationWaitTimeThe time the system should wait before informing other nodes about a new coordinator. Declared in milliseconds. This value should be larger than the database read time (including network latency) and should be less than the heartbeatInterval value.
    eventPollingIntervalTime interval used to poll the database for membership related events. Declared in milliseconds.
  • Event synchronization parameters:

    ParameterDescription
    enabled

    This property is set to "true" by default, which indicates that RDBMS-based cluster event synchronization is enabled:

    Code Block
    <RDBMSBasedClusterEventSynchronization enabled="true">
    It is not recommended to disable this setting.
    eventSyncIntervalSpecifies the interval at which the cluster events will be read from the database. Declared in milliseconds. Setting this to a very low value could decrease server performance where as setting this to a large value could increase the time taken for a cluster event to be synchronized in all the nodes in a cluster.

Anchor
network_partitioning
network_partitioning
Handling network partitioning in Hazelcast-based clustering

...