This documentation is for WSO2 Identity Server 5.6.0. View documentation for the latest release.
Page Comparison - Setting Up Deployment Pattern 1 (v.3 vs v.4) - WSO2 Identity Server 5.6.0 - WSO2 Documentation

All docs This doc

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Multiexcerpt
MultiExcerptNameDeploymentSteps

Configuring the user store

WSO2 products allow you to configure multiple user stores to store your users and their roles. Your user store can be one of the following:

  • A Directory Service that can communicate over LDAP protocol like OpenLDAP

  • Active Directory

  • A database that can communicate over JDBC

  1. Set up a user store named WSO2UserStore.

    Tip

    Note: The instructions in this tutorial demonstrate configuring a JDBC user store. Point all cluster nodes to the same user store in the master-datasources.xml and user-mgt.xml file. 

    See Configuring User Stores for more information on how to set up other types of user stores.

  2. Copy the JDBC driver (in this case MySQL driver) to the <IS_HOME>/repository/component/lib directory of both nodes. To do this, download the MySQL Java connector JAR from here and place it in the <IS_HOME>/repository/components/lib directory.

  3. By default, WSO2 Identity Server is started with an embedded LDAP which comes with the product. Disable the embedded LDAP of node 2 by modifying embedded-ldap.xml which can be found in the <IS_HOME>/repository/conf/identity directory.

    Code Block
    languagehtml/xml
    <EmbeddedLDAP>
    	<Property name="enable">false</Property>
    <--------------------->
    <EmbeddedLDAP>

Configuring the datasources

  1. Create the databases. See  Setting up the Physical Database  in the WSO2 Administration Guide for db scripts and more information. 
    This tutorial demonstrates deployment with a user management database (WSO2UMDB ) and an identity database ( IDENTITYDB) .

    Tip

    Alternatively, you can create more databases for each type of data to separate the data logically. Note that this will NOT make a difference in performance and is not actually necessary.

    However, if you do wish to separate the data logically into separate databases, see the Setting Up Separate Databases for Clustering topic.

  2. Configure the datasource for the databases in both nodes of your cluster in the  master-datasources.xml   file found in the  <IS_HOME>/repository/conf/datasources   folder. 

    The code blocks below shows a sample configuration of the user management database and identity database for a mysql database.  For instructions on how to configure the datasource depending on the type of database you created, see  Changing the Carbon Database  in the WSO2 Product Administration Guide.

    Code Block
    languagexml
    titleSample configuration of master-datasources.xml
     <datasources>
        <datasource>
            <name>WSO2_CARBON_DB</name>
            <description>The datasource used for registry and user manager</description>
            <jndiConfig>
                <name>jdbc/WSO2CarbonDB</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:h2:./repository/database/WSO2CARBON_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000</url>
                    <username>wso2carbon</username>
                    <password>wso2carbon</password>
                    <driverClassName>org.h2.Driver</driverClassName>
                    <maxActive>50</maxActive>
                    <maxWait>60000</maxWait>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                    <defaultAutoCommit>false</defaultAutoCommit>
                </configuration>
            </definition>
        </datasource>
        <datasource>
            <name>WSO2UserStore</name>
            <description>The User Store</description>
            <jndiConfig>
                <name>jdbc/WSO2UserStore</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:mysql://wso2is-pattern1-mysql-service:3306/WSO2UMDB?autoReconnect=true&amp;useSSL=false</url>
                    <username>wso2carbon</username>
                    <password>wso2carbon</password>
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                    <maxActive>80</maxActive>
                    <maxWait>60000</maxWait>
                    <minIdle>5</minIdle>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                    <defaultAutoCommit>false</defaultAutoCommit>
                </configuration>
            </definition>
        </datasource>
        <datasource>
            <name>WSO2_IDENTITY_DB</name>
            <description>The datasource used for registry, user management and identity</description>
            <jndiConfig>
                <name>jdbc/WSO2IdentityDS</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:mysql://wso2is-pattern1-mysql-service:3306/WSO2_IDENTITY_DB?autoReconnect=true&amp;useSSL=false</url>
                    <username>wso2carbon</username>
                    <password>wso2carbon</password>
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                    <maxActive>80</maxActive>
                    <maxWait>60000</maxWait>
                    <minIdle>5</minIdle>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                    <defaultAutoCommit>false</defaultAutoCommit>
                </configuration>
            </definition>
        </datasource>
    </datasources>
    Code Block
    languagexml
    titleSample configuration of bps-datasources.xml
    <datasource>
        <name>BPS_DS</name>
        <description></description>
        <jndiConfig>
            <name>bpsds</name>
        </jndiConfig>
        <definition type="RDBMS">
            <configuration>
                <url>jdbc:mysql://wso2is-pattern1-mysql-service:3306/WSO2_IDENTITY_DB?autoReconnect=true&amp;verifyServerCertificate=false&amp;useSSL=true</url>
                <username>wso2carbon</username>
                <password>wso2carbon</password>
                <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                <maxActive>100</maxActive>
                <maxWait>10000</maxWait>
                <maxIdle>20</maxIdle>
                <testOnBorrow>true</testOnBorrow>
                <validationQuery>SELECT 1</validationQuery>
                <validationInterval>30000</validationInterval>
                <useDataSourceFactory>false</useDataSourceFactory>
                <defaultAutoCommit>false</defaultAutoCommit>
            </configuration>
        </definition>
    </datasource>

Mounting the registry

Mount the governance and configuration registry in the registry.xml file found in the <IS_HOME>/repository/conf folder to share the registry across all nodes in the cluster. The code block below shows a sample configuration.

For more information on mounting the registry, see Sharing Databases in a Cluster.

Code Block
languagexml
titleSample configuration of registry.xml
<dbConfig name="sharedregistry">
    <dataSource>jdbc/WSO2IdentityDS</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9443/registry">
    <id>sharedregistry</id>
    <dbConfig>sharedregistry</dbConfig>
    <readOnly>false</readOnly>
    <registryRoot>/</registryRoot>
    <enableCache>true</enableCache>
    <cacheId>jdbc:mysql://wso2is-pattern1-mysql-service:3306/WSO2_IDENTITY_DB</cacheId>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
    <instanceId>sharedregistry</instanceId>
    <targetPath>/_system/config</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
    <instanceId>sharedregistry</instanceId>
    <targetPath>/_system/governance</targetPath>
</mount>
Tip

Note: The production recommendation is to set the <versionResourcesOnChange> property in the registry.xml file to false. This is because automatic versioning of resources can be an extremely expensive operation.

Code Block
<versionResourcesOnChange>false</versionResourcesOnChange>

In the registry browser, verify that the governance collection is shown with the symlink icon.

    1. Log in to the management console.
    2. Navigate to Home > Registry > Browse.

Clustering Identity Server for high availability

Follow the instructions below to cluster WSO2 Identity Server.

  1. Install Identity Server on each node.
  2. Do the following changes to the <IS_HOME>/repository/conf/axis2/axis2.xml file for both nodes.

    1. Enable clustering on node 1 and node 2 by setting the clustering element to true:  
      <clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

    2. Specify the name of the cluster this node will join.
      <parameter name="domain">wso2.is.domain</parameter>

    3. Use the well knownaddress(WKA) based clustering method. In WKA-based clustering, we need to have a subset of cluster members configured in all the members of the cluster. At least one well knownmemberhas to be operational at all times.
      < parameter name = "membershipScheme" >wka</ parameter >

      Note

      WSO2 supports the following membership schemes as well.

      • Multicast membership scheme
      • AWS membership scheme
      • Kubernetes membership scheme

      For more information, see Clustering WSO2 Products - About Membership Schemes.

    4. Configure the localMemberHost and localMemberPort entries. These must be different port values for the two nodes if they are on the same server to prevent any conflicts.

      < parameter name = "localMemberHost" >127.0.0.1</ parameter >
      < parameter name = "localMemberPort" >4000</ parameter >
    5. Under the members section, add the hostName and port for each WKA member. As we have only two nodes in our sample cluster configuration, we will configure both nodes as WKA nodes.

      Code Block
      languagexml
      <members>
          <member>
            <hostName>127.0.0.1</hostName>
            <port>4000</port>
          </member>
          <member>
            <hostName>127.0.0.2</hostName>
            <port>4010</port>
          </member>
      </members>
      Note

      Note: You can also use IP address ranges for the hostName. For example,  192.168.1.2-10. This should ensure that the cluster eventually recovers after failures. One shortcoming of doing this is that you can define a range only for the last portion of the IP address. You should also keep in mind that the smaller the range, the faster the time it takes to discover members since each node has to scan a lesser number of potential members.

  3. Configure caching.

    Note

    From WSO2 Identity Server 5.2.0 onwards, distributed caching is disabled and it is not recommended to use this due to many practical issues that are related to configuring and running distributed caching properly. WSO2 Identity Server employs Hazelcast as the primary method of implementing cluster messages while using distributed caching in a simple setup.

    Info
    titleAbout Clustering

    For information on clustering, see Clustering WSO2 Products.

    Info
    titleAbout Caching
    • Why caching

      Caching is an additional layer on top of databases. It enables to keep the recently used data that are fetched from the database in local memory, so that for subsequent data requests instead of fetching from the database the data can be served from the local memory. Caching has certain advantages and disadvantages that you need to evaluate when deciding on your caching strategy.

    • Advantages
      • The load on the underlying database or LDAP is reduced as data is served from already fetched data in memory.

      • Improved performance due to the reduced number of database calls for repetitive data fetching.

    • Disadvantages
      • Coherency problems may occur when the data change is not immediately reflected on cached data if one node or an external system updates the database.

      • Data in memory can become stale yet be served, e.g., serving data from memory while its corresponding record in the database is deleted.

    Tip
    titleCaching in WSO2 Identity Server

    Historically WSO2 Identity Server used distributed caching to utilize the above-mentioned advantages as well as to minimize the coherence problem. However, in newer deployment patterns where the network is not tightly controlled, distributed caching fail in unexpected ways. Hence, we no longer recommend using distributed caching. Instead, it is recommended to have local caches (if required) and cache invalidation messages (if required) by considering the information given below.

    • The ForceLocalCache property

      When Hazelcast clustering is enabled certain caches act as distributed caches. The ForceLocalCache property within the <cache> section in the carbon.xml file in the <IS_HOME>/repository/conf directory is there to mark that all the caches should act like local caches even in a clustered setup. (This is by default set to true)

      Code Block
      <ForceLocalCache>true</ForceLocalCache>

      Cache invalidation uses Hazelcast messaging to distribute the invalidation message over the cluster and invalidate the caches properly.  This is used to minimize the coherence problem in a multi-node setup.

    • Typical clustered deployment cache scenarios

      ScenarioLocal CachingDistributed CachingHazelcast ClusteringDistributed InvalidationDescription
      1. All caches are local with distributed cache invalidationEnabledNot ApplicableEnabledEnabled
      • This is the recommended approach.

      • Hazelcast messaging invalidates the caches.

      2. All caches are local without distributed cache invalidationEnabledNot ApplicableDisabledDisabled
      • Invalidation clears only the caches in specific nodes. Other caches are cleared at cache expiration. 

      • Hazelcast communication is not used.

      • As the decisions take time to propagate over nodes (default cache timeout is 15 minutes), there is a security risk in this method. To reduce the risk, reduce the default cache timeout period. To learn how to reduce the default cache timeout period, see Configuring Cache Layers - timeout.

      3. No cachingDisabledDisabledDisabledDisabled
      • The data are directly acquired from the database. 

      • Eliminates the security risks caused due to not having cache invalidation.

      • This method will create a performance degradation due to the lack of caching.

      4. Certain caches are disabled while the remaining are localEnabled for the available local cachesNot ApplicableEnabledEnabled
      • To reduce the security risk created in the second scenario and to improve performance in comparison with the third scenario, disable the security-related caches and sustain the performance-related caches as local caches. 

      • This requires identification of these caches depending on the use case.

      5. Distributed caching enabledDisabled—the ForceLocalCache property is set to false.EnabledEnabledNot Applicable
      • This scenario is only recommended if the network has tight tolerance where the network infrastructure is capable of handling high bandwidth with very low latency. 

      • Typically this applies only when you deploy all the nodes in a single server rack having fiber-optic cables. In any other environments, this implementation will cause cache losses. Thus, this implementation is not recommended for general use.

  4. Configure the following.
    1. Change the datasource name to  jdbc/WSO2UMDB  in  user-mgt.xml  (located in  <IS_HOME>/repository/conf/ ). This refers to the user store you configured in the  Configuring the user store  section above.  

      Code Block
      languagehtml/xml
      titleuser-mgt.xml
      <UserManager>
        <Realm>
        <Configuration>
        ...
        <Property name="dataSource">jdbc/WSO2CarbonDB</Property>
        </Configuration>
        ...
        </Realm>
      </UserManager>
    2. Change the datasource name to jdbc/WSO2IDENTITYDB in identity.xml (located in <IS_HOME>/repository/conf/identity) of both node1 and node2. This refers to the datasource Configuring the datasources section above.

      Code Block
      languagehtml/xml
      titleidentity.xml
      <JDBCPersistenceManager>
         	 <DataSource>
         		<Name>jdbc/WSO2IDENTITYDB</Name>
         	 </DataSource>
         	    	 <!-- <SkipDBSchemaCreation>false</SkipDBSchemaCreation> -->
      </JDBCPersistenceManager>





Changing hostnames and ports

Configure the Identity Server node 1 using the following steps.

  1. Go to the <IS_HOME>/repository/conf/tomcat/catalina-server.xml file and add the proxy port as 443.

    Code Block
    languagexml
    <Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
              port="9443"
              proxyPort="443"
              ........
    
    <!--
            optional attributes:
    
            proxyPort="80"
    -->
    
    <Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
              port="9763"
              proxyPort="80"
    Tip

    Tip: If you are using an Openshift Docker container for the deployment, do the following.

    Add the following Tomcat RemoteIPValve to the <IS_HOME>/repository/conf/tomcat/catalina-server.xml file.

    Code Block
    <Valve
      className="org.apache.catalina.valves.RemoteIpValve"
      internalProxies="reg_ex_for_internal_docker_IPs"
      remoteIpHeader="x-forwarded-for"
      proxiesHeader="x-forwarded-by"
      protocolHeader="x-forwarded-proto"
    />
  2. In the <IS_HOME>/repository/conf/carbon.xml directory, define the hostname for your server.

    Code Block
    languagehtml/xml
    <HostName>wso2.is.com</HostName>
    
    <MgtHostName>wso2.is.com</MgtHostName>

    This hostname is used by the IS cluster. It must be specified in the /etc/hosts file as:

    127.0.0.1   wso2.is.com

Follow all the configuration steps that were done in node 1 for node 2 as well. 


Enabling artifact synchronization

To enable synchronization for runtime artifacts you must have a shared file system. You can use one of the following depending on your environment.

  • Network File System (NFS): This is one of the most commonly known shared file system and can be used in a linux environment.
  • Server Message Block (SMB) file system: This can be used in a Windows environment.
  • Amazon EFS: This can be used in an AWS environment.
  1. Once you have chosen a file system, mount it in the nodes that are participating in the cluster.
  2. Next, create two directories called "Deployment" and "Tenants" in the shared file system.
  3. Create a symlink from the <IS_HOME>/repository/deployment path to the "Deployment" directory of the shared file system that you created in step 2 of this section.
  4. Create a symlink from the <IS_HOME>/repository/tenants path to the "Tenants" directory of the shared file system that you created in step 2 of this section.

    Info

    Instead of mounting the file system directly to the <IS_HOME>/repository/deployment and <IS_HOME>/repository/tenants paths, a symlink is created to avoid issues that may occur if you delete the product to redeploy it; in which case the file system would get mounted to a non existing path.


Setting up the dashboard

Follow the steps given below to set up the dashboard for the WSO2 Identity Server in a clustered environment.

  1. Change the service provider configuration for the dashboard in the < IS_HOME>/repository/conf/identity/sso-idp-config.xml file.

    Code Block
    <AssertionConsumerServiceURLs><AssertionConsumerServiceURL>https://is.wso2.com/dashboard/acs</AssertionConsumerServiceURL></AssertionConsumerServiceURLs>
    <DefaultAssertionConsumerServiceURL>https://is.wso2.com/dashboard/acs</DefaultAssertionConsumerServiceURL>
  2. Configure the  proxyHost and proxyHTTPSPort in the  <IS_HOME>/repository/deployment/server/jaggeryapps/dashboard/conf/site.json file with your IP or hostname and the port.

    Code Block
    "proxyHost" : "is.wso2.com",
    "proxyHTTPSPort" : "443",
  3. Configure the  proxyHost and proxyHTTPSPort in the  <WSO2IS_HOME>/repository/deployment/server/webapps/shindig/WEB-INF/web.xml file with your IP or hostname and the port.

    Code Block
    shindig.host=is.wso2.com
    shindig.port=443
  4. Optionally, configure the <IS_HOME>repository/conf/datasources/master-datasources.xml file to set up the user dashboard.

    Expand
    titleClick here to view a sample datasource configuration.
    Code Block
    <datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration">
       <providers>
          <provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider>
       </providers>
       <datasources>
          <datasource>
             <name>USER_DB</name>
             <description>The datasource used for users and authorization management</description>
             <jndiConfig>
                <name>jdbc/UserDB</name>
             </jndiConfig>
             <definition type="RDBMS">
                <configuration>
                   <url>jdbc:mysql://localhost:3306/USER_DB</url>
                   <username>root</username>
                   <password>root</password>
                   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                   <maxActive>50</maxActive>
                   <maxWait>60000</maxWait>
                   <testOnBorrow>true</testOnBorrow>
                   <validationQuery>SELECT 1</validationQuery>
                   <validationInterval>30000</validationInterval>
                </configuration>
             </definition>
          </datasource>
          <datasource>
             <name>IDENTITY_DB</name>
             <description>The datasource used for WSO2 Identity Server specific data management</description>
             <jndiConfig>
                <name>jdbc/IdentityDB</name>
             </jndiConfig>
             <definition type="RDBMS">
                <configuration>
                   <url>jdbc:mysql://localhost:3306/IDENTITY_DB</url>
                   <username>root</username>
                   <password>root</password>
                   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                   <maxActive>50</maxActive>
                   <maxWait>60000</maxWait>
                   <testOnBorrow>true</testOnBorrow>
                   <validationQuery>SELECT 1</validationQuery>
                   <validationInterval>30000</validationInterval>
                </configuration>
             </definition>
          </datasource>
          <datasource>
             <name>REG_DB</name>
             <description>The datasource used for registry- config/governance</description>
             <jndiConfig>
                <name>jdbc/RegistryDB</name>
             </jndiConfig>
             <definition type="RDBMS">
                <configuration>
                   <url>jdbc:mysql://localhost:3306/REG_DB?autoReconnect=true</url>
                   <username>root</username>
                   <password>root</password>
                   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                   <maxActive>50</maxActive>
                   <maxWait>60000</maxWait>
                   <testOnBorrow>true</testOnBorrow>
                   <validationQuery>SELECT 1</validationQuery>
                   <validationInterval>30000</validationInterval>
                </configuration>
             </definition>
          </datasource>
          <datasource>
             <name>LOCAL_REG_DB_1</name>
             <description>The datasource used for local registry</description>
             <jndiConfig>
                <name>jdbc/WSO2CarbonDB</name>
             </jndiConfig>
             <definition type="RDBMS">
                <configuration>
                   <url>jdbc:h2:repository/database/WSO2CARBON_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000</url>
                   <username>wso2carbon</username>
                   <password>wso2carbon</password>
                   <driverClassName>org.h2.Driver</driverClassName>
                   <maxActive>50</maxActive>
                   <maxWait>60000</maxWait>
                   <testOnBorrow>true</testOnBorrow>
                   <validationQuery>SELECT 1</validationQuery>
                   <validationInterval>30000</validationInterval>
                   <defaultAutoCommit>false</defaultAutoCommit>
                </configuration>
             </definition>
          </datasource>
       </datasources>
    </datasources-configuration>

Fronting with a load balancer (Nginx)

If you need to set up the above WSO2 Identity Server cluster with Nginx, you can follow the instructions given below (you must do this after setting up the cluster following the above instructions). When clustering WSO2 Identity Server with a load balancer, make sure to enable sticky sessions. This is required for the management console and the dashboard to work and if we disable temporary session data persistence in the <IS_HOME>/repository/conf/identity/identity.xml file. 

Tip
titleSticky sessions for SSO

Sticky sessions are required to ensure a flawless SSO workflow when temporary session data persistence is disabled. It is recommended to use sticky sessions for SSO in order to have a higher throughput.

For more information on sticky sessions, see Sticky Sessions with Manager Nodes. The following is the deployment diagram with the load balancer.

Configuring Nginx

Use the following steps to configure NGINX Plus version 1.7.11 or nginx community version 1.9.2 as the load balancer for WSO2 products. (In these steps, we refer to both versions collectively as "Nginx".)

  1. Install Nginx (NGINX Plus or nginx community) in a server configured in your cluster.
  2. Configure Nginx to direct the HTTP requests to the two worker nodes via the HTTP 80 port using the  http://is.wso2.com/<service>. To do this, create a VHost file (is.http.conf) in the /etc/nginx/conf.d directory and add the following configurations into it.

    Note: Shown below is a general Nginx configuration. Click this link for more specific configuration with exposing various endpoints:

    Expand
    titleNginx configuration with exposing /oauth2, /commonauth, and other endpoints
    Code Block
    titleNginx configuration with exposing /oauth2, /commonauth, and other endpoints
    upstream ssl.nginx.com {
    	server z.z.z.z:9443;  
     	server x.x.x.x:9yyy  
      ip_hash; 
    }
    
    server {
    	listen 443;
    	server_name nginx.mycomp.org;   
    	ssl on;
    	ssl_certificate /home/abc/mycomp_org.crt; 
    	ssl_certificate_key /home/abc/mycomporg.key;
    
    	location /oauth2/token {
     		proxy_set_header X-Forwarded-Host $host;
    		proxy_set_header X-Forwarded-Server $host;
    		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host;
    		proxy_read_timeout 5m;
    		proxy_send_timeout 5m;
    		
    		proxy_pass  https://ssl.nginx.com/oauth2/token ;
    		proxy_redirect https://z.z.z.z:9443/oauth2/token https://nginx.mycomp.org/oauth2/token ;
    		proxy_redirect https://server x.x.x.x:9yyy/oauth2/token https://nginx.mycomp.org/oauth2/token ; 
    	}
    
    	location /commonauth {
    		proxy_set_header X-Forwarded-Host $host;
    		proxy_set_header X-Forwarded-Server $host;
    		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host;
    		proxy_read_timeout 5m;
    		proxy_send_timeout 5m;
    		proxy_pass https://ssl.nginx.com/commonauth;
    		proxy_redirect https://z.z.z.z:9443/commonauth https://nginx.mycomp.org/commonauth ;
    		proxy_redirect https://server x.x.x.x:9yyy/commomnauth https://nginx.mycomp.org/commonauth;
    	}
    
    	location /oauth2/authorize {
    		proxy_set_header X-Forwarded-Host $host;
    		proxy_set_header X-Forwarded-Server $host;
    		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host;
    		proxy_read_timeout 5m;
    		proxy_send_timeout 5m;
    		proxy_pass https://ssl.nginx.com/oauth2/authorize;
    		proxy_redirect https://z.z.z.z:9443/oauth2/authorize https://nginx.mycomp.org/oauth2/authorize ;
    		proxy_redirect https://server x.x.x.x:9yyy/oauth2/authorize https://nginx.mycomp.org/oauth2/ authorize;
    	}
    
    	location /authenticationendpoint/ {
    		proxy_set_header X-Forwarded-Host $host;
    		proxy_set_header X-Forwarded-Server $host;
    		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host;
    		proxy_read_timeout 5m;
    		proxy_send_timeout 5m;
    		proxy_pass https://ssl.nginx.com/authenticationendpoint/;
    		proxy_redirect https://z.z.z.z:9443/authenticationendpoint/ https://nginx.mycomp.org/authenticationendpoint/ ;
    		proxy_redirect https://server x.x.x.x:9yyy/authenticationendpoint https://nginx.mycomp.org/ authenticationendpoint;
    	}
    
    	location /oauth2/userinfo {
    		proxy_set_header X-Forwarded-Host $host;
    		proxy_set_header X-Forwarded-Server $host;
    		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host;
    		proxy_read_timeout 5m;
    		proxy_send_timeout 5m;
    		proxy_pass https://ssl.nginx.com/oauth2/userinfo;
    		proxy_redirect https://z.z.z.z:9443/oauth2/userinfo https://nginx.mycomp.org/oauth2/userinfo ;
    		proxy_redirect https://server x.x.x.x:9yyy/oauth2/userinfo https://nginx.mycomp.org/oauth2/ userinfo;
    	}
    }
    Code Block
    titleHTTP configurations
    upstream wso2.is.com {
            server xxx.xxx.xxx.xx3:9763;
            server xxx.xxx.xxx.xx4:9763;
    }
    
    server {
            listen 80;
            server_name is.wso2.com;
            location / {
                   proxy_set_header X-Forwarded-Host $host;
                   proxy_set_header X-Forwarded-Server $host;
                   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                   proxy_set_header Host $http_host;
                   proxy_read_timeout 5m;
                   proxy_send_timeout 5m;
                   proxy_pass http://wso2.is.com;
     
    			   proxy_http_version 1.1;
            	   proxy_set_header Upgrade $http_upgrade;
            	   proxy_set_header Connection "upgrade";
            }
    }
  3. Now that you've configured HTTP requests, you must also configure HTTPS requests. Configure Nginx to direct the HTTPS requests to the two worker nodes via the HTTPS 443 port using  https://is.wso2.com/<service>. To do this, create a VHost file (is.https.conf) in the /etc/nginx/conf.d directory and add the following configurations into it.

    Note

    Note: The configurations for nginx community version and NGINX Plus are different here since the community version does not support the sticky directive.

    Localtab Group
    Localtab
    activetrue
    titlenginx Community Version
    Code Block
    upstream ssl.wso2.is.com {
        server xxx.xxx.xxx.xx3:9443;
        server xxx.xxx.xxx.xx4:9443;
        ip_hash;
    }
     
    server {
    listen 443;
        server_name is.wso2.com;
        ssl on;
        ssl_certificate /etc/nginx/ssl/wrk.crt;
        ssl_certificate_key /etc/nginx/ssl/wrk.key;
        location / {
                   proxy_set_header X-Forwarded-Host $host;
                   proxy_set_header X-Forwarded-Server $host;
                   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                   proxy_set_header Host $http_host;
                   proxy_read_timeout 5m;
                   proxy_send_timeout 5m;
                   proxy_pass https://ssl.wso2.is.com;
     
                   proxy_http_version 1.1;
                   proxy_set_header Upgrade $http_upgrade;
                   proxy_set_header Connection "upgrade";
            }
    }
    
    
    Localtab
    titleNGINX Plus
    Code Block
    upstream ssl.wso2.is.com {
    	server xxx.xxx.xxx.xx3:9443;
    	server xxx.xxx.xxx.xx4:9443;
     
    			sticky learn create=$upstream_cookie_jsessionid
    			lookup=$cookie_jsessionid
    			zone=client_sessions:1m;
    }
    
    server {
    listen 443;
    	server_name is.wso2.com;
    	ssl on;
    	ssl_certificate /etc/nginx/ssl/wrk.crt;
    	ssl_certificate_key /etc/nginx/ssl/wrk.key;
    	location / {
                   proxy_set_header X-Forwarded-Host $host;
                   proxy_set_header X-Forwarded-Server $host;
                   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                   proxy_set_header Host $http_host;
                   proxy_read_timeout 5m;
                   proxy_send_timeout 5m;
    			   proxy_pass https://ssl.wso2.is.com;
     
    			   proxy_http_version 1.1;
    			   proxy_set_header Upgrade $http_upgrade;
    			   proxy_set_header Connection "upgrade";
        	}
    }
  4. Configure Nginx to access the Management Console as  https://mgt.is.wso2.com/carbon  via HTTPS 443 port. This is to direct requests to the manager node. To do this, create a VHost file ( mgt.is.https.conf) in the /etc/nginx/conf.d directory and add the following configurations into it.

    Code Block
    titleManagement Console configurations
    server {
    	listen 443;
    	server_name mgt.is.wso2.com;
    	ssl on;
    	ssl_certificate /etc/nginx/ssl/mgt.crt;
    	ssl_certificate_key /etc/nginx/ssl/mgt.key;
    
    	location / {
                   proxy_set_header X-Forwarded-Host $host;
                   proxy_set_header X-Forwarded-Server $host;
                   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                   proxy_set_header Host $http_host;
                   proxy_read_timeout 5m;
                   proxy_send_timeout 5m;
    			   proxy_pass https://xxx.xxx.xxx.xx2:9443/;
     
    			   proxy_http_version 1.1;
    			   proxy_set_header Upgrade $http_upgrade;
    			   proxy_set_header Connection "upgrade";
        	}
    	error_log  /var/log/nginx/mgt-error.log ;
               access_log  /var/log/nginx/mgt-access.log;
    }
  5. Reload the Nginx server.

    Code Block
    $sudo service nginx reload
    Tip

    If you have made modifications to anything other than the VHost files, you may need to restart the Nginx server instead of reloading:  

    Code Block
    $sudo service nginx restart

Create SSL certificates

Create SSL certificates for both the manager and worker nodes using the instructions that follow:

  1. Create the server key.
    $sudo openssl genrsa -des3 -out server.key 1024
  2. Create the certificate signing request.
    $sudo openssl req -new -key server.key -out server.csr
  3. Remove the password.
    $sudo cp server.key server.key.org
    $sudo openssl rsa -in server.key.org -out server.key
  4. Sign your SSL certificate.
    $sudo openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
  5. Execute the following command to import the created certificate file to the client truststore:

    Code Block
    keytool -import -trustcacerts -alias server -file server.crt -keystore client-truststore.jks

While creating keys, enter the host name ( is.wso2.com or mgt.is.wso2.com ) as the common name.

Configure the Proxy Port in IS Nodes

By default, WSO2 Identity Server runs on 9443 port. The following steps describe how you can configure a proxy port of 443.

  1. Open <IS_HOME>/repository/conf/tomcat/catalina-server.xml file and add the proxy port 443 in https connector as follows.

    Code Block
    languagexml
    <Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
    	port="9443"
    	proxyPort="443" 	
    Note

    It is not possible to configure proxy port from load balancer itself since there is a post request while authenticating to IS Dashboard. So, If you are planning to use Identity server Dashboard, you must do this configuration. Below configurations are also needed if you are using the dashboard. 

  2. Configure proxy port and host in <IS_HOME>/repository/deployment/server/jaggeryapps/dashboard/conf/site.json file as follows:

    Code Block
    languagexml
    {
      "proxy":{
     	"proxyHost":"nginx.mycomp.org" 
      	"proxyHTTPSPort":"443", 
      	"proxyContextPath":"", 
      	"servicePath":"/services"
      }		
    }	
  3. Configure proxy port and host in <IS_HOME>/repository/deployment/server/jaggeryapps/portal/conf/site.json file as follows:

    Code Block
    languagexml
    {
      "proxy":{
     	"proxyHost":"nginx.mycomp.org" 
      	"proxyHTTPSPort":"443", 
      	"proxyContextPath":"" 
      },
      "fido":{
      	"appId":""
      }		
    }
  4.  Configure proxy port and host in <IS_HOME>/repository/deployment/server/webapps/shindig/WEB-INF/web.xml 

    Code Block
    languagexml
    <context-param>
    	<param-name> system.properties </param-name>
    	<param-value>
      		<![CDATA[
     	shindig.host= 
     	shindig.port=443
     	aKey=/shindig/gadgets/proxy?container=default&url=
     	]]>
Note

For instructions on adding a custom proxy path to your server, see Adding a Custom Proxy Path.



Running the cluster

  1. Start Nginx and the Identity Server nodes.
  2. Now you can access the management console using the following URL: https://wso2.is.com/carbon/


Starting up and verifying product nodes

  1. If both nodes will be running on the same server, set the port offset to avoid port conflicts. 

  2. Start the nodes using the following command on both nodes.

    Localtab Group
    Localtab
    activetrue
    titleLinux/Unix
    Code Block
    sh wso2server.sh
    Localtab
    titleWindows
    Code Block
    .\wso2server.bat