||
Skip to end of metadata
Go to start of metadata

The requirements for deploying WSO2 products can change based on the deployment scenario and pattern used. The recommendations in this topic are for general production use, assuming moderate load conditions. For situations where a high volume of traffic is expected and larger deployments, these guidelines may not be sufficient. See Troubleshooting in Production Environments for information on how to obtain and analyze information to solve production issues. The following are the topics addressed in this section.



Installation prerequisites

Prior to installing any WSO2 Carbon-based product, it is necessary to have the appropriate hardware and software necessary for running the product.

Hardware requirements (per WSO2 product instance)

Physical
  • 3 GHz Dual-core Xeon/Opteron (or latest)
  • 4 GB RAM (2 GB for JVM and 2 GB for the operating system)
  • 10 GB free disk space

Disk space is based on the expected storage requirements that are calculated by considering the file uploads and the backup policies. For example, if three WSO2 product instances are running in a single machine, it requires a 4 GHz CPU, 8 GB RAM (2 GB for the operating system and 6 GB (2 GB for each WSO2 product instance)) and 30 GB of free space.

Virtual Machine
  • 2 compute units minimum (each unit having 1.0-1.2 GHz Opteron/Xeon processor)
  • 4 GB RAM
  • 10 GB free disk space
  • One CPU unit for the operating system and one for JVM.

Three WSO2 product instances running would require VM of 4 compute units, 8 GB RAM, and 30 GB free space.

EC2
  • 1 c3.large instance to run one WSO2 product instance.

Three WSO2 product instances can be run in 1 EC2 Extra-Large instance. Based on the I/O performance of the c3.large instance, it is recommended to run multiple instances in a larger instance (c3.xlarge or c3.2xlarge).

Cassandra data nodes
  • 4 core processors
  • 8 GB RAM

For more information, see the Cassandra documentation on hardware recommendations for enterprise implementations.

Environment compatibility

  • All WSO2 Carbon-based products are Java applications that can be run on any platform that is Oracle JDK 1.7/1.8 compliant. Also, OpenJDK is not recommended or supported. 
  • All WSO2 Carbon-based products are generally compatible with most common DBMSs. For more information, see Working with Databases.
  • It is not recommended to use Apache DS in a production environment due to issues with scalability. Instead, it is recommended to use an LDAP like OpenLDAP for user management.
  • On a production deployment, it is recommended that WSO2 products are installed on latest releases of RedHat Enterprise Linux or Ubuntu Server LTS.

  • For environments that WSO2 products are tested with, see Environments Tested with WSO2 Products. 
  • If you have difficulty in setting up any WSO2 product on a specific platform or with a database, please contact us.

Required applications

The following applications are required for running the WSO2 product and its samples or for building from the source code. Mandatory installs are marked with *.

Application

Purpose

Version

Download Links

Oracle Java SE Development Kit (JDK)*

Required by all WSO2 products to:
  • To launch the product as each product is a Java application.
  • To build the product from the source distribution (both JDK and Apache Maven are required).
  • To run Apache Ant.

JDK 1.7 or 1.8

  • Oracle and IBM JRE 1.7 are also supported when running (not building) WSO2 products.
  • If you are using JDK 1.7 on Mac OS or Solaris, install the snappy-java library using the following steps:

    1. Download the snappy-java JAR and extract it to a preferred location. This folder will be referred to as <SNAPPY_HOME>.

    2. Copy the appropriate snappy-java library file i386.jnilib (32bit) or x86_64.jnilib (64bit), which is in the <SNAPPY_HOME>/org/xerial/snappy/native/Mac/directory, to the <PRODUCT_HOME> directory.
    For more information on installing snappy-java library, see Snappy-java fails on Mac OS JDK 1.7.
http://java.sun.com/javase/downloads/index.jsp

You are now ready to install the WSO2 product.


Installing the WSO2 product

Before you beginplease see our compatibility matrix to find out if this version of the product is fully tested on your operating system. 

Do the following to install the WSO2 product.

  1. If you have not done so already, download the latest version of the WSO2 product from the product section of wso2.com.
  2. Extract the archive file to a dedicated directory for the WSO2 product. This will hereafter be referred to as <PRODUCT_HOME> in this guide.

Setting JAVA_HOME

You must set your JAVA_HOME environment variable to point to the directory where the Java Development Kit (JDK) is installed on the computer.

Environment variables are global system variables accessible by all the processes running under the operating system.

  1. In your home directory, open the BASHRC file (.bash_profile file
 on OS X) in a Linux text editor, such as vi, emacs, pico, or mcedit.
  2. Add the following two lines at the bottom of the file, replacing /usr/java/jdk1.7.0_80 with the actual directory where the JDK is installed.

    On Linux:
    export JAVA_HOME=/usr/java/jdk1.7.0_80
    export PATH=${JAVA_HOME}/bin:${PATH}
     
    On OS X:
    export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_80/Contents/Home
  3. Save the file.

    If you do not know how to work with text editors in a Linux SSH session, run the following command:

    cat >> .bashrc

    Paste the string from the clipboard and press "Ctrl+D".

  4. To verify that the JAVA_HOME variable is set correctly, execute the following command:

    On Linux:
    echo $JAVA_HOME
     
    On OS X:
    which java
     
    If the above command gives you a path like /usr/bin/java, then it is a symbolic link to the real location. To get the real location, run the following:
    ls -l `which java`

    The system returns the JDK installation path.

Setting system properties

If you need to set additional system properties when the server starts, you can take the following approaches:

  • Set the properties from a script. Setting your system properties in the startup script (i.e. the <PRODUCT_HOME>/bin/wso2server.sh file), is ideal because it ensures that you set the properties every time you start the server. To avoid having to modify the script each time you upgrade, the best approach is to create your own startup script that wraps the WSO2 startup script and adds the properties you want to set, rather than editing the WSO2 startup script directly.

    Be sure to set the org.wso2.ignoreHostnameVerification system property in the <PRODUCT_HOME>/bin/wso2server.sh file to false as follows:

    org.wso2.ignoreHostnameVerification=false 

    This setting will enable hostname verification of HTTP requests and responses in the Carbon server, and thereby avoid security issues in production environments.

  • Set the properties from an external registry. If you want to access properties from an external registry, you could create Java code that reads the properties at runtime from that registry. Be sure to store sensitive data such as username and password to connect to the registry in a properties file instead of in the Java code and secure the properties file with the secure vault.

Note: When using SUSE Linux, it ignores /etc/resolv.conf and only looks at the /etc/hosts file. This means that the server will throw an exception on startup if you have not specified anything besides localhost. To avoid this error, add the following line above 127.0.0.1 localhost in the /etc/hosts file: <ip_address>  <machine_name> localhost

You are now ready to run the product.


Running the product

To run WSO2 products, you start the product server at the command line. You can then run the Management Console application to configure and manage the product. 

Before you begin:

  • The config-validation.xml file in the <PRODUCT_HOME>/repository/conf/etc directory contains a list of recommended system parameters, which are validated against your system when the server starts. See Configuring config-validation.xml for details on modifying these parameters before starting the server.
  • The Management Console uses the default HTTP-NIO transport, which is configured in the catalina-server.xml file in the <PRODUCT_HOME>/repository/conf/tomcat directory. This transport must be properly configured in this file for the management console to be accessible.

Starting the server

To start the server, you run the script wso2server.sh (on Linux) from the bin folder.

To start and stop the server in the background mode of Linux, run wso2server.sh start and wso2server.sh stop commands.

  1. Open the command line on Linux and establish an SSH connection to the server or log into the text Linux console.
  2. Execute the following command, where <PRODUCT_HOME> is the directory where you installed the product distribution:

    sh <PRODUCT_HOME>/bin/wso2server.sh

    If you want to provide access to the production environment without allowing any user group (including admin) to log into the management console, execute the following command.
    sh <PRODUCT_HOME>/bin/wso2server.sh -DworkerNode

    If you want to check any additional options available to be used with the startup commands, type -help after the command, such as: sh <PRODUCT_HOME>/bin/wso2server.sh -help.

    Alternatively, you can restrict access to the management console of your product by binding the management console with selected IP addresses. Note that you can either restrict access to the management console only, or you can restrict access to all web applications in your server as explained below.

    • To control access only to the management console, add the IP addresses to the <PRODUCT_HOME>/repository/conf/tomcat/carbon/META-INF/context.xml file as follows:

      <Valve className="org.apache.catalina.valves.RemoteAddrValve" allow="<IP-address-01>|<IP-address-02>|<IP-address-03>"/>

      The RemoteAddrValve Tomcat valve defined in this file will only apply to the Carbon management console, and thereby all outside requests to the management console will be blocked. 

    • To control access to all web applications deployed in your server, add the IP addresses to the <PRODUCT_HOME>/repository/conf/context.xml file as follows:

      <Valve className="org.apache.catalina.valves.RemoteAddrValve" allow="<IP-address-01>|<IP-address-02>|<IP-address-03>"/>

      The RemoteAddrValve Tomcat valve defined in this file will apply to each web application hosted on the Carbon server. Therefore, all outside requests to any web application will be blocked.

    • You can also restrict access to particular servlets in a web application by adding a Remote Address Filter to the web.xml file (stored in the <PRODUCT_HOME>/repository/conf/tomcat/ directory), and by mapping that filter to the servlet URL. In the Remote Address Filter that you add, you can specify the IP addresses that should be allowed to access the servlet.


      The following example from a web.xml file illustrates how access to the management page (/carbon/admin/login.jsp) is granted only to one IP address:
      <filter>
          <filter-name>Remote Address Filter</filter-name>
          <filter-class>org.apache.catalina.filters.RemoteAddrFilter</filter-class>
              <init-param>
                  <param-name>allow</param-name>
                  <param-value>127.0.01</param-value>
              </init-param>
      </filter>
      
      <filter-mapping>
          <filter-name>Remote Address Filter</filter-name>
          <url-pattern>/carbon/admin/login.jsp</url-pattern>
      </filter-mapping>
    Note: Any configurations (including valves) defined in the <PRODUCT_HOME>/repository/conf/tomcat/catalina-server.xml file applies to all web applications and is globally available across server, regardless of host or cluster. See the official Tomcat documentation for more information about using remote host filters.

    The operation log appears. When the product server is running, the log displays the message "WSO2 Carbon started in 'n' seconds."

Stopping the server

To stop the server, press Ctrl+C in the command window, or click the Shutdown/Restart link in the navigation pane in the Management Console.

Tuning parameters for latency

The latency numbers (~50ms) are based on a two datacenter setup with a high-speed network connection. With the default configuration, you might notice intermittent behavior, so it is important to tune the system. The following tuning parameters can be used as a starting point for your system.

The following configurations must be placed in <PRODUCT_HOME>/repository/conf/hazelcast.properties file. Create this file if it does not exist.

hazelcast.shutdownhook.enabled=false
hazelcast.heartbeat.interval.seconds=1
hazelcast.master.confirmation.interval.seconds=5
hazelcast.max.no.heartbeat.seconds=45 
hazelcast.max.no.master.confirmation.seconds=60

Additionally, Hazelcast indicates that if all members are not mentioned in the well-known member list, there can be a split-brain (network partition) situation. If the cluster spans across data centers, it is important that all the members are added to the well-known members list that can be found in the <PRODUCT_HOME>/repository/conf/axis2/axis2.xml file.

Running recommendations for security

The following are security related recommendations to be followed when running the product.

  • Running as a different user: For security reasons, it's recommended to run the product as an unprivileged user. After adding a user to the system, apply your organizational security policies to that user.
  • Running on a different port: If you want to run on a different port, like port 80, the recommended way is to add a port forwarding rule from your firewall.
  • Running as a Unix daemon: You have the option of running each product as a standard Unix service. You can start, stop and restart the WSO2 product instances as follows.
    # sh bin/wso2server.sh [start | stop | restart]

Host machine security hardening

Integrate the newly added server to your organizational backup policy of critical services. This will help you to recover a critical system in case of a compromise. We recommend you apply all the critical security patches from the respective OS vendor. If you have RedHat Enterprise Linux installed you must have an RHN subscription to receive updates. Turning off all services that you're not using is also critical. You can further add a firewall rule to close all ports other than the ports the WSO2 product is running on.

The following links give some sample guidelines on server hardening:


Common guidelines and checklist

The following table lists out the common guidelines and details pertaining to them. These are common to all products and are followed for making an installed WSO2 product ready for production.

GuidelineDetails
Default credentials

Once you have installed the product, it is a good practice to change the default passwords. All WSO2 products have an administrator account set to “admin” as username and password. You can change it in the user management section of the Management Console.

Related links

See Changing a Password for more information on how to change the password of the administrator.

Secure Sockets Layer (SSL) keys

WSO2 products by default come with a self-signed SSL key. Since these keys are public, it is recommended to configure your own keys for security purposes.

Related links
JVM version

The recommended version is JDK 1.7 or 1.8.

 For JDK 1.7, set the appropriate Heap and Permgen values for the JVM based on your deployment scenario. These can be set in the <PRODUCT_HOME>/bin/wso2server.sh file. You do not need to set this in JDK 1.8 as the MaxPermSize value has been removed from Hotspot JVM.

For example
-Xms512m -Xmx2048m -XX:MaxPermSize=1024m

Tip: To run the JVM with 2 GB (-Xmx2048m), you should ideally have about 4GB of memory on the physical machine.

 Host name

By default, WSO2 products identify the host nameof the current machine through the Java API. However, this value sometimes yields erroneous results onsome environments. Therefore, users are recommended to configure the host name by setting the HostName parameter in the <PRODUCT_HOME>/repository/conf/carbon.xml file.

<HostName>your.host.name</HostName>

To configure host namesfor WSDLs and endpoints, users are recommended to add the following parameter in the <transportReceiver> section in the <PRODUCT_HOME>/repository/conf/axis2/axis2.xml file as shown below.

<parameter name="WSDLEPRPrefix" locked="false">[http]://your.host.name:[port]</parameter>
Related links
Managing logs

It is recommended to configure logging using the management console.

Related links

See Monitoring Carbon Products for details on how to configure logging details in WSO2 products.

Registry and governance

All WSO2 products make use of an instance of a registry to store configurations. The registry uses a database as the persistent storage. By default, the registry uses an embedded H2 database.

H2 is not recommended in production

The embedded H2 database is NOT recommended in enterprise testing and production environments. It has lower performance, clustering limitations, and can cause file corruption failures. Please use an industry-standard RDBMS such as Oracle, PostgreSQL, MySQL, or MS SQL instead.

You can use the embedded H2 database in development environments and as the local registry in a registry mount.

Moreover, it is worth noting that the default setup does not include database backup procedures. The production setup should obviously need to have regular database backup procedures configured.

When the registry database is pointed to a remote database, multiple running instances of the same product can boot up and run against the same configuration stored in the registry. This, in turn, helps with governance.

Related links

See here for more information on sharing a registry space across multiple WSO2 product instances.

User stores

WSO2 products offer three choices to store user details:

  • Using a database
  • Using an LDAP server
  • Using an Active Directory service

The default is to use the embedded H2 database with the user store schema. Like in the case of the registry database, H2 is not recommended in enterprise testing and production environments. We recommend that you switch to a database like Oracle, MySQL or MSSQL. You can point to an existing LDAP or an Active Directory to make use of existing user bases and grant access privileges for WSO2 products based on those userstores.

Related links

See Configuring User Stores for more information on userstores, how they work, and how to configure them.

Monitoring with JMX

WSO2 Products supportJMX for monitoring. By default, JMX uses port 9999. You can configure this to the desired port by setting the JMX port parameter in the <PRODUCT_HOME>/repository/conf/carbon.xml file.

<Ports>
	<JMX>9999</JMX>
</Ports>
Related links

See JMX-Based Monitoring for information on monitoring WSO2 products using JMX.

Tuning WSO2 products
Securing plain-text passwords

WSO2 products use a tool called “Secure Vault” to secure the plain-text passwords in configuration files. With this tool, you can encrypt the passwords. Use Secure Vault to encrypt all the passwords in the following configuration files.

  • <PRODUCT_HOME>/repository/conf/user-mgt.xml
  • <PRODUCT_HOME>/ repository /conf/carbon.xml
  • <PRODUCT_HOME>/ repository /conf/axis2/axis2.xml
  • <PRODUCT_HOME>/repository/conf/datasources/master-datasources.xml
  • <PRODUCT_HOME>/repository/conf/tomcat/catalina-server.xml
Related links

See Carbon Secure Vault Implementation for more information on how to secure plain-text passwords in WSO2 product configuration files.

Firewalls

The following ports must be accessed when operating within a firewall.

  • 9443 - Used by the management console and services that use the servlet transport, and is defined in the <PRODUCT_HOME>/repository/conf/tomcat/catalina-server.xml file.
  • 9763 - Used by the services that use servlet transport, and is defined in the <PRODUCT_HOME>/repository/conf/tomcat/catalina-server.xml file.
  • 9999 - Used for JMX monitoring, defined in the <PRODUCT_HOME>/repository/conf/carbon.xml file.
  • 8280 - Default HTTP port used by ESB for proxy services, and is defined in the <PRODUCT_HOME>/repository/conf/axis2/axis2.xml file.
  • 8243 - Default HTTPS port used by ESB for proxy services, and is defined in the <PRODUCT_HOME>/repository/conf/axis2/axis2.xml file.
Related links

See Default Ports of WSO2 Products for a list of common and product-specific ports used by WSO2 products.


Proxy servers

If the product is hosted behind a proxy such as Apache HTTPD, users can configure products to use the proxy server by providing the following system properties at the start-up.

-Dhttp.proxyHost=xxxx
-Dhttps.proxyHost=xxxx
-Dhttp.proxyPort=xxxx
-Dhttps.proxyPort=xxxx

Alternatively, this can be done by adding the following configurations in the <PRODUCT_HOME>/repository/conf/axis2/axis2.xml file.

<parameter name="Proxy">
    <Configuration>
    <proxyhost>your.proxy.host</proxyhost>
    <proxyport>your.proxy.port</proxyport>
    </configuration>
</parameter>

Also, see https://docs.oracle.com/javase/7/docs/api/java/net/doc-files/net-properties.html if you have hosts that you want to exclude from proxying via the http.nonProxyHosts property.

High availability

For high availability, WSO2 products must run on a cluster. This enables the WSO2 products to still work in the case of failover. Databases used for the repository, user management, and business activity monitoring can also be configured in a cluster or can use replication management provided by the RDBMS.

Related links
Data backup and archivingFor data backup and for archiving of data, use the functionality provided by the RDBMS.
Feature management

Compatible features can be installed on any WSO2 product using the WSO2 Carbon Feature Manager that comes with the product. The feature manager is powered by Equinox P2 and allows you to connect to a remote or local P2 repository and get any compatible WSO2 feature installed into the product's runtime.

Related links


Spooling and log rotation

Ensure that you have a relevant log rotation scheme to manage logs. Log4J properties for WSO2 products can be configured in the <PRODUCT_HOME>/repository/conf/log4j.properties file.

To roll the wso2carbon.log based on size, the following configurations can be used.

log4j.appender.CARBON_LOGFILE=org.apache.log4j.RollingFileAppender
log4j.appender.CARBON_LOGFILE=${carbon.home}/repository/logs/${instance.log}/wso2carbon${instance.log}.log
log4j.appender.CARBON_LOGFILE.MaxFileSize=1000KB
log4j.appender.CARBON_LOGFILE.MaxBackupIndex=10
Configure the UUID for logs

The log pattern defines the output format of the log file. From Carbon 4.4.3 onwards, the conversion character 'K' can be used in the pattern layout to log a UUID. For example, the log pattern can be [%K] [%T] [%S] [%d] %P%5p {%c} - %x %m {%c}%n, where [%K] is the UUID.

 

The UUID can be used for identifying forged messages in the log. By default, the UUID will be generated every time the server starts. If required, you can configure the UUID regeneration period by manually adding the following property to the log4j.properties file (stored in the <PRODUCT_HOME>/repository/conf directory):

 

log4j.appender.CARBON_LOGFILE.layout.LogUUIDUpdateInterval=<number_of_hours>
Related links

See Configuring Log4j Properties for more information on configuring the log4j.properties file.



Backup and recovery recommendations

None of the WSO2 products persist data in the file systems or retain or generate artifacts. By default, we only store log files in the file system and data and artifacts in the databases and the repository. 

What you should back up

  1. Database backups
    • Back up of all the databases defined in <PRODUCT_HOME>/repository/conf/datasources/master-datasources.xml
    • Back up any other databases configured in any files in the <PRODUCT_HOME>/repository/conf/datasources directory. 
  2. Artifact backups
    T
    his includes hot-deployment artifacts, web applications, synapse files, tenant directories, etc. Back up of the <PRODUCT_HOME>/repository directory periodically. The frequency of the back ups depends on your usage. For example, if you are creating or updating APIs daily, take this backup daily.
  3. WSO2 product instance backups
    A one-time-only backup that you take of the entire server directory. This includes all the configuration files, logs, server extensions, and deployment artifacts for both tenants and super tenants. This back up is ideally done when the server is ready to be deployed in a production environment. 

Backup recommendations

We recommend that you use a proper artifact management system such as Puppet to back up and manage your artifacts before deploying them in the WSO2 Carbon runtime. Also, use the WSO2 Update Manager (WUM) tool, which is a command-line utility that allows you to get the latest updates (bug fixes and security fixes) of a particular product release.

Diagram: managing your artifacts using a configuration management system

Recovery recommendations

Be sure to determine the following depending on your business-continuity requirements:

  • Recovery Point Objective (RPO): Up to what points are you to recover. This is determined by what the latest, knows, good point is.
  • Recovery Time Objective (RTO): How long does it take to recover to the RPO.
  • Backup Frequency: How frequently you should take backups. If your RPO is one day, your backup frequency should be daily.
  • Disaster Recovery Site: The place where the latest copy of your backup is. This can be from a different shelf in your data canter to a completely different geographical location.

We also recommend the following:

  1. Align your artifact deployment and recovery processes.
  2. Schedule disaster recovery drills to test the recoverability of the system.
  3. Test your artifacts in an environment that is identical to the production environment before deploying them into production.

Recovery strategy

The following steps include how to recover your setup using the backups:

  1. Recover the hot-deployment artifacts by replacing the <PRODUCT_HOME>/repository directory with the backed up copy.
  2. Recover the entire WSO2 product by directly replacing the existing WSO2 server directory in the production setup with the backup server directory. This will ensure that all the files, logs, and configurations made to the product do not need to be redone.
  3. To recover the databases, follow the recovery strategy recommended by the databases you are using. For information on supported and tested databases, see Tested Database Management Systems.

Minimum deployment patterns

See the section on minimum deployment patterns for more information on instance recommendations for each product.

  • No labels