This documentation is for WSO2 API Manager 2.1.0. View documentation for the latest release.

All docs This doc
||
Skip to end of metadata
Go to start of metadata

Deprecated!

These Puppet Modules based API-M instructions have been deprecated due to the following reasons:

  • The Puppet Modules are outdated.
    The Puppet Modules used in these instructions are supported only with Puppet 3.3.8 and above and below Puppet 4.0.0. As a result, these Puppet Modules are not compatible with the latest operating systems.
  • The Puppet Modules are based on older API-M deployment patterns.
    Click here for the latest API-M deployment patterns.

The following diagram illustrates pattern 4 which consists of a fully distributed API-M carry including two Gateway clusters, where each has one manager and one worker, with a single wso2am-analytics server instance. You can have the gateway environments in any preferred environment (e.g., local-area network (LAN) and demilitarized zone (DMZ)).

Prerequisites

HardwareEnsure that the minimum hardware requirements mentioned in the hardware requirements section are met. You can further fine tune your operating system for production by tuning performance.
Software
  • Oracle JDK 1.8
  • Puppet 3.3.8 and above and below 4.0.0
  • Debian 6 (or higher) or Ubuntu 12.04 (or higher)

Follow the instructions below to use the Puppet Modules in pattern 4 to deploy WSO2 API-M in a production environment. 

This guide assumes that you are familiar with Puppet, which is a configuration automation platform. Puppet uses a client-server model where the managed servers, which are referred to as Puppet Agents, communicate and retrieve configuration profiles from the Puppet Master. If you are not familiar with Puppet, you can follow the self-paced training, which should take about 2-3 days.

The Puppet Modules related instructions have been tested with Puppet 3.3.8 and on the following operating systems: Ubuntu 14.04, RedHat Enterprise Linux 6.7. The following installation instructions are specific to Ubuntu 14.04 LTS.

Step 1 - Create ten instances

Create ten vanilla instances, which do not have any configurations. In the subsequent steps you will configure one instance as the Puppet Master and the rest of the instances as Puppet Agents for the following nodes. The server that acts as the Puppet Master controls the configuration information in the WSO2 API-M the deployment, and the managed agent node, which is referred to as the Puppet Agent, requests their configuration catalog from the Puppet Master.

The latter mentioned instances can be either cloud instances, physical computer instances, or local Virtual Machine (VM) instances. You should be able to SSH in to all the instances.

The instructions on this page is for using the default puppet environment, production. However, if you are using a different puppet environment, you should either specify at the Puppet master side or the Puppet Agent side:

Specify environment in the main section of the puppet.conf file in Puppet Master.

[main]
environment=<environment-name>

Prepend --environment flag to puppet agent run command.

puppet agent -vt --environment=<environment-name>

NodeHieradata fileHostname

Store

api-store.yamlstore.dev.wso2.org
Publisherapi-publisher.yamlpub.dev.wso2.org
Gateway Manager DMZgateway-manager-dmz.yamldmz-mgt-gw.dev.wso2.org
Gateway Worker DMZgateway-worker-dmz.yamldmz-gw.dev.wso2.org
Gateway Manager LANgateway-manager-lan.yaml mgt-gw.dev.wso2.org
Gateway Worker LANgateway-worker-lan.yaml gw.dev.wso2.org
Key Managerapi-key-manager.yamlkm.dev.wso2.org
Traffic Managertraffic-manager.yamltm.dev.wso2.org
Analyticsdefault.yamlanalytics.dev.wso2.org


Step 2 - Set up the Puppet Master 

Carry out the following instructions on any one of the instances, which you created in step 1, to configure and set up the Puppet Master.

2.1 - Install Puppet Master

  1. Log in to the server that you are going to set up as the Puppet Master, as a super user.

  2. Execute the following commands to install Puppet Master.

    1. Sync time between the Master and the Agent.

      ntpdate pool.ntp.org ; apt-get update && sudo apt-get -y install ntp ; service ntp restart
    2. Navigate to the /tmp directory

      cd /tmp
    3. Download and install Puppet distribution.

      wget https://apt.puppetlabs.com/puppetlabs-release-trusty.deb && dpkg -i puppetlabs-release-trusty.deb && apt-get update && apt-get install puppetmaster
  3. Check the Puppet version to ensure that the Puppet version is 3.3.8 and above and below 4.0.0.

    puppet -V
  4. Set the hostname property by removing the current entry and adding the following in the etc/hostname file.

    puppetmaster
  5. Edit your Puppet Master hosts file /etc/hosts as follows and set your Puppet Master hostname.
    The following configuration is added in order to define the IP address that corresponds to the Puppet Master.

    127.0.0.1 localhost 127.0.0.1 puppetmaster
  6. For production deployments configure Puppet Master with Passenger and Apache

    This step is recommended in a production environment. However, it is optional in a testing environment.

2.2 - Configure the Puppet Master

  1. Log in to the server that you are going to set up as the Puppet Master, as a super user.

  2. Create the following directory.

    /etc/puppet/environments/production/

  3. Modify the Puppet Master /etc/puppet/puppet.conf file by appending the following to the [main] and [master] sections respectively.
    You need to update this command in order to define where to look for hiera.yaml file.

    [main]
    dns_alt_names=puppetmaster,puppet
    environmentpath = $confdir/environments
    hiera_config = /etc/puppet/hiera.yaml
     
    [master]
    autosign=true


    • dns_alt_names=puppetmaster,puppet
      This defines the hostnames that the Puppet Master uses to sign the certificates.

    • autosign=true
      This determines that the certificate will be signed automatically.

  4. Before restarting the Puppet Master, clean all certificates, including the Puppet Master’s certificate that has its old Domain Name Service (DNS) alt names.

    puppet cert clean --all
  5. Restart the Puppet Master for the new configuration changes to take effect and to regenerate the certificate with the new dns_alt_names.

    service puppetmaster restart
  6. Download the following packages from the respective GitHub pages.

    Package

    Download location

    wso2-puppet-common-1.0.0.zip https://github.com/wso2/puppet-common/releases/tag/v1.0.0
    wso2-wso2base-1.0.0.tar.gz https://github.com/wso2/puppet-base/releases/tag/v1.0.0
    wso2base-puppet-module-hieradata-1.0.0.zip https://github.com/wso2/puppet-base/releases/tag/v1.0.0
    wso2-wso2am_runtime-2.1.0.tar.gz https://github.com/wso2/puppet-apim/releases/tag/v2.1.0.1
    wso2am-runtime-puppet-module-hieradata-2.1.0.zip https://github.com/wso2/puppet-apim/releases/tag/v2.1.0.1
    wso2-wso2am_analytics-2.1.0.tar.gz https://github.com/wso2/puppet-apim/releases/tag/v2.1.0.1
    wso2am-analytics-puppet-module-hieradata-2.1.0.zip https://github.com/wso2/puppet-apim/releases/tag/v2.1.0.1
  7. Formulate the Puppet Modules.
    When using Puppet to configure WSO2 API-M, Puppet Modules are used to install, configure, and start the product.

    1. Extract the wso2-wso2base-1.0.0.tar.gz file, rename the folder name as wso2base, and copy that folder to the /etc/puppet/environments/production/modules/ directory.

    2. Extract the wso2-wso2am_runtime-2.1.0.tar.gz file, rename the folder name as wso2am_runtime, and copy that folder to the /etc/puppet/environments/production/modules/ directory.

    3. Extract wso2-wso2am_analytics-2.1.0.tar.gz, rename the folder name as wso2am_analytics, and copy that folder to the /etc/puppet/environments/production/modules/ directory.
    4. Optionally, install the Java module from Puppet Forge

      By default there is a Java module built into WSO2 Base Puppet Module. You need to carry out this step only when you need to configure an external Java module. 

      puppet module install 7terminals-java
    5. Install the stdlib module from Puppet Forge.
      You need to install the stdlib module, because WSO2 Puppet Modules use some of the functions in the stdlib module for data validation purposes.

      puppet module install puppetlabs-stdlib
  8. Formulate the Hiera data.
    Hiera provides a mechanism to separate the configuration data from Puppet scripts and manage them in a set of YAML files in a hierarchical manner. Therefore, the API-M related configuration data is managed using Hiera. Follow the instructions below to formulate the Hiera data.

    1. Unzip the wso2-puppet-common-1.0.0.zip file and copy the hiera.yaml file to the /etc/puppet directory.

    2. Edit the YAML data directory location as follows in the hiera.yaml file.

      :yaml: :datadir: "/etc/puppet/hieradata/%{::environment}"
    3. Unzip the wso2base-puppet-module-hieradata-1.0.0.zip file and copy the hieradata folder to the /etc/puppet/ directory.
      This folder contains the common Hiera data.

    4. Unzip wso2am-runtime-puppet-module-hieradata-2.1.0.zip and copy the hieradata folder to the /etc/puppet/ directory.
      This folder contains WSO2 API Manager specific Hiera data. You need to merge the common Hiera data and API Manager specific Hiera data.

        $deploymentdir     = '/mnt/jdk-8u112',
        $source            = 'jdk-8u112-linux-x64.tar.gz',
        $java_home         = '/opt/java',
    5. Unzip wso2am-analytics-puppet-module-hieradata-2.1.0.zip and copy the hieradata folder to the /etc/puppet/ directory.
      This folder contains WSO2 API Manager Analytics specific Hiera data. You need to merge the common Hiera data and API Manager Analytics specific Hiera data.
    6. Rename the /etc/puppet/hieradata/dev directory to the /etc/puppet/hieradata/production directory.

    If you already having the common hieradata and API Manager specific Hiera data, You have to merge the analytics hieradata with common hieradata and API Manager specific hieradata.

  9. Formulate the site.pp manifest.
    Puppet programs are referred to as manifests. Manifests are composed using Puppet code and possess a .pp extension. The site.pp manifest is the default main manifest that Puppet executes first. Follow the instructions below to formulate the site.pp manifest:

    1. Copy the  wso2-puppet-common-1.0.0/manifests/site.pp file to the /etc/puppet/environments/production/manifests directory. 

    2. Construct the files/packs as follows:

      1. Copy the WSO2 API-M pack file (wso2am-2.1.0.zip) to the /etc/puppet/environments/production/modules/wso2am_runtime/files directory.

      2. Copy the WSO2 API-M Analytics pack file (wso2am-analytics-2.1.0.zip) to the /etc/puppet/environments/production/modules/wso2am_analytics/files directory.
      3. Create the following directory - /etc/puppet/environments/production/modules/wso2base/files

      4. Copy the JDK installation file (e.g., jdk-8u112-linux-x64.tar.gz) to the /etc/puppet/environments/production/modules/wso2base/files directory.

      5. Add the JDK archive name, Java home and installation directory information in the /etc/puppet/environments/production/modules/wso2base/manifests/java.pp file. Note that these values are treated as the defaults if no other value is specified via the hiera files.

      6. Add the host entry in the /etc/puppet/hieradata/production/wso2/common.yaml file as follows:

        puppetmaster: ip_address: [your_puppet_master_ip_here] hostname: puppet

        This is an optional step. You can add any host entry in this manner.

Step 3 - Set up the WSO2 API-M Analytics and API-M Puppet Agents

You need to carry out the following steps separately on each of the nine instances to configure and set up an instance for the Store, Publisher, Key Manager, Traffic Manager, Analytics, Gateway Manager and the Gateway Worker.

The steps involved in setting up a Puppet Agent are identical irrespective of the WSO2 API-M pattern that you are deploying.

3.1 - Install Puppet Agent

  1. Log in to the server that you are going to set up as the Puppet Agent as a super user.
  2. Execute the following commands to install Puppet.

    wget https://apt.puppetlabs.com/puppetlabs-release-trusty.deb && dpkg -i puppetlabs-release-trusty.deb && apt-get update && apt-get install puppet 
  3. Edit the /etc/hosts file that corresponds to your Puppet Agent as follows:
    The following configuration is added in order to define the Puppet Master's IP address as the Puppet Agent needs to know the Puppet Master's IP address to be able to communicate with it.

    127.0.0.1 localhost 
    192.168.19.XXX puppet # Puppet Master's IP address

3.2 - Configure Puppet Agent

  1. Log in to the server that you are going to set up as the Puppet Agent as a super user.
  2. Modify the Puppet Agent /etc/puppet/puppet.conf file by appending the following to the [main] section.
    You need to do this to add the hostname of the Puppet Master. This hostname is mapped to the IP address in the /etc/hosts file. The Puppet Agent uses this information to discover the Puppet Master’s hostname in order to connect to the Puppet Master.

    [main] 
    server = puppet 

Step 4 - Set Facter variables for the Puppet Agents

Facter refers to Puppet's cross-platform system profiling library. It discovers and reports node specific information, which is available in your Puppet manifests in the form of variables. The Puppet Agent uses Facter to send its node information to the Puppet Master. Follow the instructions below to set the Facter variables.

  1. Set the Facter variables by modifying the deployment.conf file in each of the Puppet Agent instances.

    The following are the configurations that you need when running the Puppet Agent node for WSO2 API-M Analytics with pattern 1.

    product_name=wso2am_analytics
    product_version=2.1.0
    product_profile=default
    vm_type=openstack
    environment=production
    platform=default
    use_hieradata=true
    pattern=pattern-1

    The following are the configurations that you need when running the Puppet Agent node for WSO2 API-M Store with pattern 4.

    product_name=wso2am_runtime
    product_version=2.1.0
    product_profile=api-store
    vm_type=openstack
    environment=production
    platform=default
    use_hieradata=true
    pattern=pattern-4

    The following are the configurations that you need when running the Puppet Agent node for WSO2 API-M Publisher with pattern 4.

    product_name=wso2am_runtime
    product_version=2.1.0
    product_profile=api-publisher
    vm_type=openstack
    environment=production
    platform=default
    use_hieradata=true
    pattern=pattern-4

    The following are the configurations that you need when running the Puppet Agent node for WSO2 API-M Key Manager with pattern 4.

    product_name=wso2am_runtime
    product_version=2.1.0
    product_profile=api-key-manager
    vm_type=openstack
    environment=production
    platform=default
    use_hieradata=true
    pattern=pattern-4

    The following are the configurations that you need when running the Puppet Agent node for WSO2 API-M Gateway Manager in DMZ with pattern 4.

    product_name=wso2am_runtime
    product_version=2.1.0
    product_profile=gateway-manager-dmz
    vm_type=openstack
    environment=production
    platform=default
    use_hieradata=true
    pattern=pattern-4

    The following are the configurations that you need when running the Puppet Agent node for WSO2 API-M Gateway Worker in DMZ with pattern 4.

    product_name=wso2am_runtime
    product_version=2.1.0
    product_profile=gateway-worker-dmz
    vm_type=openstack
    environment=production
    platform=default
    use_hieradata=true
    pattern=pattern-4

    The following are the configurations that you need when running the Puppet Agent node for WSO2 API-M Gateway Manager in LAN with pattern 4.

    product_name=wso2am_runtime
    product_version=2.1.0
    product_profile=gateway-manager-lan
    vm_type=openstack
    environment=production
    platform=default
    use_hieradata=true
    pattern=pattern-4

    The following are the configurations that you need when running the Puppet Agent node for WSO2 API-M Gateway Worker in LAN with pattern 4.

    product_name=wso2am_runtime
    product_version=2.1.0
    product_profile=gateway-worker-lan
    vm_type=openstack
    environment=production
    platform=default
    use_hieradata=true
    pattern=pattern-4

    The following are the configurations that you need when running the Puppet Agent node for WSO2 API-M Traffic Manager with pattern 4.

    product_name=wso2am_runtime
    product_version=2.1.0
    product_profile=traffic-manager
    vm_type=openstack
    environment=production
    platform=default
    use_hieradata=true
    pattern=pattern-4
    product_name=<product_name>_runtime
    product_version=<product_version>
    product_profile=<hiera_file_name_without_extension>
    environment=production
    vm_type=openstack
    use_hieradata=true
    platform=default
    pattern=<pattern>

Step 5 - Update the clustering related configurations

For more information, see Configuring Clustering for a Selected Pattern.

Step 6 - Configure the keyStore and client trustStore

For more information, see Configuring the keyStore and client trustStore for a Selected Pattern.

Step 7 - Run the Puppet Agents

Carry out the instructions to run the Puppet Agent separately on each of the nine Puppet Agent instances in the following order.

  1. Initially, start the following nodes in given order:

    1. WSO2 API-M Analytics
    2. Traffic Manager
    3. Gateway Manager in DMZ and LAN
    4. Gateway Worker in DMZ and LAN

      The respective Gateway DMZ and LAN nodes can start in parallel.

  2. Thereafter, start the Key Manager, Publisher and Store in any order.

  1. Log in to Puppet Agent as a super user.
  2. Copy the modified deployment.conf file to the /opt directory.
  3. Copy the setup.sh file to the /opt directory and execute the following command. 
    As the root user, you need to execute this command to add the execute permission to the setup.sh file.

    chmod 755 setup.sh 
    #!/bin/bash
    echo "#####################################################"
    echo " Starting cleanup "
    echo "#####################################################"
    ps aux | grep -i wso2 | awk '{print $2}' | xargs kill -9
    echo "#####################################################"
    echo " Setting up environment "
    echo "#####################################################"
    mkdir -p /etc/facter/facts.d
    cp deployment.conf /etc/facter/facts.d/deployment_pattern.txt
    echo "#####################################################" 
    echo " Installing " 
    echo "#####################################################"
    puppet agent --enable 
    puppet agent -vt 
    puppet agent --disable
  4. Run the following command to set up and install the Puppet Agent.

    ./setup.sh

When running WSO2 API-M or WSO2 API-M Analytics, <PRODUCT_HOME> corresponds to the following directory.

NodeInstalled Directory
WSO2 API-M Analytics

/mnt/wso2am-analytics-2.1.0

WSO2 API-M

This is applicable to the following nodes.
Store, Publisher, Key Manager, Traffic Manager, Analytics, Gateway Manager (DMZ & LAN), and Gateway Worker (DMZ & LAN).

/mnt/wso2am-2.1.0

  • No labels