The latest version for DAS is WSO2 Data Analytics Server 3.1.0. View documentation for the latest release.
WSO2 Data Analytics Server is succeeded by WSO2 Stream Processor. To view the latest documentation for WSO2 SP, see WSO2 Stream Processor Documentation.

All docs This doc
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
||
Skip to end of metadata
Go to start of metadata

HDFS datasource is used to set up a connection to a remote HDFS. In WSO2 DAS, you can create a HDFS datasource by specifying the datasource configuration in the <DAS_HOME>/repository/conf/datasources/analytics-datasources.xml file as follows.

<datasource>
    <name>WSO2_ANALYTICS_FS_DB_HDFS</name>
    <description>The datasource used for analytics file system</description>
    <jndiConfig>
        <name>jdbc/WSO2HDFSDB</name>
    </jndiConfig>
    <definition type="HDFS">
        <configuration>
            <property>
                <name>fs.default.name</name>
                <value>hdfs://localhost:9000</value>
            </property>
            <property>
                <name>dfs.data.dir</name>
                <value>/dfs/data</value>
            </property>
            <property>
                <name>fs.hdfs.impl</name>
                <value>org.apache.hadoop.hdfs.DistributedFileSystem</value>
            </property>
            <property>
                <name>fs.file.impl</name>
                <value>org.apache.hadoop.fs.LocalFileSystem</value>
            </property>
        </configuration>
    </definition>
</datasource>

For information on the above properties, and other properties that you can specify in this datasource configuration, go to Apache Hadoop Documentation.

Permission enforcement for remote HDFS clients

While the HDFS cluster is being set up for use by WSO2 DAS, please ensure that the UNIX user account in all clients are present in the HDFS nodes and that all users have R/W permission to HDFS root directory.

Alternatively, you can suspend the enforcement of distributed filesystem permission enforcement through adding the following configuration to <HADOOP_HOME>/etc/hadoop/hdfs-site.xml in all HDFS nodes:

<property>
	<name>dfs.permissions.enabled</name>
	<value>false</value>
</property>
  • No labels