Configuring Adapter Connections

Adapter for HDFS 9.8 | webMethods Adapter for HDFS Documentation | webMethods Adapter for HDFS Installation and User’s Documentation | Managing Adapter for HDFS Connections | Configuring Adapter Connections

Configuring Adapter Connections

Adapter Connection Parameters that Override the Configuration File Properties

Encrypting Communication with the HDFS Cluster

When you configure Adapter for HDFS connections, you specify information that Integration Server uses to connect to an HDFS cluster. You must configure Adapter for HDFS connections manually by using the Integration Server Administrator.

Note:
Ensure that you have performed the prerequisite tasks before configuring Adapter for HDFS, as specified in Before Configuring or Managing Adapter Connections.

To configure an adapter connection

1. In the Adapters menu of the Integration Server Administrator's navigation area, click webMethods Adapter 9.8 for HDFS.

The Connections screen appears.

2. On the Connections screen, click Configure New Connection.

3. In the webMethods Adapter for HDFS section, provide following information about the package and folder in which you are creating the connection:

Field	Description/Action
Package	The package in which to create the connection. You must have created the package before you can select it from the drop-down list. For general information about creating packages, see the webMethods Service Development Help for your release. Note: Configure the connection in a user-defined package. For other important considerations when creating packages for Adapter for HDFS, see Package Dependency Requirements and Guidelines.
Folder Name	The folder in which to create the connection.
Connection Name	The name you want to provide for the connection. Connection names cannot have spaces or use special characters reserved by Integration Server and Designer. For more information about the use of special characters in package, folder, and element names, see the webMethods Service Development Help for your release.

4. In the Connection Properties section, provide following information about the HDFS NameNode to which Integration Server connects:

Field	Description/Action
Description	The description you want to provide for the connection. This is an optional field.
NameNode URI	URI of the NameNode in the HDFS cluster to which Integration Server connects. Specify this value in the hdfs://<server>:<port> format. If you have configured High-Availability (HA) for your HDFS cluster, you must specify the NameNode URI in the hdfs://<nameservices> format.

5. In the File System Properties section, provide following information about the data blocks added to or updated in the HDFS cluster:

Field	Description/Action
Replication	The number of copies of the data blocks into which a file that is updated in or added to the HDFS cluster is split. The default value of this parameter is one.
Block Size (MB)	The size, in MB, of each data block of a file that is updated in or added to the HDFS cluster. The default value of this parameter is 1MB.
Buffer Size (KB)	The size, in KB, of the buffer that Integration Server uses for each read or write operation to the HDFS cluster. The default value of this parameter is 4KB.

6. In the Authentication Properties section, select the authentication scheme for users accessing the HDFS cluster.

You can select one of the following authentication schemes from the drop-down list:

Simple (No Authentication): Specify user and group names with valid access to the HDFS cluster.

Kerberos (Using Keytab): Integration Server checks the contents of a keytab file for information about the specified principal, and then authenticates the principal.

Kerberos (Using Password): Integration Server checks the KDC for information about the specified principal, and then authenticates the principal.

You can select either of the Kerberos authentication schemes only if you have configured Integration Server to use Kerberos. For more information about configuring Kerberos in Integration Server, see the webMethods Integration Server Administrator’s Guide for your release.

7. In the Authentication Properties section, provide information about the selected authentication scheme.

The information that you need to provide depends on the type of authentication scheme that you select.

For this scheme...	Specify...
Simple (No Authentication)	Select this parameter value to specify user and group names with valid access to the HDFS cluster.
	Option	Description
	User	A user on the host operating system with valid access to the HDFS cluster.
	Group	The group to which the specified User belongs.
Kerberos (Using Keytab)	Select this parameter to enable Kerberos authentication for a principal based on information provided in the keytab file. You can select this scheme only if you have configured Integration Server to use Kerberos. For more information, see the webMethods Integration Server Administrator’s Guide for your release.
	Option	Description
	Kerberos Principal	The name of the principal to use for Kerberos authentication. Specify the fully qualified domain name of the principal in the hdfs/fully.qualified.domain.name@REALM format.
	Path to Keytab File	The local path to the keytab file that contains information about the specified principal. For example, you can specify a keytab file at the following path: C:/users/foo/hdfs.keytab
	Group	The group to which the user associated with the specified Kerberos principal belongs.
Kerberos (Using Password)	Select this parameter to enable Kerberos authentication for a principal based on the credentials available in the KDC. You can select this scheme only if you have configured Integration Server to use Kerberos. For more information, see the webMethods Integration Server Administrator’s Guide for your release.
	Option	Description
	Kerberos Principal	The name of the principal to use for Kerberos authentication. Specify the principal name in the hdfs/fully.qualified.domain.name@REALM format.
	Principal Password	The password used to authenticate the principal to the KDC.
	Group	The group to which the user associated with the specified Kerberos Principal belongs.

8. In the Advanced Settings section, specify the local paths to the HDFS configuration files core-site.xml and hdfs-site.xml.

Based on your requirements, you can use these files for advanced configurations on the HDFS cluster.

Important:
Software AG recommends that you maintain identical versions of both these advanced configuration files on the HDFS cluster and the computer on which webMethods Adapter for HDFS is running.

Field	Description/Action
Path to core-site.xml	The local path to the core-site.xml file. For example, you can specify core-site.xml at the following path: C:/users/foo/core-site.xml
Path to hdfs-site.xml	The local path to the hdfs-site.xml file. For example, you can specify hdfs-site.xml at the following path: C:/users/foo/hdfs-site.xml

Note:

The values of the connection parameters that you specify on this screen override certain properties in the configuration files core-site.xml and hdfs-site.xml on the HDFS cluster. For details, see Adapter Connection Parameters that Override the Configuration File Properties

For the clients authenticated to access the HDFS cluster using Kerberos, you can configure their communication with the HDFS cluster through Integration Server by setting values of certain properties in core-site.xml and hdfs-site.xml. For more information, see Encrypting Communication with the HDFS Cluster.

9. Click Save Connection.

If the parameters are valid, the connection you created appears on the adapter's Connections screen.

10. Click Return to webMethods Adapter for HDFS Connections to return to the Connections screen.