Implementing a Lightweight User Authentication Mechanism for Hadoop at Tongcheng Travel

Large enterprises now commonly use shared Hadoop clusters, where public and private directories coexist in HDFS and resource queues are allocated per department; however, this model introduces significant security challenges, especially regarding client identity.

By default Hadoop lacks robust security: the client username and group are derived from environment variables (HADOOP_USER_NAME) or system properties, allowing arbitrary impersonation and unauthorized access to files and compute resources.

The Hadoop team recognized these gaps and added an authentication layer based on SASL/Kerberos, but Kerberos deployment proved complex, difficult to debug, poorly scalable, and introduced a single point of failure at the KDC.

To address these issues, Tongcheng Travel’s big‑data architecture team built a lightweight Hadoop user authentication mechanism that validates users via a simple username‑password pair, avoiding the overhead of full Kerberos while still providing strong access control.

Basic concept : before any Hadoop interaction, a user reads a pre‑configured password associated with their account, attaches it to each RPC request, and the NameNode validates the credentials. The username‑password mapping can be hot‑loaded via RPC.

Implementation details :

1. User credentials are stored in Hadoop’s UserGroupInformation class. During the first login, getLoginUser() creates a LoginContext and calls login() , after which the password is saved in the subject’s credentials.

2. A new protobuf file RefreshCheckUserPasswordProtocol.proto defines empty request/response messages and a service:

message RefreshCheckUserPasswordRequestProto {} message RefreshCheckUserPasswordResponseProto {} service RefreshCheckUserPasswordProtocolService { rpc refreshCheckUserPassword(RefreshCheckUserPasswordRequestProto) returns (RefreshCheckUserPasswordResponseProto); }

3. The service is added to the NamenodeProtocols interface:

public interface NamenodeProtocols extends ClientProtocol, DatanodeProtocol, NamenodeProtocol, RefreshAuthorizationPolicyProtocol, RefreshUserMappingsProtocol, RefreshCallQueueProtocol, GenericRefreshProtocol, GetUserMappingsProtocol, HAServiceProtocol, RefreshCheckUserPasswordProtocol {}

4. Server‑side translator and blocking service are registered:

RefreshCheckUserPasswordProtocolServerSideTranslatorPB refreshCheckUserPasswordXlator = new RefreshCheckUserPasswordProtocolServerSideTranslatorPB(this); BlockingService refreshCheckUserPasswordService = RefreshCheckUserPasswordProtocolService.newReflectiveBlockingService(refreshCheckUserPasswordXlator);

5. The dfsadmin command line gains a -refreshCheckUserPassword option, invoking the following method (simplified):

private int refreshCheckUserPassword() throws IOException { Configuration conf = getConf(); conf.set(CommonConfigurationKeys.HADOOP_SECURITY_SERVICE_USER_NAME_KEY, conf.get(DFSConfigKeys.DFS_NAMENODE_KERBEROS_PRINCIPAL_KEY, "")); DistributedFileSystem dfs = getDFS(); URI dfsUri = dfs.getUri(); boolean isHaEnabled = HAUtil.isLogicalUri(conf, dfsUri); if (isHaEnabled) { String nsId = dfsUri.getHost(); List > proxies = HAUtil.getProxiesForAllNameNodesInNameservice(conf, nsId, RefreshCheckUserPasswordProtocol.class); for (ProxyAndInfo proxy : proxies) { try { proxy.getProxy().refreshCheckUserPassword(); System.out.println("Refresh checkuser password successful for " + proxy.getAddress()); } catch (Exception e) { System.err.println("error refresh checkuser password for " + proxy.getAddress()); System.err.println(e.getMessage()); } } } else { RefreshCheckUserPasswordProtocol refreshProtocol = NameNodeProxies.createProxy(conf, FileSystem.getDefaultUri(conf), RefreshCheckUserPasswordProtocol.class).getProxy(); refreshProtocol.refreshCheckUserPassword(); System.out.println("Refresh checkuser password successful"); } return 0; }

6. During NameNode startup, the initialize(Configuration conf) method loads the username‑password mapping once, ensuring the data is ready for authentication.

7. Additional rollout features include a global switch to enable/disable the new security checks and a whitelist mechanism to gradually migrate users.

8. Deployment steps involved recompiling hadoop-common and hadoop-hdfs , replacing the relevant JARs on NameNode and DataNode, configuring the mapping tables, and enabling the security flag with a single command, all performed without service downtime.

The security enhancements have been successfully rolled out, operating for several months with stable performance and zero business impact.