当前位置：首页 > news >正文

网站开发趋势微信营销平台系统

news 2026/4/7 17:10:02

网站开发趋势,微信营销平台系统,360百度网站怎么做,链接提交入口背景介绍由于 CDH（Clouderas Distribution Hadoop ）近几年已经开始收费并限制节点数量和版本升级，最近使用开源的 hadoop 搭了一套测试集群，其中的权限管理组件用到了Apache Ranger，所以记录一下编译打包过程。组件…

背景介绍

由于 CDH（Cloudera's Distribution Hadoop ）近几年已经开始收费并限制节点数量和版本升级，最近使用开源的 hadoop 搭了一套测试集群，其中的权限管理组件用到了Apache Ranger，所以记录一下编译打包过程。

组件版本

Hadoop 2.8.5
Hive 2.3.7
Hbase 1.4.9
Ranger 1.2.0

注意：由于 Ranger 2.0 版本之后支持的 Hadoop 是 3.x 的版本，所以选择了 2.0 之前最新的 Ranger 1.2.0 版本

编译步骤

下载源码

# 克隆源码
git clone https://github.com/apache/ranger.git
# 切换分支
git checkout release-ranger-1.2.0

修改配置

在根目录的 pom.xml 中修改版本：

Hadoop 从 2.7.1 修改到 2.8.5

Hive 从 2.3.4 修改到 2.3.7

Hbase 从 1.3.2 修改 1.4.9

各位可以根据需要，进行修改，当然仅修改这里很简单，但不代表编译就兼容

进行编译

mvn clean compile package assembly:assembly install -Dmaven.test.skip  -DskipTests -Drat.skip=true -Dpmd.skip=true

成功后，可见如下信息：

打包后的产物就在 target 目录中：

这里面有我们部署的 admin 包，还有各个支持 ranger 权限控制的插件

错误记录

1，Hadoop API 不兼容

https://lists.apache.org/thread/rpk62z0x1drs7oqss1rmy0rrwnp1r5j6

修改 KMSAuthenticationFilter.java 类的

props.setProperty(DelegationTokenAuthenticationHandler.TOKEN_KIND,KMSClientProvider.TOKEN_KIND);
// 调整为Text 转成 toString
props.setProperty(DelegationTokenAuthenticationHandler.TOKEN_KIND,KMSClientProvider.TOKEN_KIND.toString());

2，Hbase API 不兼容

a. 修改 ranger-hbase-plugin 子模块：

RangerAuthorizationCoprocessor.java：新增实现接口

	@Overridepublic void preGetClusterStatus(ObserverContext<MasterCoprocessorEnvironment> observerContext) throws IOException {}@Overridepublic void postGetClusterStatus(ObserverContext<MasterCoprocessorEnvironment> observerContext, ClusterStatus clusterStatus) throws IOException {}@Overridepublic void preClearDeadServers(ObserverContext<MasterCoprocessorEnvironment> observerContext) throws IOException {}@Overridepublic void postClearDeadServers(ObserverContext<MasterCoprocessorEnvironment> observerContext, List<ServerName> list, List<ServerName> list1) throws IOException {}@Overridepublic void preMoveServers(ObserverContext<MasterCoprocessorEnvironment> ctx, Set<Address> servers, String targetGroup) throws IOException {}@Overridepublic void postMoveServers(ObserverContext<MasterCoprocessorEnvironment> observerContext, Set<Address> servers, String s) throws IOException {}@Overridepublic void preMoveServersAndTables(ObserverContext<MasterCoprocessorEnvironment> observerContext, Set<Address> set, Set<TableName> set1, String s) throws IOException {}@Overridepublic void postMoveServersAndTables(ObserverContext<MasterCoprocessorEnvironment> observerContext, Set<Address> set, Set<TableName> set1, String s) throws IOException {}@Overridepublic void preRemoveServers(ObserverContext<MasterCoprocessorEnvironment> observerContext, Set<Address> set) throws IOException {}@Overridepublic void postRemoveServers(ObserverContext<MasterCoprocessorEnvironment> observerContext, Set<Address> set) throws IOException {}

RangerAuthorizationCoprocessorBase.java：修改参数类型

    public void preMoveServers(final ObserverContext<MasterCoprocessorEnvironment> ctx, Set<HostAndPort> servers, String targetGroup) throws IOException {}public void postMoveServers(ObserverContext<MasterCoprocessorEnvironment> ctx, Set<HostAndPort> servers, String targetGroup) throws IOException {}// 调整参数类型，成如下：public void preMoveServers(final ObserverContext<MasterCoprocessorEnvironment> ctx, Set<Address> servers, String targetGroup) throws IOException {}public void postMoveServers(ObserverContext<MasterCoprocessorEnvironment> ctx, Set<Address> servers, String targetGroup) throws IOException {}

b.修改 ranger-hbase-plugin-shim 子模块

RangerAuthorizationCoprocessor.java：新增实现接口，注释掉部分接口

// 新增@Overridepublic void preCommitStoreFile(ObserverContext<RegionCoprocessorEnvironment> observerContext, byte[] bytes, List<Pair<Path, Path>> list) throws IOException {}@Overridepublic void postCommitStoreFile(ObserverContext<RegionCoprocessorEnvironment> observerContext, byte[] bytes, Path path, Path path1) throws IOException {}@Overridepublic void preMoveServersAndTables(ObserverContext<MasterCoprocessorEnvironment> observerContext, Set<Address> set, Set<TableName> set1, String s) throws IOException {}@Overridepublic void postMoveServersAndTables(ObserverContext<MasterCoprocessorEnvironment> observerContext, Set<Address> set, Set<TableName> set1, String s) throws IOException {}@Overridepublic void preRemoveServers(ObserverContext<MasterCoprocessorEnvironment> observerContext, Set<Address> set) throws IOException {}@Overridepublic void postRemoveServers(ObserverContext<MasterCoprocessorEnvironment> observerContext, Set<Address> set) throws IOException {}public void preBalanceRSGroup(final ObserverContext<MasterCoprocessorEnvironment> ctx, String groupName) throws IOException {}@Overridepublic void preGetClusterStatus(ObserverContext<MasterCoprocessorEnvironment> observerContext) throws IOException {}@Overridepublic void postGetClusterStatus(ObserverContext<MasterCoprocessorEnvironment> observerContext, ClusterStatus clusterStatus) throws IOException {}@Overridepublic void preClearDeadServers(ObserverContext<MasterCoprocessorEnvironment> observerContext) throws IOException {}@Overridepublic void postClearDeadServers(ObserverContext<MasterCoprocessorEnvironment> observerContext, List<ServerName> list, List<ServerName> list1) throws IOException {}@Overridepublic void preMoveServers(ObserverContext<MasterCoprocessorEnvironment> observerContext, Set<Address> set, String s) throws IOException {}@Overridepublic void postMoveServers(ObserverContext<MasterCoprocessorEnvironment> observerContext, Set<Address> set, String s) throws IOException {}// 注释掉
//  public void preMoveServers(final ObserverContext<MasterCoprocessorEnvironment> ctx, Set<HostAndPort> servers, String targetGroup) throws IOException {}
//  public void postMoveServers(ObserverContext<MasterCoprocessorEnvironment> ctx, Set<HostAndPort> servers, String targetGroup) throws IOException {}