Apache ranger | DebugAH

Version information is as follows:

hadoop2.9.2

hive 2.x

Ranger latest version 2.1.0

After deploying the Ranger plug-in on the hive side, an error is reported when using beeline to connect and query the database. The error information is as follows:

verbose: on
0: jdbc:hive2://192.168.0.9:10000> show databases;
Getting log thread is interrupted, since query is done!
Error: Error running query: java.lang.NoSuchFieldError: REPLLOAD (state=,code=0)
org.apache.hive.service.cli.HiveSQLException: Error running query: java.lang.NoSuchFieldError: REPLLOAD
    at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:264)
    at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:250)
    at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:309)
    at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:250)
    at org.apache.hive.beeline.Commands.executeInternal(Commands.java:977)
    at org.apache.hive.beeline.Commands.execute(Commands.java:1148)
    at org.apache.hive.beeline.Commands.sql(Commands.java:1063)
    at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1134)
    at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:965)
    at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:875)
    at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:499)
    at org.apache.hive.beeline.BeeLine.main(BeeLine.java:482)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hive.service.cli.HiveSQLException: Error running query: java.lang.NoSuchFieldError: REPLLOAD
    at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:218)
    at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:269)
    at org.apache.hive.service.cli.operation.Operation.run(Operation.java:324)
    at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:460)
    at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:447)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
    at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
    at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
    at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
    at com.sun.proxy.$Proxy52.executeStatementAsync(Unknown Source)
    at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:294)
    at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:497)
    at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437)
    at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422)
    at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
    at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
    at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoSuchFieldError: REPLLOAD
    at org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.checkPrivileges(RangerHiveAuthorizer.java:700)
    at org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:817)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:604)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:472)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:329)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1158)
    at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1145)
    at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:184)
    ... 27 more

Most of these errors are caused by version conflicts or version mismatches

Go to the software package of Ranger admin and check the hive version supported by Ranger 2.1.0:

My ranger-admin
#cd /data/ranger/ranger-2.1.0-SNAPSHOT-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive
 Check hive package information:
#ll
total 41504
-rw-r--r-- 1 ranger ranger   492033 Nov  5 15:18 hive-common-3.1.0.jar
-rw-r--r-- 1 ranger ranger 40603464 Nov  5 15:20 hive-exec-3.1.0.jar
-rw-r--r-- 1 ranger ranger   125261 Nov  5 15:20 hive-jdbc-3.1.0.jar
-rw-r--r-- 1 ranger ranger    36852 Nov  5 15:19 hive-metastore-3.1.0.jar
-rw-r--r-- 1 ranger ranger   566570 Nov  5 15:19 hive-service-3.1.0.jar
-rw-r--r-- 1 ranger ranger   313702 Nov  5 15:19 libfb303-0.9.3.jar
-rw-r--r-- 1 ranger ranger   246445 Nov  5 15:57 libthrift-0.12.0.jar
-rw-r--r-- 1 ranger ranger    99652 Nov  5 17:18 ranger-hive-plugin-2.1.0-SNAPSHOT.jar

It is found that the supported hive version is 3. X, while the version I currently use is 2. X, resulting in an error using beeline

Solution 1: upgrade hive to 3. X

[hduser@yjt ~]$ beeline -u jdbc:hive2://192.168.0.230:10000 -n hduser
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data1/hadoop/hive_3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data1/hadoop/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://192.168.0.230:10000
Connected to: Apache Hive (version 3.1.2)
Driver: Hive JDBC (version 3.1.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 3.1.2 by Apache Hive
0: jdbc:hive2://192.168.0.230:10000> show databases;
INFO  : Compiling command(queryId=hduser_20191108203220_9ae51d7b-38f0-4f3a-af96-c4ddb91ee9ad): show databases
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
INFO  : Completed compiling command(queryId=hduser_20191108203220_9ae51d7b-38f0-4f3a-af96-c4ddb91ee9ad); Time taken: 1.348 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Executing command(queryId=hduser_20191108203220_9ae51d7b-38f0-4f3a-af96-c4ddb91ee9ad): show databases
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hduser_20191108203220_9ae51d7b-38f0-4f3a-af96-c4ddb91ee9ad); Time taken: 0.039 seconds
INFO  : OK
INFO  : Concurrency mode is disabled, not creating a lock manager
+----------------+
| database_name  |
+----------------+
| default        |
+----------------+
1 row selected (1.878 seconds)
0: jdbc:hive2://192.168.0.230:10000> !q
Closing: 0: jdbc:hive2://192.168.0.230:10000
[hduser@yjt ~]$ hive --version
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data1/hadoop/hive_3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data1/hadoop/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive 3.1.2
Git git://HW13934/Users/gates/tmp/hive-branch-3.1/hive -r 8190d2be7b7165effa62bd21b7d60ef81fb0e4af
Compiled by gates on Thu Aug 22 15:01:18 PDT 2019
From source with checksum 0492c08f784b188c349f6afb1d8d9847
[hduser@yjt ~]$

Solution 2: delete the package under Ranger admin hive, and then copy the corresponding package from hive client to this directory (I didn’t test it)

DebugAH

How to Solve Your Programmer Error

Tag Archives: Apache ranger

Apache Ranger integrates hive error records [How to Solve]