Why are there a series of technical challenges behind “OMG buy it”>>> 
Issue
When hive metadata has partition information partition = x and HDFS path does not have partition directory partition = X. Executing some hive SQL will report an error: org. Apache. Hadoop. Mapred. Invalidinputexception: input path does not exist
This is tez’s usual directory inconsistency problem
See issue: https://issues.apache.org/jira/browse/HIVE-13781
Impact: hive on tez. Hive3
Solutions
When executing hive SQL, the consistency between HDFS directory and hive metadata is guaranteed
Retreat on MR
Drop all partitions, execute msck repair table to repair the whole table (or use mask repair table sync partitions to synchronize the partition folder information on HDFS)
Similar Posts:
- “Execution error, return code 1 from org. Apache. Hadoop. Hive. QL. Exec. Movetask” error occurred when hive imported data locally
- [Solved] hadoop Configuration Modify Error: hive.ql.metadata.HiveException
- [Solved] HIVE Error: need to specify partition columns because…
- MySQL master-slave synchronization error 1507
- HDFS problem set (1), use the command to report an error: com.google.protobuf.servicee xception:java.lang.OutOfMemoryError :java heap space
- [Solved] spark Connect hive Error: javax.jdo.JDODataStoreException: Required table missing : “`DBS`” in Catalog “” Schema “”
- hdfs dfs -rm -r cause GC overhead limit exceeded
- [Solved] Hive Update and Delete Error: Attempt to do update or delete using transaction manager
- User xxx does not have privileges for CREATETABLE [How to Solve]
- [Solved] Hadoop Error: The directory item limit is exceeded: limit=1048576 items=1048576