Tag Archives: kylin

Solve kylin error: java.lang.arrayindexoutofboundsexception: – 1

 

 

Error message:

2017-06-26 14:17:30,167 INFO  [pool-9-thread-3] cli.DictionaryGeneratorCLI:57 : Building snapshot of ENERGON_DM.DM_DIAGNOSIS_FACT_SGYY_ROOT_SET_FLAG
2017-06-26 14:17:31,484 ERROR [pool-9-thread-3] common.HadoopShellExecutable:65 : error execute HadoopShellExecutable{id=8188ef71-3bc6-4b9a-b87d-5559e3ea6b4a-03, name=Bu
ild Dimension Dictionary, state=RUNNING}
java.lang.ArrayIndexOutOfBoundsException: -1
        at org.apache.kylin.dict.lookup.SnapshotTable.takeSnapshot(SnapshotTable.java:89)
        at org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.java:132)
        at org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:270)
        at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:58)
        at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41)
        at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

solve:

There are two tables with the same name but different case under/table/in the meta. The ID of the column in the meta file starts with 0 and should be 1 under normal conditions. Kylin internally saves the table meta into a HashMap, and the key is the upper case table name, whether it is upper case or not. In this case, the content of the small and medium-sized meta overwrites the uppercase meta content. When checking the ID, one operation is to set the ID value to – 1. If the original ID is 0, – 1 becomes – 1, and then the above exception is thrown.

 

org.apache.kylin.dict.lookup.SnapshotTable:

org.apache.kylin.metadata.model.ColumnDesc:

Question meta:

PS: class org.apache.kylin.metadata.model.columndesc corresponds to a column in the table:

v.s.

 

Solve kylin error: java.lang.illegalstateexception

When a kylin build job is executed to the third step, an error is reported in extract fact table distinct columns:

2017-05-24 20:04:07,930 ERROR [pool-9-thread-3] common.MapReduceExecutable:127 : error execute MapReduceExecutable{id=a79c9625-39aa-4f17-8015-73b640558425-02, name=Extra
ct Fact Table Distinct Columns, state=RUNNING}
java.lang.IllegalStateException
        at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:98)
        at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
        at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Solution: find the kylin instance that executes the build task, reload the metadata or restart the kylin service (not recommended), and then rebuild

Cause analysis: the meta in the memory of the job server executing the build is not updated, which is caused by using the meta in the cache to execute the build

Kylin Packaged deployment: java.lang.NoSuchMethodError:org.apache.jasper.ParserUtils:method()v not found

The error is that the Tomcat version of kylin is wrong. My Hadoop version is 3.1, which should be matched with Tomcat 8. X, but my packaged version of kylin is Tomcat 7. X. download the corresponding version from GIT – All I get from the official website are Hadoop 2