Hive and Hadoop Exceptions

I installed Hive 1.0.0 on Hadoop 1.2.1. When I try to enter the Hive CLI, it reports following exceptions

1
org.apache.hadoop.hive.ql.metadata.HiveException:java.io.IOException:Filesystem closed

According to the search here, the mainly reason for this is that when multiple nodes read HFDS files if one node is offline, it will throw such exception when the other nodes are still reading the data cached. There are two ways to resolve this.

  • Turn off JVM reuse
1
2
3
4
<property>
<name>mapred.job.reuse.jvm.num.tasks</name>
<value>-1</value>
</property>
  • Disable caches
1
2
3
4
<property>
<name>fs.hdfs.impl.disable.cache</name>
<value>true</value>
</property>