Hive 2.1.0

I hope this helps anyone who wants to use Hiverunner with Hive 2.1.0. I had to make some modifications for the tool to work on 2.1.0:

1. Did not work on Windows, I had to run on Linux (see #45 )
2. Switched Metastore to embedded Derby

• set “datanucleus.schema.autoCreateAll” to “true”
• modify StandaloneHiveServerContext.java

//metaStorageUrl = "jdbc:hsqldb:mem:" + UUID.randomUUID().toString();
metaStorageUrl = "jdbc:derby:;databaseName=" + basedir.getRoot().getAbsolutePath() + "/metastore_db";

1. Hive API change

• HiveServerContainer.java

21a22
> import org.apache.hadoop.hive.conf.HiveVariableSource;
23c24
< import org.apache.hadoop.hive.ql.parse.VariableSubstitution;
---
> import org.apache.hadoop.hive.conf.VariableSubstitution;
206c207,214
<         return new VariableSubstitution();
---
>         final SessionState ss = currentSessionState;
>         return new VariableSubstitution(new HiveVariableSource() {
>
>                       @Override
>                       public Map<String, String> getHiveVariable() {
>                               return ss.getHiveVariables();
>                       }
>               });

[patduin]

Hi Per,

I am updating one of our hive metastore testing tools (https://github.com/HotelsDotCom/beeju) to use Hive 2.1.1 and I am seeing similar issues with HSQL DB looks like Hive doesn’t support it anymore due to transactions. We’re hitting this code in hive: https://github.com/apache/hive/blob/rel/release-2.1.1/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L1994

So we are considering going back to Derby as well, have you got any other recommendations?

Thanks, Patrick

Update, fyi: I’ve updated our project (BeeJU) to use derby in memory db (jdbc:derby:memory) so fairly similar to hsqldb I haven’t seen a slow down in test. But our project is a tiny one. Hopefully it won’t be a problem for HiveRunner.

[massdosage]

We have tested the 4.0.0 release against Hive 2.3.3 and HIve 2.1.0. Admittedly we haven’t tested all Hive functionality but our test cases (in downstream projects) passed. So I’ve closed this, we can open new tickets if users report other things not working in 4.0.0