Inaccurate schema error when same exists in primary Metastore
See original GitHub issueAssume there is a schema in primary metastore: pm_mydb
In different hive metastore, there is schema named as mydb_another.
When configuring waggle dance, we have provided prefix for external metastore as pm_.
When we ran show databases, we get following result
hive> show databases;
pm_mydb
pm_mydb_another
After this when we tried to execute query against the pm_mydb, schema within primary MS, I am getting error
hive> use pm_mydb;
ERROR: FAILED: SemanticException [Error 10072]: Database does not exist: pm_mydb
...
When I changed the prefix to pm2_ for external MS, everything works fine.
hive> show databases;
pm_mydb
pm2_mydb_another
hive> use pm_mydb;
LIST_OF_TABLES
It looks like that waggle dance is resolving pm_mydb as external schema (mydb schema under the MS with pm_ as prefix).
Issue Analytics
- State:
- Created 5 years ago
- Comments:5 (3 by maintainers)
Top Results From Across the Web
Trying to reinstall Hive but have metastore errors
Hi, I am trying to reinstall Hive because i have issue with the metastore. I don't have an ibdata file anymore. I have...
Read more >How to troubleshoot several Apache Hive metastore problems
Either your MetaData is incorrect, or you need to enable "datanucleus.schema.autoCreateTables" org.datanucleus.store.rdbms.exceptions.
Read more >Upgrade Amazon EMR Hive Metastore from 5.X to 6.X
To get the Metastore schema version in the current EMR 5.X cluster, run the following command in the primary node:.
Read more >FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68 ...
Solution : Go to below directory: C:\hadoop_new\apache-hive-3.1.2\scripts\metastore\upgrade\derby\hive-schema-3.1.0.derby.sql comment the ...
Read more >Troubleshooting Guide - Apache Software Foundation
This error generally occurs when the schema has evolved in backwards incompatible way by deleting some column 'col1' and we are trying to...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
This is expected behaviour. Waggle Dance uses the prefix to redirect to the external metastore. The
pm_
prefix is configured to go to the federated metastore. Which is in turn tell you that pm_mydb (or mydb in the external MS) doesn’t exist. This is a weird situation I suspect because the database was created bypassing Waggle Dance. Best to pick a prefix that doesn’t clash with existing databases. (or rename your database)If this is not an option there is the alternative to switch to MANUAL database resolution mode (README has more details). This doesn’t use prefixes.
@massdosage PR looks good to me.