[SUPPORT] spark-sql schema_evolution
See original GitHub issueDescribe the problem you faced
I have a hudi table
create table hudi_mor_test (
uuid int,
name string,
price double,
ts bigint
) using hudi
tblproperties (
type = 'mor',
primaryKey = 'uuid',
preCombineField = 'ts'
);
then I want to change the order of the field and follow the instruction of document https://hudi.apache.org/docs/schema_evolution
alter table hudi_mor_test alter column point after uuid;
but I get a error
Error in query: ALTER COLUMN ... FIRST | ALTER is only supported with v2 tables.
I want know what is the problem
Environment Description
- Hudi version : 0.11.1
- Spark version : 3.2.2
- Hive version : 2.3.9
- Hadoop version : 2.7.3
- Storage hdfs
Issue Analytics
- State:
- Created a year ago
- Comments:8 (4 by maintainers)
Top Results From Across the Web
Schema Evolution - Apache Hudi
Schema evolution allows users to easily change the current schema of a Hudi table to adapt to the data that is changing over...
Read more >Data processing with Spark: schema evolution - Dev Genius
Managing schema evolution adds complexity in a data management system. Irrespective of the load mode, schema evolution is important for the data ...
Read more >Advanced schema management for Spark applications at scale
Schema evolution management ... As new tables and views are created, and as table schemas evolve, schemas of views dependent on them also...
Read more >Schema Merging (Evolution) with Parquet in Spark and Hive
With schema evolution, one set of data can be stored in multiple files with different but compatible schema. In Spark, Parquet data source...
Read more >Schema evolution in parquet format - apache spark
Schema evolution can be (very) expensive. In order to figure out schema, you basically have to read all of your parquet files and...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
look like is hive side check,
hive.metastore.disallow.incompatible.col.type.changes
should be disable in you metastore@MihawkZoro The document seems to be a little problem, try
set hoodie.schema.on.read.enable=true