Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

upgrading from 1.0.6 to 1.1.2 fails for partitioned table

See original GitHub issue

CrateDB version:1.1.2

JVM version:docker

OS version / environment description:docker

Problem description:

[/data/zelos-crate/nodes/0/indices/.partitioned.sometable.04732d1n68r3ge1g60o30c1g/1] please check and remove them if possible
	at io.crate.metadata.doc.CrateMetaDataUpgradeService.upgradeIndexMetaData(CrateMetaDataUpgradeService.java:76)
	at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:232)
	at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:89)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:50)
	at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
	at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
	at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
	at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:886)
	at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
	at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
	at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
	at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:201)
	at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
	at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:879)
	at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
	at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
	at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
	at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:96)
	at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
	at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:46)
	at org.elasticsearch.node.Node.<init>(Node.java:213)
	at io.crate.node.CrateNode.<init>(CrateNode.java:50)
	at org.elasticsearch.bootstrap.BootstrapProxy.setup(BootstrapProxy.java:193)
	at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:300)
	at io.crate.bootstrap.CrateDB.main(CrateDB.java:50)
Caused by: java.lang.IllegalStateException: [[.partitioned.sometable.04732d1n68r3ge1g60o30c1g][1]] no shard state found in any of: [/data/zelos-crate/nodes/0/indices/.partitioned.sometable.04732d1n68r3ge1g60o30c1g/1] please check and remove them if possible
	at io.crate.metadata.doc.CrateMetaDataUpgradeService.upgradeShardMetaData(CrateMetaDataUpgradeService.java:198)
	at io.crate.metadata.doc.CrateMetaDataUpgradeService.upgradeIndexMapping(CrateMetaDataUpgradeService.java:112)
	at io.crate.metadata.doc.CrateMetaDataUpgradeService.upgradeIndexMetaData(CrateMetaDataUpgradeService.java:73)
	... 28 more

table: doc.sometable shards: 36 docs: 23285982 size: 4459487836

+-------------+---------------+--------------------+----------------+----------------+----------------+------+-------------------------------------------------+-------------------------------------------------------------------------------+
| name        | crate_version |          memory_gb |   heap_size_gb |   disk_size_gb | max_open_files | cpus | uname                                           | jvm                                                                           |
+-------------+---------------+--------------------+----------------+----------------+----------------+------+-------------------------------------------------+-------------------------------------------------------------------------------+
| zelos1crate | 1.0.6         | 15.406768798828125 | 3.821044921875 | 460.7607421875 |        1048576 |    4 | Linux 007da64bf71a 3.10.0-514.el7.x86_64; amd64 | Java 1.8.0_121 Oracle Corporation OpenJDK 64-Bit Server VM (build 25.121-b13) |
| zelos2crate | 1.0.6         | 15.406768798828125 | 3.821044921875 | 460.7607421875 |        1048576 |    4 | Linux e644453c6d47 3.10.0-514.el7.x86_64; amd64 | Java 1.8.0_121 Oracle Corporation OpenJDK 64-Bit Server VM (build 25.121-b13) |
| zelos3crate | 1.0.6         | 15.406768798828125 | 3.821044921875 | 460.7607421875 |        1048576 |    4 | Linux de03eb1f2ebb 3.10.0-514.el7.x86_64; amd64 | Java 1.8.0_121 Oracle Corporation OpenJDK 64-Bit Server VM (build 25.121-b13) |
| zelos4crate | 1.0.6         | 15.406768798828125 | 3.821044921875 | 460.7607421875 |        1048576 |    4 | Linux decc74e53c2f 3.10.0-514.el7.x86_64; amd64 | Java 1.8.0_121 Oracle Corporation OpenJDK 64-Bit Server VM (build 25.121-b13) |
| zelos5crate | 1.0.6         | 15.406768798828125 | 3.821044921875 | 460.7607421875 |        1048576 |    4 | Linux 69d4f25db19a 3.10.0-514.el7.x86_64; amd64 | Java 1.8.0_121 Oracle Corporation OpenJDK 64-Bit Server VM (build 25.121-b13) |
| zelos6crate | 1.0.6         | 15.406768798828125 | 3.821044921875 | 460.7607421875 |        1048576 |    4 | Linux dd506673414e 3.10.0-514.el7.x86_64; amd64 | Java 1.8.0_121 Oracle Corporation OpenJDK 64-Bit Server VM (build 25.121-b13) |
| zelos7crate | 1.0.6         | 15.406768798828125 | 3.821044921875 | 460.7607421875 |        1048576 |    4 | Linux ce5a3557834c 3.10.0-514.el7.x86_64; amd64 | Java 1.8.0_121 Oracle Corporation OpenJDK 64-Bit Server VM (build 25.121-b13) |
+-------------+---------------+--------------------+----------------+----------------+----------------+------+-------------------------------------------------+-------------------------------------------------------------------------------+
SELECT 7 rows in set (0.043 sec)

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
matrivcommented, Apr 21, 2017

@sonix07 as a conclusion for our direct discussion,

The issue you encounter is caused because you tried to perform a rolling upgrade. Unfortunately rolling upgrade is not supported for 1.0.x -> 1.1.2 as you can see also here, so please do an upgrade with full cluster shutdown as suggested.

Your idea for preventing nodes with incompatible versions to join each other in a cluster is very good and has been added to our backlog, thanks!

0reactions
matrivcommented, Apr 20, 2017

Thanks for the info, we’ll try to reproduce the issue.

Read more comments on GitHub >

github_iconTop Results From Across the Web

“We couldn't update system reserved partition” error installing ...
Press the Windows key + R. In the Run window that comes up, type diskmgmt.msc and press Enter. Select the partition marked System...
Read more >
Bug listing with status RESOLVED with resolution FIXED as at ...
... Bug:241 - "nautilus-1.0.6-r1 fails to build" status:RESOLVED resolution:FIXED ... Bug:357 - "Save partition table to file on boot" status:RESOLVED ...
Read more >
Maintenance Operations for Partitioned Tables and Indexes
With asynchronous global index maintenance, a drop partition update indexes operation is on metadata only and all global indexes remain valid. Dropping a ......
Read more >
pg_partman/CHANGELOG.txt at master - GitHub
Running these functions will now move data out of the default partition and into the proper child tables, creating them if necessary (Github...
Read more >
Changelog - Confluent Documentation
Version 1.1.2¶. PR-151/CCDB-3993: Fixes a bug that blocks the connector forever when the partitioned snapshot option is enabled on non-partitioned tables ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Query fails with EOFException after upgrading to 1.1.2

Next page

Query performance