upgrade to Hadoop 3.1.1 to be compatible with current HDP version
See original GitHub issueCurrently, the latest version of Hadoop supported seems to be 2.7.1, so Halyard 2.2 will not work with the latest version 3.1 of Hadoop which is included in the latest HDP 3.0.1 (released already last November). This makes initial setup, testing and adoption unnecessarily hard, as you have to dig out an old version from the archive. Newer HDP versions also include Docker images which would provide an additional benefit for prototyping and quick deployment. See: https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/release-notes/content/comp_versions.html
It may be because Halyard 2.2 still uses deprecated methods (see #6), or because of the major version change, didn’t check closely.
So, creating the repository works, but opening fails:
> create repository
No template called repository found in /root/.RDF4J/console/templates
> create hbase
Please specify values for the following variables:
Repository ID: halyardproto
Repository title: Halyard Prototype
HBase Table Name:
Create HBase Table if missing (true|false) [true]:
HBase Table presplit bits [0]:
Use Halyard Push Evaluation Strategy (true|false) [true]:
Query Evaluation Timeout [180]:
Optional ElasticSearch Index URL:
Repository created
> open halyardproto
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Admin.createTable(Lorg/apache/hadoop/hbase/HTableDescriptor;[[B)V
at com.msd.gin.halyard.common.HalyardTableUtils.getTable(HalyardTableUtils.java:166)
at com.msd.gin.halyard.common.HalyardTableUtils.getTable(HalyardTableUtils.java:144)
at com.msd.gin.halyard.sail.HBaseSail.initialize(HBaseSail.java:193)
at org.eclipse.rdf4j.repository.sail.SailRepository.initializeInternal(SailRepository.java:151)
at org.eclipse.rdf4j.repository.base.AbstractRepository.initialize(AbstractRepository.java:34)
at org.eclipse.rdf4j.repository.manager.LocalRepositoryManager.createRepository(LocalRepositoryManager.java:273)
at org.eclipse.rdf4j.repository.manager.RepositoryManager.getRepository(RepositoryManager.java:424)
at org.eclipse.rdf4j.console.command.Open.openRepository(Open.java:80)
at org.eclipse.rdf4j.console.command.Open.execute(Open.java:65)
at org.eclipse.rdf4j.console.Console.executeCommand(Console.java:376)
at org.eclipse.rdf4j.console.Console.start(Console.java:337)
at org.eclipse.rdf4j.console.Console.main(Console.java:239)
Issue Analytics
- State:
- Created 5 years ago
- Comments:6
Top Results From Across the Web
Solved: Hadoop 3.1.1 fixes in HDP 3.0 - Cloudera Community
I would like to know if these fixes (3.1.1 release) are already included with HDP 3.0 when I install it via Ambari-2.7.0.0? If...
Read more >Apache Hadoop 3.3.4 – HDFS Rolling Upgrade
Introduction. HDFS rolling upgrade allows upgrading individual HDFS daemons. For examples, the datanodes can be upgraded independent of the ...
Read more >HDP 2.6.4 to HDP 3.1.0.0 - IBM
Supports upgrading only from HDP 2.6.4 to HDP 3.1.0.0 version and not to a ... versions are compatible with the HDP, Mpack and...
Read more >Isilon: Upgrade Hortonworks HDP2.6.5 to HDP3.* on Dell ...
Prerequisites · Hadoop cluster running HDP 2.6.5 and Ambari Server 2.6.2.2. · DellEMC PowerScale OneFS updated to 8.1.2 and patch 240163 installed ...
Read more >Apache Spark 2.3.1 compatibility with Hadoop 3.0 in HDP 3.0
I am plannig to upgrade from Hortonworks Data platform[HDP] (version 2.6.x) to HDP 3.0. But, there seems to be some major bugs in...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Halyard 3.0 has been released with HDP 3.1 support.
Preliminary testing progress of the 3.0 nightly build on HDP 3.1 seems to be quite promising 😃 Please report any inconveniences observed.