Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

sql worker on Ubuntu 18: failing, and failing JVM vendor check

See original GitHub issue

Describe the bug

On Ubuntu 18.04, using the pulsar 2.4.1 binary distribution, the sql worker doesn’t start.

After fixing python/python3 (#5369), an error is returned about the ELF format of file and nm say that the file is corrupt.

To Reproduce

$ bin/pulsar sql-worker run
ERROR: object '/home/ubuntu/apache-pulsar-2.4.1/lib/presto/bin/procname/Linux-x86_64/' from LD_PRELOAD cannot be preloaded (ELF file's phentsize not the expected size): ignored.
Presto requires an Oracle or OpenJDK JVM (found Private Build)
$ file lib/presto/bin/procname/Linux-x86_64/
lib/presto/bin/procname/Linux-x86_64/ ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), corrupted program header size, corrupted section header size
$ nm -D --defined-only lib/presto/bin/procname/Linux-x86_64/
nm: lib/presto/bin/procname/Linux-x86_64/ File format not recognized
$ ls -l lib/presto/bin/procname/Linux-x86_64/
-rw-r--r-- 1 ubuntu ubuntu 4700 Aug 28 09:06 lib/presto/bin/procname/Linux-x86_64/
$ shasum lib/presto/bin/procname/Linux-x86_64/
5849887050c21d07eb87b4cd513e9135de20942f  lib/presto/bin/procname/Linux-x86_64/

Desktop (please complete the following information):

  • OS: Ubuntu 18.04, openjdk-8-jre-headless:amd64 8u222-b10-1ubuntu1~18.04.1

Additional context I have checked that the extracted file I have matches the one in the tarball:

$ tar -tvzf apache-pulsar-2.4.1-bin.tar.gz | grep procname
drwxr-xr-x jia/staff         0 2019-08-28 09:06 apache-pulsar-2.4.1/lib/presto/bin/procname/
drwxr-xr-x jia/staff         0 2019-08-28 09:06 apache-pulsar-2.4.1/lib/presto/bin/procname/Linux-x86_64/
drwxr-xr-x jia/staff         0 2019-08-28 09:06 apache-pulsar-2.4.1/lib/presto/bin/procname/Linux-ppc64le/
-rw-r--r-- jia/staff      4700 2019-08-28 09:06 apache-pulsar-2.4.1/lib/presto/bin/procname/Linux-x86_64/
-rw-r--r-- jia/staff     70397 2019-08-28 09:06 apache-pulsar-2.4.1/lib/presto/bin/procname/Linux-ppc64le/

4700 bytes does seem rather small for an .so, but not impossibly so. Indeed, if I separately download presto-server-3.2.0.tar.gz (895MB!) it contains a smaller file:

$ tar -tvzf presto-server-320.tar.gz | grep \\.so
-rw-r--r--  0 0      0        4144  6 Oct 20:27 presto-server-320/bin/procname/Linux-x86_64/
-rw-r--r--  0 0      0       69576  6 Oct 20:27 presto-server-320/bin/procname/Linux-ppc64le/

If I examine this file, it is not considered corrupt by Ubuntu:

$ file ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, stripped
$ nm -D --defined-only
0000000000200838 A __bss_start
0000000000200838 A _edata
0000000000200848 A _end
00000000000005c8 T _fini
0000000000000428 T _init

If I use this to replace the one supplied by pulsar, the original ELF message goes away, but the sql worker still won’t start, continuing to complain about JVM version.

$ bin/pulsar sql-worker run
Presto requires an Oracle or OpenJDK JVM (found Private Build)

However, I am using OpenJDK:

$ dpkg-query -l | grep openjdk
ii  openjdk-8-jre-headless:amd64   8u222-b10-1ubuntu1~18.04.1         amd64        OpenJDK Java runtime, using Hotspot JIT (headless)

That error message appears in older versions of Presto source, but was removed in this commit in Aug 2018. Could it simply be that pulsar is bundling an obsolete version of presto?

I believe the source of the “Private Build” text is here:

$ java -XshowSettings 2>&1 | grep vendor
    java.specification.vendor = Oracle Corporation
    java.vendor = Private Build
    java.vendor.url =
    java.vendor.url.bug =
    java.vm.specification.vendor = Oracle Corporation
    java.vm.vendor = Private Build

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:9 (7 by maintainers)

github_iconTop GitHub Comments

Antti-Kaikkonencommented, Jan 7, 2020

I got it working with OpenJDK 8 by running ./bin/pulsar sql-worker run -D "java.vendor"="Oracle Corporation" Also tested same with OpenJDK 11 but it failed with unrelated nullpointer exception.

tisonkuncommented, Nov 14, 2022

Closed as stale. Please open a new issue if it’s still relevant in maintained versions.

Read more comments on GitHub >

github_iconTop Results From Across the Web

"#!/usr/bin/env python" doesn't work on Ubuntu 18.04 / python3
To Reproduce On Ubuntu 18.04, try to start the sql worker: $ bin/pulsar ... sql worker on Ubuntu 18: failing, and failing...
Read more >
starting SQL Server 2019 failure after shutdown and start ...
All are smoothly until I shutdown my VM, and start VM, the mssql-server start failure. The check command and error message are as...
Read more >
Unable to connect to SQL server instance ubuntu 18.04
So the reason for this error was getting was because I messed up with the file ownership.. running the following command fixed it...
Read more >
Fiji with Ubuntu 18.04 and OpenJDK 10 won't start up with the ...
You are trying to run Fiji with the JRE that was shipped with Fiji, but Fiji tries to fall back to system Java...
Read more >
Elasticsearch failed to start on Ubuntu 18.04 - Elastic Discuss
Did you change the memory options? If so, check that you don't use more than 4gb of HEAP and that this memory is...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found