Unable to build leveldb on Ubuntu 18.04 with libsnappy-dev installed: python setup_leveldb.py fails with error db/snappy_compressor.cc:6:10: fatal error: snappy/snappy.h: No such file or directory

Description of Issue

Trying to build leveldb on Ubuntu 18.04 with libsnappy-dev package installed, using setup_leveldb.py on pymclevel directory fails with error db/snappy_compressor.cc:6:10: fatal error: snappy/snappy.h: No such file or directory. Searching for snappy.h in apt repositories shows libsnappy-dev in results, trying to install it shows that it’s already installed. Leveldb builds fine in a schroot profile w/o libsnappy libs installed.

Update: This issue might be caused be leveldb upstream bug/outdated commit as the file that doesn’t compile is within the directory created while running the script, cleaning directory tree using git (running git clean -d -x -i) doen’t help much. Update2: symlinking snappy/snappy.h to snappy.h doesn’t help much, fails with other error later.

Symlinking snappy header and rebuilding, still resulting in error

Symlinking snappy

sudo mkdir snappy
cd snappy
sudo ln -s ../snappy.h snappy.h

Error while trying to rebuild

g++ -pthread -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG table/table_test.cc out-static/db/builder.o out-static/db/c.o out-static/db/dbformat.o out-static/db/db_impl.o out-static/db/db_iter.o out-static/db/dumpfile.o out-static/db/filename.o out-static/db/log_reader.o out-static/db/log_writer.o out-static/db/memtable.o out-static/db/repair.o out-static/db/snappy_compressor.o out-static/db/table_cache.o out-static/db/version_edit.o out-static/db/version_set.o out-static/db/write_batch.o out-static/db/zlib_compressor.o out-static/db/zopfli_compressor.o out-static/db/zstd_compressor.o out-static/table/block_builder.o out-static/table/block.o out-static/table/filter_block.o out-static/table/format.o out-static/table/iterator.o out-static/table/merger.o out-static/table/table_builder.o out-static/table/table.o out-static/table/two_level_iterator.o out-static/util/arena.o out-static/util/bloom.o out-static/util/cache.o out-static/util/coding.o out-static/util/comparator.o out-static/util/crc32c.o out-static/util/env_boost.o out-static/util/env.o out-static/util/env_posix.o out-static/util/env_win.o out-static/util/env_winrt.o out-static/util/filter_policy.o out-static/util/hash.o out-static/util/histogram.o out-static/util/logging.o out-static/util/options.o out-static/util/status.o out-static/util/win_logger.o out-static/port/port_posix.o out-static/port/port_posix_sse.o out-static/util/testharness.o out-static/util/testutil.o -o out-static/table_test -lsnappy /lib/x86_64-linux-gnu/libz.so.1.2.11
table/table_test.cc: In function ‘bool leveldb::SnappyCompressionSupported()’:
table/table_test.cc:834:16: error: ‘Snappy_Compress’ is not a member of ‘leveldb::port’
   return port::Snappy_Compress(in.data(), in.size(), &out);
                ^~~~~~~~~~~~~~~
Makefile:371: recipe for target 'out-static/table_test' failed
make: *** [out-static/table_test] Error 1

Update3: Created an issue in upstream git: Mojang/leveldb-mcpe#25 Update4: Leveldb builds in schroot profile w/o libsnappy libs installed, ~but mcedit doesn’t load, it shows error about missing leveldbpocket python library, both in host and schroot environments.~ Update5: Disregard above, fixed by running git reset --hard (error caused by accidentally deleting wrong files)

MCEdit Version

MCEdit-Unified v1.5.6.0, Commit bd5d10dd232c40f089cb119f2fe36785e633a642

Minecraft Version the world was last opened in

N/A

Operating System

Ubuntu Linux 18.04 (bionic) 64 Bits

A screenshot or full text of the error from the console

Full terminal output including build log with error messages included

juozas@xubuntu:~/MCEdit/MCEdit-Unified/pymclevel$ LC_ALL=C python setup_leveldb.py
========================================================================
Building Linux Minecraft Pocket Edition for MCEdit...
-----------------------------------------------------
Searching for the needed binaries (gcc, g++, unzip, wget|curl)...
g_name wget_curl True
All the needed binaries were found.
Downloading sources for leveldb
URL: https://codeload.github.com/Mojang/leveldb-mcpe/zip/5722a489c0fabf70f7bb36f70adc2ac70ff90377
Unpacking leveldb
Cleaning archive.
Downloading sources for zlib
URL: https://codeload.github.com/madler/zlib/zip/4a090adef8c773087ec8916ad3c2236ef560df27
Unpacking zlib
Cleaning archive.
Checking zlib.
Scanning /usr/lib/x86_64-linux-gnu/libfakeroot for libz.so.1.2.8
Scanning /usr/lib/x86_64-linux-gnu for libz.so.1.2.8
Scanning /lib/x86_64-linux-gnu for libz.so.1.2.8
Scanning /usr/lib64 for libz.so.1.2.8
Scanning /lib64 for libz.so.1.2.8
Scanning /lib for libz.so.1.2.8
Scanning /usr/lib for libz.so.1.2.8
Scanning /usr/local/lib for libz.so.1.2.8
Scanning /home/juozas/.local/lib for libz.so.1.2.8
Scanning /usr/local/lib for libz.so.1.2.8
Scanning /lib32 for libz.so.1.2.8
Scanning /usr/lib32 for libz.so.1.2.8
Scanning . for libz.so.1.2.8
Scanning /lib/i386-linux-gnu for libz.so.1.2.8
Scanning /usr/lib/i386-linux-gnu for libz.so.1.2.8
Scanning /lib32 for libz.so.1.2.8
Scanning /usr/lib32 for libz.so.1.2.8
Scanning /libx32 for libz.so.1.2.8
Scanning /usr/libx32 for libz.so.1.2.8
Scanning /usr/lib/x86_64-linux-gnu/libfakeroot for libz.so.1.2
Scanning /usr/lib/x86_64-linux-gnu for libz.so.1.2
Scanning /lib/x86_64-linux-gnu for libz.so.1.2
Scanning /usr/lib64 for libz.so.1.2
Scanning /lib64 for libz.so.1.2
Scanning /lib for libz.so.1.2
Scanning /usr/lib for libz.so.1.2
Scanning /usr/local/lib for libz.so.1.2
Scanning /home/juozas/.local/lib for libz.so.1.2
Scanning /usr/local/lib for libz.so.1.2
Scanning /lib32 for libz.so.1.2
Scanning /usr/lib32 for libz.so.1.2
Scanning . for libz.so.1.2
Scanning /lib/i386-linux-gnu for libz.so.1.2
Scanning /usr/lib/i386-linux-gnu for libz.so.1.2
Scanning /lib32 for libz.so.1.2
Scanning /usr/lib32 for libz.so.1.2
Scanning /libx32 for libz.so.1.2
Scanning /usr/libx32 for libz.so.1.2
Scanning /usr/lib/x86_64-linux-gnu/libfakeroot for libz.so.1
Scanning /usr/lib/x86_64-linux-gnu for libz.so.1
Scanning /lib/x86_64-linux-gnu for libz.so.1
('/lib/x86_64-linux-gnu/libz.so.1.2.11', True, '1.2.11')
*** WARNING: zlib was found, but its version is 1.2.11.
             You can try to build with this version, but it may fail,
             or the generated libraries may not work...
             Enter 'b' to build zlib v1.2.8 only for leveldb.
             Enter 'a' to quit now and install zlib yourself.
             Enter 'c' to continue.
Build continues with zlib v1.2.11
Building leveldb...
mkdir out-shared
mkdir out-shared/db
mkdir out-shared/port
mkdir out-shared/table
mkdir out-shared/util
mkdir -p out-shared/helpers/memenv
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/builder.cc -o out-shared/db/builder.o
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/c.cc -o out-shared/db/c.o
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/db_impl.cc -o out-shared/db/db_impl.o
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/db_iter.cc -o out-shared/db/db_iter.o
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/dbformat.cc -o out-shared/db/dbformat.o
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/dumpfile.cc -o out-shared/db/dumpfile.o
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/filename.cc -o out-shared/db/filename.o
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/log_reader.cc -o out-shared/db/log_reader.o
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/log_writer.cc -o out-shared/db/log_writer.o
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/memtable.cc -o out-shared/db/memtable.o
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/repair.cc -o out-shared/db/repair.o
g++ -DDLLX= -I. -I./include -std=c++0x -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -DSNAPPY -O2 -DNDEBUG -fPIC -c db/snappy_compressor.cc -o out-shared/db/snappy_compressor.o
db/snappy_compressor.cc:6:10: fatal error: snappy/snappy.h: No such file or directory
 #include <snappy/snappy.h>
          ^~~~~~~~~~~~~~~~~
compilation terminated.
Makefile:414: recipe for target 'out-shared/db/snappy_compressor.o' failed
make: *** [out-shared/db/snappy_compressor.o] Error 1
PE support build failed.
juozas@xubuntu:~/MCEdit/MCEdit-Unified/pymclevel$ LC_ALL=C apt-file search snappy.h #search for snappy.h in apt repositories
libsnappy-dev: /usr/include/snappy.h
regina-normal-doc: /usr/share/doc/regina-normal-doc/docs/en/regina/python-snappy.html
thunderbird-dev: /usr/include/thunderbird/snappy/snappy.h
juozas@xubuntu:~/MCEdit/MCEdit-Unified/pymclevel$ LC_ALL=C sudo apt-get install libsnappy-dev #try to install libsnappy-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libsnappy-dev is already the newest version (1.1.7-1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
juozas@xubuntu:~/MCEdit/MCEdit-Unified/pymclevel$ LC_ALL=C lsb_release -a #full distro info
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04 LTS
Release:	18.04
Codename:	bionic
juozas@xubuntu:~/MCEdit/MCEdit-Unified/pymclevel$ grep -ir snappy.h * #find all references to snappy.h in mcedit dir
leveldb/port/port_winrt.cc:#include <snappy/snappy.h>
leveldb/port/port_posix.h:#include <snappy.h>
leveldb/port/port_wp8.cc:#include <snappy/snappy.h>
leveldb/port/port_win.cc:	#include <snappy/snappy.h>
leveldb/db/snappy_compressor.cc:#include <snappy/snappy.h>
leveldb/build_detect_platform:      #include <snappy.h>

How to reproduce

1. Install/upgrade to ubuntu 18.04
2. Get the latest version (bd5d10dd232c40f089cb119f2fe36785e633a642) of mcedit using git
3. Install required dependencies to build leveldb using ubuntu package manager, make sure libsnappy-dev is installed
4. (optional) Run mcedit
5. Change Directory to pymclevel in mcedit root dir
6. (optional) run python setup.py build_ext --inplace
7. run python setup_leveldb.py
8. See an error near the end of build

[juozaspo]

You didn’t install libsnappy-dev, the problem happens only when libsnappy-dev is installed, read the title and updated first post for more details. In leveldb/build_detect_platform:

# Test whether Snappy library is installed
    # http://code.google.com/p/snappy/
    $CXX $CXXFLAGS -x c++ - -o $CXXOUTPUT 2>/dev/null  <<EOF
      #include <snappy.h>
      int main() {}
EOF
    if [ "$?" = 0 ]; then
        COMMON_FLAGS="$COMMON_FLAGS -DSNAPPY"
        PLATFORM_LIBS="$PLATFORM_LIBS -lsnappy"
    fi

in leveldb/build_config.mk line 1:

SOURCES=db/builder.cc db/c.cc db/dbformat.cc db/db_impl.cc db/db_iter.cc db/dumpfile.cc db/filename.cc db/log_reader.cc db/log_writer.cc db/memtable.cc db/repair.cc db/snappy_compressor.cc db/table_cache.cc db/version_edit.cc db/version_set.cc db/write_batch.cc db/zlib_compressor.cc db/zopfli_compressor.cc db/zstd_compressor.cc table/block_builder.cc table/block.cc table/filter_block.cc table/format.cc table/iterator.cc table/merger.cc table/table_builder.cc table/table.cc table/two_level_iterator.cc util/arena.cc util/bloom.cc util/cache.cc util/coding.cc util/comparator.cc util/crc32c.cc util/env_boost.cc util/env.cc util/env_posix.cc util/env_win.cc util/env_winrt.cc util/filter_policy.cc util/hash.cc util/histogram.cc util/logging.cc util/options.cc util/status.cc util/win_logger.cc port/port_posix.cc port/port_posix_sse.cc

In leveldb/db/snappy_compressor.cc: Line 2: #ifdef SNAPPY Line 6: #include <snappy/snappy.h> Line 30: #endif This means if libsnappy-dev is detected it attempts to include snappy/snappy.h which doesn’t exist in my installation nor is included in package. When libsnappy-dev is installed, theres /usr/include/snappy.h but not /usr/include/snappy/snappy.h Workaround:

cd /usr/include #or other path containing snappy.h, included in the PATH environment variable
sudo mkdir snappy
cd snappy
sudo ln -s ../snappy.h snappy.h

The contents of leveldb directory is refering to commit (Mojang/leveldb-mcpe@5722a489c0fabf70f7bb36f70adc2ac70ff90377) on Mojang repository, this should be updated when it gets fixed upstream. Asuming libsnappy-dev is already installed, uninstall it to build sucessfully or use seperate build environment such as schroot to do the same. Also check Mojang/leveldb-mcpe#25, since this is an upstream issue, this needs to be fixed there first then pymclevel build script needs to be changed to adapt to updated upstream code.

[juozaspo]

The library development package were installed when I wrote the issue. Only later I found it’s upstream problem and related to libleveldb-dev not just libleveldb. It builds fine with libleveldb, it’s the development package that’s causing it when it’s installed. By default build script uses commit on Mojang repository, that’s why I created issue there. For future reference you should refer to upstream issue and the build script should be updated when that issue gets fixed. There’s no need to change anything in build-script itself until upstream gets fixed, only change it to refer to updated commit when original code builds correctly.