Crash when importing fiona after using sqlite3_modspatialite_init

Expected behavior and actual behavior.

On import of fiona in a particular script I get a crash:

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted (core dumped)

edit: (when building the PROJ library in the fiona wheel with debug flags, this turns into a segfault)

In my script I first load data through sqlalchemy and then output it through fiona. For this I need to enable spatialite extensions inside the sqlite3 driver, and this is where the cause of the error is. The issue only happens when importing fiona after loading the spatalite extension.

When I compile Fiona myself (against system GDAL 3.4.1) the issue vanishes.

Using GDB I can trace the crash back to the import of fiona.ogrext. This stack trace leads via the vendored PROJ to the system’s PROJ. (edited stack trace 9th Nov)

libproj.so.22!std::__shared_ptr<osgeo::proj::metadata::GeographicExtent, (__gnu_cxx::_Lock_policy)2>::operator=(std::__shared_ptr<osgeo::proj::metadata::GeographicExtent, (__gnu_cxx::_Lock_policy)2> * const this) (/usr/include/c++/11/bits/shared_ptr_base.h:1153)
libproj.so.22!std::shared_ptr<osgeo::proj::metadata::GeographicExtent>::operator=(std::shared_ptr<osgeo::proj::metadata::GeographicExtent> * const this) (/usr/include/c++/11/bits/shared_ptr.h:359)
libproj.so.22!dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::GeographicExtent> >::operator=(dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::GeographicExtent> > * const this) (/home/casper/code/proj/include/proj/nn.hpp:89)
libproj.so.22!std::__copy_move<false, false, std::random_access_iterator_tag>::__copy_m<dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::GeographicExtent> > const*, dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::GeographicExtent> >*>(const dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::GeographicExtent> > * __first, const dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::GeographicExtent> > * __last, dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::GeographicExtent> > * __result) (/usr/include/c++/11/bits/stl_algobase.h:385)
libproj-1ae2ebf9.so.22.2.1![Unknown/Just-In-Time compiled code] (Unknown Source:0)
libproj-1ae2ebf9.so.22.2.1!osgeo::proj::metadata::Extent::create(osgeo::proj::util::optional<std::string> const&, std::vector<dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::GeographicExtent> >, std::allocator<dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::GeographicExtent> > > > const&, std::vector<dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::VerticalExtent> >, std::allocator<dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::VerticalExtent> > > > const&, std::vector<dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::TemporalExtent> >, std::allocator<dropbox::oxygen::nn<std::shared_ptr<osgeo::proj::metadata::TemporalExtent> > > > const&) (Unknown Source:0)
libproj-1ae2ebf9.so.22.2.1!osgeo::proj::metadata::Extent::createFromBBOX(double, double, double, double, osgeo::proj::util::optional<std::string> const&) (Unknown Source:0)
libproj-1ae2ebf9.so.22.2.1![Unknown/Just-In-Time compiled code] (Unknown Source:0)
ld-linux-x86-64.so.2!call_init(struct link_map * l, int argc, char ** argv, char ** env) (dl-init.c:70)
(...)

Steps to reproduce the problem.

Preparation:

$ sudo apt install libsqlite3-mod-spatialite
$ pip install fiona

Actual script:

import sqlite3

con = sqlite3.connect("")
con.enable_load_extension(True)
con.execute("select load_extension('mod_spatialite', 'sqlite3_modspatialite_init')")

import fiona

Operating system

Ubuntu 22.04 LTS, with system python 3.10.6

Fiona and GDAL version and provenance

Used the wheel: Fiona-1.8.22-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Which has PROJ 8.2.1. My system PROJ is also 8.2.1.

[rouault]

in case you have seen this before or have an idea this could be an issue in PROJ or in our packaging

Well, may I say that pip packages embedding binaries is a broken concept by itself, especially on Linux where the dynamic loader doesn’t try to be “consistant” in such situation, and that this issue just shows it 😃 That said, to be more constructive, the issue could probably be worked around by building both PROJ and GDAL of the Fiona wheel with -DPROJ_RENAME_SYMBOLS -DPROJ_INTERNAL_CPP_NAMESPACE in CFLAGS and CXXFLAGS whose effect is to prefix PROJ C symbols with internal_ and change its C++ namespace to osgeo::internalproj Note: the above scenario would likely break when mixing mod_spatialite from the system and pyproj from pip as it can’t use that renaming trick

[elfjes]

As a workaround, I found that import order does make a difference. So make sure to import fiona or pyogrio before loading mod_spatialite and you should be fine. mod_spatialite either always uses the os version of PROJ or it is fine with fiona’s version, not sure which one it uses.