google provider requires leveldb installed in system
See original GitHub issuegoogle provider now requires plyvel, which is used by leveldb hook.
to install plyvel, the user needs leveldb headers installed in the system, otherwise it will fail:
plyvel/_plyvel.cpp:632:10: fatal error: 'leveldb/db.h' file not found
#include "leveldb/db.h"
^~~~~~~~~~~~~~
1 error generated.
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for plyvel
the average GCP user may not use leveldb and it seems a little burdernsome to require its installation just to use GCP hooks, when most of them don’t need this library.
perhaps there is a way to make this dependency optional?
or perhaps is there a way to install only certain components of a provider?
update
after looking more closely, i see that levelDB isn’t a GCP service
it seems to me levelDB might belong as a distinct provider, e.g. google-gcp
vs google-level
like there is currently with apache and microsoft? @potiuk any thoughts?
cc: @mik-laj related: https://github.com/apache/airflow/pull/14105
Issue Analytics
- State:
- Created 2 years ago
- Comments:18 (17 by maintainers)
Top GitHub Comments
Yeah. As mentioned in #15933 the right solution to your problem is to upgrade to Airflow 2.1+.
Why not - will you make PR for that @javatarz ? I kept on using devel anyway.