Comprehensive list of broken python3 recipes

Just like it was done before for python2 and python3crystax, we need a list of broken python3 recipes https://github.com/kivy/python-for-android/issues/1402

[RobertFlatt]

A little more detail…

Build environment:

sudo apt-get install -y autoconf libtool cmake libffi-dev pkg-config libjpeg9-dev python3-pil Using python-for-android - master, installed 1/19/2019

.p4a

--dist_name=tstlib
--private .
--package=com.example.tstlib
--name Tstlib
--arch=armeabi-v7a
--sdk_dir /home/bobf/androidtools/sdk
--ndk_dir /home/bobf/androidtools/android-ndk-r17c
--ndk_version 17c
--android_api 28
--ndk-api 21
--version 0.0.0

Build command: p4a apk --requirements=<module>,python3,kivy Resulting module builds can be divided into four groups:

1. Modules that build and APK contains object files (complete set ?)
2. Modules that build and APK contains no extra files (silent fails)
3. Modules that do not build APK because of dependency on ‘python2’, etc.
4. Modules that do not build APK because of other dependency.

1. Contains object files:

['Pillow', 'audiostream', 'cffi', 'cryptography', 'enum34', 'ffmpeg',
'ffpyplayer_codecs', 'freetype', 'harfbuzz', 'idna', 'ipaddress', 'jedi',
'jpeg', 'leveldb', 'libcurl', 'libexpat', 'libffi', 'libiconv', 'libogg',
'libsecp256k1', 'libshine',  'libx264', 'libxml2', 'libxslt','pycrypto',
'lxml','netifaces','numpy', 'openal','openssl', 'pbkdf2', 'png', 'pycparser',
'pycryptodome', 'pyjnius','pymunk', 'pysdl2','reportlab','requests', 'sdl2_image',
'sdl2_mixer','sdl2_ttf', 'six', 'snappy', 'spine','sqlite3']

These respectively contain:

Pillow : site-packages/PIL/_imaging.so site-packages/PIL/_imagingft.so site-packages/PIL/_imagingmath.so site-packages/PIL/_imagingmorph.so audiostream : site-packages/audiostream/core.so site-packages/audiostream/platform/plat_android.so site-packages/audiostream/sources/puredata.so site-packages/audiostream/sources/thread.so site-packages/audiostream/sources/wave.so cffi : modules/_ctypes.cpython-37m.so site-packages/_cffi_backend.so cryptography : modules/_ctypes.cpython-37m.so modules/_hashlib.cpython-37m.so modules/_ssl.cpython-37m.so site-packages/cryptography/hazmat/bindings/_constant_time.so site-packages/cryptography/hazmat/bindings/_openssl.so site-packages/cryptography/hazmat/bindings/_padding.so site-packages/_cffi_backend.so libcurl : modules/_hashlib.cpython-37m.so modules/_ssl.cpython-37m.so libffi : modules/_ctypes.cpython-37m.so lxml : site-packages/lxml/builder.so site-packages/lxml/etree.so site-packages/lxml/html/clean.so site-packages/lxml/html/diff.so site-packages/lxml/objectify.so site-packages/lxml/_elementpath.so netifaces : site-packages/netifaces.so
numpy : site-packages/numpy/core/multiarray.so site-packages/numpy/core/umath.so site-packages/numpy/core/_dummy.so site-packages/numpy/core/_multiarray_tests.so site-packages/numpy/core/_operand_flag_tests.so site-packages/numpy/core/_rational_tests.so site-packages/numpy/core/_struct_ufunc_tests.so site-packages/numpy/core/_umath_tests.so site-packages/numpy/fft/fftpack_lite.so site-packages/numpy/linalg/lapack_lite.so site-packages/numpy/linalg/_umath_linalg.so site-packages/numpy/random/mtrand.so openssl : modules/_hashlib.cpython-37m.so modules/_ssl.cpython-37m.so pycrypto : modules/_hashlib.cpython-37m.so modules/_ssl.cpython-37m.so site-packages/Crypto/Cipher/_AES.so site-packages/Crypto/Cipher/_ARC2.so site-packages/Crypto/Cipher/_ARC4.so site-packages/Crypto/Cipher/_Blowfish.so site-packages/Crypto/Cipher/_CAST.so site-packages/Crypto/Cipher/_DES.so site-packages/Crypto/Cipher/_DES3.so site-packages/Crypto/Cipher/_XOR.so site-packages/Crypto/Hash/MD2.so site-packages/Crypto/Hash/MD4.so site-packages/Crypto/Hash/RIPEMD160.so site-packages/Crypto/Hash/SHA224.so site-packages/Crypto/Hash/SHA256.so site-packages/Crypto/Hash/SHA384.so site-packages/Crypto/Hash/SHA512.so site-packages/Crypto/Util/cpuid.so site-packages/Crypto/Util/galois.so site-packages/Crypto/Util/strxor.so site-packages/Crypto/Util/_counter.so site-packages/netifaces.so pycryptodome : modules/_ctypes.cpython-37m.so site-packages/Crypto/Cipher/_ARC4.so site-packages/Crypto/Cipher/_chacha20.so site-packages/Crypto/Cipher/_raw_aes.so site-packages/Crypto/Cipher/_raw_arc2.so site-packages/Crypto/Cipher/_raw_blowfish.so site-packages/Crypto/Cipher/_raw_cast.so site-packages/Crypto/Cipher/_raw_cbc.so site-packages/Crypto/Cipher/_raw_cfb.so site-packages/Crypto/Cipher/_raw_ctr.so site-packages/Crypto/Cipher/_raw_des.so site-packages/Crypto/Cipher/_raw_des3.so site-packages/Crypto/Cipher/_raw_ecb.so site-packages/Crypto/Cipher/_raw_ocb.so site-packages/Crypto/Cipher/_raw_ofb.so site-packages/Crypto/Cipher/_Salsa20.so site-packages/Crypto/Hash/_BLAKE2b.so site-packages/Crypto/Hash/_BLAKE2s.so site-packages/Crypto/Hash/_keccak.so site-packages/Crypto/Hash/_MD2.so site-packages/Crypto/Hash/_MD4.so site-packages/Crypto/Hash/_RIPEMD160.so site-packages/Crypto/Hash/_SHA224.so site-packages/Crypto/Hash/_SHA256.so site-packages/Crypto/Hash/_SHA384.so site-packages/Crypto/Hash/_SHA512.so site-packages/Crypto/Protocol/_scrypt.so site-packages/Crypto/Util/_cpuid.so site-packages/Crypto/Util/_galois.so site-packages/Crypto/Util/_strxor.so site-packages/_cffi_backend.so pymunk : modules/_ctypes.cpython-37m.so site-packages/pymunk/libchipmunk.so site-packages/_cffi_backend.so reportlab : site-packages/numpy/core/multiarray.so site-packages/numpy/core/umath.so site-packages/numpy/core/_dummy.so site-packages/numpy/core/_multiarray_tests.so site-packages/numpy/core/_operand_flag_tests.so site-packages/numpy/core/_rational_tests.so site-packages/numpy/core/_struct_ufunc_tests.so site-packages/numpy/core/_umath_tests.so site-packages/numpy/fft/fftpack_lite.so site-packages/numpy/linalg/lapack_lite.so site-packages/numpy/linalg/_umath_linalg.so site-packages/numpy/random/mtrand.so site-packages/reportlab/graphics/_renderPM.so site-packages/reportlab/lib/_rl_accel.so
spine : site-packages/spine/animation/animation.so site-packages/spine/animation/animationstate.so site-packages/spine/animation/attachmenttimeline.so site-packages/spine/animation/colortimeline.so site-packages/spine/animation/curvetimeline.so site-packages/spine/animation/drawordertimeline.so site-packages/spine/animation/eventtimeline.so site-packages/spine/animation/ffdtimeline.so site-packages/spine/animation/flipxtimeline.so site-packages/spine/animation/flipytimeline.so site-packages/spine/animation/ikconstrainttimeline.so site-packages/spine/animation/rotatetimeline.so site-packages/spine/animation/scaletimeline.so site-packages/spine/animation/timeline.so site-packages/spine/animation/trackentry.so site-packages/spine/animation/translatetimeline.so site-packages/spine/attachment/attachment.so site-packages/spine/attachment/boundingboxattachment.so site-packages/spine/attachment/meshattachment.so site-packages/spine/attachment/regionattachment.so site-packages/spine/attachment/skinnedmeshattachment.so site-packages/spine/bone.so site-packages/spine/ikconstraint.so site-packages/spine/skeleton/skeleton.so site-packages/spine/skeleton/skeletonbounds.so site-packages/spine/skeleton/skeletondata.so site-packages/spine/slot.so site-packages/spine/spineevent.so site-packages/spine/utils.so sqlite3 : modules/_sqlite3.cpython-37m.so

1. Silent fails:

['enum34', 'ffmpeg', 'ffpyplayer_codecs', 'freetype', 'harfbuzz', 'idna', 'ipaddress', 'jedi', 'jpeg', 'leveldb', 'libexpat', 'libiconv', 'libogg', 'libsecp256k1', 'libshine', 'libx264', 'libxml2', 'libxslt', 'openal', 'pbkdf2', 'png', 'pycparser', 'pyjnius', 'pysdl2', 'requests', 'sdl2_image', 'sdl2_mixer', 'sdl2_ttf', 'six', 'snappy','zeroconf']

Lots of reasons here, not clear to me I can correctly diagnose, but I see things like: enum34 bypasses build on python3 (this might be OK in this case?)) libshine builds .a but it is not copied for apk libxml2 builds .al but not .a, then tries to copy .a for apk

1. Dependency is Python2, etc:

['apsw','atom','babel','boost','cdecimal','cherrypy','coverage','cymunk',
'dateutil','decorator','doubleratchet','enaml','ethash','evdev','feedparser',
'gevent','gevent-websocket','greenlet','groestlcoin_hash','icu','ifaddrs',
'kiwisolver','libglob','libnacl','libpq','libsodium','libtorrent',
'libtribler','libvorbis','libzbar','libzmq','m2crypto','msgpack-python',
'omemo','omemo-backend-signal','pil','preppy','psycopg2','pyaml','pyasnl',
'pyethereum','pyicu','pyleveldb','pynacl','pyogg',
'pyopenal','pyopenssl','pyproj','pyrxp','pysha3','pyusb','pyyaml','pyzmq',
'regex','ruamel.yaml','scrypt','secp256k1','shapely','simple-crypt',
'sqlalchemy','storm','sympy','ujson','vispy','websocket-client','wsaccl',
'x3dh','xeddsa','zbar','zope','zope_interface']

1. Other dependencies:

['flask','kivent_core','kivent_cymunk','kivent_particles','kivent_polygen',
'libmysqlclient','opencv','twisted','vlc']

Build Log Files:

log_no_object.zip log_no_apk.zip log_has_object.zip

[RobertFlatt]

Getting better. Results of my attempt to build the recipes follow. If anybody knows how to build any of the failed recipes please let me know.

Recipes looking good: ['Pillow', 'android', 'apsw', 'atom', 'audiostream', 'babel', 'boost', 'cffi', 'coverage', 'cryptography', 'cymunk', 'dateutil', 'decorator', 'doubleratchet', 'enum34', 'ffmpeg', 'ffpyplayer_codecs', 'freetype', 'gevent', 'gevent-websocket', 'greenlet', 'harfbuzz', 'idna', 'ifaddrs', 'ipaddress', 'jedi', 'jpeg', 'leveldb', 'libcurl', 'libexpat', 'libffi', 'libiconv', 'libnacl', 'libogg', 'libpq', 'libsecp256k1', 'libshine', 'libsodium', 'libtorrent', 'libx264', 'libxml2', 'libxslt', 'lxml', 'msgpack-python', 'netifaces', 'numpy', 'openal', 'opencv', 'openssl', 'pbkdf2', 'png', 'preppy', 'protobuf_cpp', 'pyaml', 'pyasn1', 'pycparser', 'pycrypto', 'pyjnius', 'pymunk', 'pynacl', 'pyogg', 'pyopenal', 'pyopenssl', 'pysdl2', 'pysha3', 'pytz', 'pyusb', 'pyyaml', 'reportlab', 'requests', 'scrypt', 'sdl2', 'sdl2_image', 'sdl2_mixer', 'sdl2_ttf', 'secp256k1', 'simple-crypt', 'six', 'snappy', 'spine', 'sqlite3', 'vispy', 'websocket-client', 'wsaccel', 'zeroconf']

Recipes with no APK created: ['cdecimal', 'cherrypy', 'enaml', 'ethash', 'evdev', 'feedparser', 'ffpyplayer', 'flask', 'fontconfig', 'groestlcoin_hash', 'icu', 'kivent_core', 'kivent_cymunk', 'kivent_particles', 'kivent_polygen', 'kivykiwisolver', 'libgeos', 'libglob', 'libmysqlclient', 'libtribler', 'libvorbis', 'libzbar', 'libzmq', 'm2crypto', 'mysqldb', 'omemo', 'omemo-backend-signal', 'pil', 'psycopg2', 'pycryptodome', 'pyethereum', 'pyicu', 'pyleveldb', 'pyproj', 'pyrxp', 'pyzmq', 'regex', 'ruamel.yaml', 'shapely', 'sqlalchemy', 'storm', 'sympy', 'twisted', 'ujson', 'vlc', 'x3dh', 'xeddsa', 'zbar', 'zope', 'zope_interface']

Test .p4a

--requirements=<recipe>,python3,kivy
--arch=armeabi-v7a
--ndk_version 17c
--android_api 28
--ndk-api 21

Test depends sudo apt-get install -y autoconf libtool cmake libffi-dev pkg-config libjpeg9-dev python3-pil

Test log from failed builds:

cdecimal :
   mpdecimal.h:63:4: error: #error "cannot use CONFIG_64 with 32-bit header."
   mpdecimal.h:152:4: error: #error "unsupported platform: need mpd_size_t == mpd_uint_t"
   typearith.h:601:4: error: #error "adapt mul_size_t() and mulmod_size_t()"
cherrypy :
   Build failed: Didn't find any valid dependency graphs
enaml :
   Build failed: Didn't find any valid dependency graphs
ethash :
    fatal error: crypt.h: No such file or directory
evdev :
   The linux/input.h header file is missing
feedparser :
   Build failed: Didn't find any valid dependency graphs
ffpyplayer :
   Found in BROKEN_RECIPES_PYTHON3
   distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('cython')
flask :
   Build failed: Didn't find any valid dependency graphs
fontconfig :
   688: Android NDK: Module main depends on undefined modules: python_shared
groestlcoin_hash :
    fatal error: crypt.h: No such file or directory
    fatal error: crypt.h: No such file or directory
icu :
   Found in BROKEN_RECIPES_PYTHON3
   RuntimeError: icu needs an android api <= 19
kivent_core :
   Found in BROKEN_RECIPES_PYTHON3
   ModuleNotFoundError: No module named 'kivy'
kivent_cymunk :
   Found in BROKEN_RECIPES_PYTHON3
   ModuleNotFoundError: No module named 'kivy'
kivent_particles :
   Found in BROKEN_RECIPES_PYTHON3
   ModuleNotFoundError: No module named 'kivy'
kivent_polygen :
   Found in BROKEN_RECIPES_PYTHON3
   ModuleNotFoundError: No module named 'kivy'
kivykiwisolver :
     Could not fetch URL https://pypi.org/simple/kivykiwisolver/:
libgeos :
    fatal error: vector: No such file or directory
libglob :
   Found in BROKEN_RECIPES_PYTHON3
   ValueError: '--sysroot' is not in list
libmysqlclient :
   /bin/cp: cannot stat '/usr/local/lib/python3.6/dist-packages/pythonforandroid/recipes/libmysqlclient/p4a.cmake': No such file or directory
libtribler :
   Build failed: Didn't find any valid dependency graphs
libvorbis :
   configure: error: Ogg >= 1.0 required !
libzbar :
   configure.ac:109: error: possibly undefined macro: AM_ICONV
libzmq :
   arm-linux-androideabi-g++: error: crtbegin_so.o: No such file or directory
   arm-linux-androideabi-g++: error: crtend_so.o: No such file or directory
m2crypto :
   SWIG/_m2crypto_wrap.c:9493:19: error: dereferencing pointer to incomplete type
   SWIG/_m2crypto_wrap.c:9720:48: error: invalid application of 'sizeof' to incomplete type 'struct stack_st'
mysqldb :
   /bin/cp: cannot stat '/usr/local/lib/python3.6/dist-packages/pythonforandroid/recipes/libmysqlclient/p4a.cmake': No such file or directory
omemo :
   /bin/cp: cannot stat '_crypto_sign.so': No such file or directory
omemo-backend-signal :
   /bin/cp: cannot stat '_crypto_sign.so': No such file or directory
pil :
   SyntaxError: Missing parentheses in call to 'print'
psycopg2 :
   Found in BROKEN_RECIPES_PYTHON3
   Error: pg_config executable not found.
pycryptodome :
   src/common.h:62:9: error: unknown type name '__int8'
   src/common.h:62:16: error: conflicting types for 'int8_t'
   src/common.h:63:25: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'uint8_t'
   src/common.h:64:9: error: unknown type name '__int16'
   src/common.h:64:17: error: conflicting types for 'int16_t'
   src/common.h:65:26: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'uint16_t'
   src/common.h:66:9: error: unknown type name '__int32'
   src/common.h:67:26: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'uint32_t'
   src/common.h:68:9: error: unknown type name '__int64'
   src/common.h:68:17: error: conflicting types for 'int64_t'
   src/common.h:69:26: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'uint64_t'
   src/common.h:176:2: error: #error No routines for aligned memory
pyethereum :
    fatal error: crypt.h: No such file or directory
pyicu :
   RuntimeError: icu needs an android api <= 19
pyleveldb :
    fatal error: crypt.h: No such file or directory
pyproj :
   error: cannot open crtbegin_dynamic.o: No such file or directory
   error: cannot open crtend_android.o: No such file or directory
   error: cannot find -lc
   error: cannot find -ldl
   nad2bin.c:80: error: undefined reference to 'strcmp'
   collect2: error: ld returned 1 exit status
pyrxp :
    fatal error: crypt.h: No such file or directory
pyzmq :
   arm-linux-androideabi-g++: error: crtbegin_so.o: No such file or directory
   arm-linux-androideabi-g++: error: crtend_so.o: No such file or directory
regex :
    fatal error: crypt.h: No such file or directory
ruamel.yaml :
   Build failed: Didn't find any valid dependency graphs
shapely :
    fatal error: vector: No such file or directory
sqlalchemy :
   IndexError: list index out of range
storm :
   Found in BROKEN_RECIPES_PYTHON3
   SyntaxError: invalid syntax
sympy :
   Found in BROKEN_RECIPES_PYTHON3
   Please install the mpmath package with a version >= 0.19
twisted :
   Build failed: Didn't find any valid dependency graphs
ujson :
    fatal error: crypt.h: No such file or directory
vlc :
   Found in BROKEN_RECIPES_PYTHON3
x3dh :
   /bin/cp: cannot stat '_crypto_sign.so': No such file or directory
xeddsa :
   /bin/cp: cannot stat '_crypto_sign.so': No such file or directory
zbar :
   configure.ac:109: error: possibly undefined macro: AM_ICONV
zope :
   Build failed: Didn't find any valid dependency graphs
zope_interface :
   Build failed: Didn't find any valid dependency graphs