Why the --no-deps in build.py?

Checklist

• the issue is indeed a bug and not a support request
• issue doesn’t already exist: https://github.com/kivy/python-for-android/issues
• I have a short, runnable example that reproduces the issue
• I reproduced the problem with the latest development version (p4a.branch = develop)
• I used the grave accent (aka backticks) to format code or logs when appropriated

Versions

• Python: 3.8.12
• OS: Ubuntu 20.04.3 LTS and Android 10
• Buildozer: 1.2.0

Description

App works on my Ubuntu VM but not on my Android device. 3rd party python package (telenium in this case), encounters an error on import in main.py. Package is added to requirements list in buildozer.spec. In fact, judging from the build output it is being addressed by buildozer to some degree. Judging from the python stacktrace however, the 3rd party package isn’t able to import one of its own dependencies (werkzeug in this case). This dependency is listed in the install_requireslist argument in the 3rd party package’s setup.py. Therefore, shouldn’t this dependency also automatically be installed by buildozer during the build process? Am I doing anything wrong here?

After talking to someone on the Kivy support Google group, I was nudged to conclude that I needed to abide by the workaround of listing all the requirements of telenium and so on (recursively)… I did it and it works BUT why do I have to go through all this trouble? So, I looked a little deeper and I found this line. Why is the --no-deps argument being used here? I manually removed this argument from my local python-for-android installation (under the .buildozer directory) and it seems to solve my problem without having to do this recursive and semi-manual gathering of indirect app requirements.

I am not very experienced in desktop or mobile application/package building, so please pardon my ignorance. Is there a concrete reason why this argument is being used? Would removing it break any use cases or other parts of this package? Judging from some of the amount of answers I found which proposed the tedious “solution” (the listing countless requirements) to me, I’m pretty confident that there would be less user frustration if the --no-deps argument given to pip can be removed safely.

Basically, I’m suggesting that it should be a python-for-android design requirement to make the --requirements argument usage, practically, 1:1 with normal pip install usage. If there are indirect dependencies properly specified in our direct one’s (like telenium in this case) setup.py, I don’t think a python-for-android user should have to specify them again. Many python packages don’t have a requirements.txt and so python-for-android shouldn’t depend on it.

Pardon me if I sound arrogant. This is a small problem with a functional workaround, as I’ve mentioned. However, as The Pragmatic Programmer book says early on “Don’t live with broken windows” and I think this use case fits the metaphor.

main.py

import platform

import kivy
import telenium

from kivy.app import App
from kivy.uix.anchorlayout import AnchorLayout
from kivy.uix.boxlayout import BoxLayout
from kivy.uix.label import Label


class MyApp(App):
    def build(self):
        root_layout = AnchorLayout(anchor_x="center", anchor_y="center")
        widget_layout = BoxLayout(orientation="vertical", size_hint=(None, None))
        widget_layout.add_widget(Label(text='My App'))
        widget_layout.add_widget(Label(text=f'Python {platform.python_version()}'))
        widget_layout.add_widget(Label(text=f'Kivy {kivy.__version__}'))
        root_layout.add_widget(widget_layout)
        return root_layout


if __name__ == "__main__":
    telenium.install()
    MyApp().run()

buildozer.spec

Command:

$ buildozer android clean debug deploy
$ ~/.buildozer/android/platform/android-sdk/platform-tools/adb logcat -c
$ buildozer android run logcat | grep python

Spec file:

[app]

# (str) Title of your application
title = My App

# (str) Package name
package.name = myapp

# (str) Package domain (needed for android/ios packaging)
package.domain = org.pdallair

# (str) Source code where the main.py live
source.dir = .

# (list) Source files to include (let empty to include all the files)
source.include_exts = py,png,jpg,kv,atlas

# (list) List of inclusions using pattern matching
#source.include_patterns = assets/*,images/*.png

# (list) Source files to exclude (let empty to not exclude anything)
#source.exclude_exts = spec

# (list) List of directory to exclude (let empty to not exclude anything)
#source.exclude_dirs = tests, bin

# (list) List of exclusions using pattern matching
#source.exclude_patterns = license,images/*/*.jpg

# (str) Application versioning (method 1)
version = 0.1.0

# (str) Application versioning (method 2)
# version.regex = __version__ = ['"](.*)['"]
# version.filename = %(source.dir)s/main.py

# (list) Application requirements
# comma separated e.g. requirements = sqlite3,kivy
requirements = python3,kivy,telenium

# (str) Custom source folders for requirements
# Sets custom source for any requirements with recipes
# requirements.source.kivy = ../../kivy

# (list) Garden requirements
#garden_requirements =

# (str) Presplash of the application
#presplash.filename = %(source.dir)s/data/presplash.png

# (str) Icon of the application
#icon.filename = %(source.dir)s/data/icon.png

# (str) Supported orientation (one of landscape, sensorLandscape, portrait or all)
orientation = portrait

# (list) List of service to declare
#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY

#
# OSX Specific
#

#
# author = © Copyright Info

# change the major version of python used by the app
#osx.python_version = 3

# Kivy version to use
#osx.kivy_version = 2.0.0

#
# Android specific
#

# (bool) Indicate if the application should be fullscreen or not
fullscreen = 0

# (string) Presplash background color (for new android toolchain)
# Supported formats are: #RRGGBB #AARRGGBB or one of the following names:
# red, blue, green, black, white, gray, cyan, magenta, yellow, lightgray,
# darkgray, grey, lightgrey, darkgrey, aqua, fuchsia, lime, maroon, navy,
# olive, purple, silver, teal.
#android.presplash_color = #FFFFFF

# (list) Permissions
#android.permissions = INTERNET

# (int) Target Android API, should be as high as possible.
#android.api = 27

# (int) Minimum API your APK will support.
#android.minapi = 21

# (int) Android SDK version to use
#android.sdk = 20

# (str) Android NDK version to use
#android.ndk = 19b

# (int) Android NDK API to use. This is the minimum API your app will support, it should usually match android.minapi.
#android.ndk_api = 21

# (bool) Use --private data storage (True) or --dir public storage (False)
#android.private_storage = True

# (str) Android NDK directory (if empty, it will be automatically downloaded.)
#android.ndk_path =

# (str) Android SDK directory (if empty, it will be automatically downloaded.)
#android.sdk_path =

# (str) ANT directory (if empty, it will be automatically downloaded.)
#android.ant_path =

# (bool) If True, then skip trying to update the Android sdk
# This can be useful to avoid excess Internet downloads or save time
# when an update is due and you just want to test/build your package
# android.skip_update = False

# (bool) If True, then automatically accept SDK license
# agreements. This is intended for automation only. If set to False,
# the default, you will be shown the license when first running
# buildozer.
# android.accept_sdk_license = False

# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity

# (str) Android app theme, default is ok for Kivy-based app
# android.apptheme = "@android:style/Theme.NoTitleBar"

# (list) Pattern to whitelist for the whole project
#android.whitelist =

# (str) Path to a custom whitelist file
#android.whitelist_src =

# (str) Path to a custom blacklist file
#android.blacklist_src =

# (list) List of Java .jar files to add to the libs so that pyjnius can access
# their classes. Don't add jars that you do not need, since extra jars can slow
# down the build process. Allows wildcards matching, for example:
# OUYA-ODK/libs/*.jar
#android.add_jars = foo.jar,bar.jar,path/to/more/*.jar

# (list) List of Java files to add to the android project (can be java or a
# directory containing the files)
#android.add_src =

# (list) Android AAR archives to add (currently works only with sdl2_gradle
# bootstrap)
#android.add_aars =

# (list) Gradle dependencies to add (currently works only with sdl2_gradle
# bootstrap)
#android.gradle_dependencies =

# (list) add java compile options
# this can for example be necessary when importing certain java libraries using the 'android.gradle_dependencies' option
# see https://developer.android.com/studio/write/java8-support for further information
# android.add_compile_options = "sourceCompatibility = 1.8", "targetCompatibility = 1.8"

# (list) Gradle repositories to add {can be necessary for some android.gradle_dependencies}
# please enclose in double quotes 
# e.g. android.gradle_repositories = "maven { url 'https://kotlin.bintray.com/ktor' }"
#android.add_gradle_repositories =

# (list) packaging options to add 
# see https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.PackagingOptions.html
# can be necessary to solve conflicts in gradle_dependencies
# please enclose in double quotes 
# e.g. android.add_packaging_options = "exclude 'META-INF/common.kotlin_module'", "exclude 'META-INF/*.kotlin_module'"
#android.add_gradle_repositories =

# (list) Java classes to add as activities to the manifest.
#android.add_activities = com.example.ExampleActivity

# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME

# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =

# (str) launchMode to set for the main activity
#android.manifest.launch_mode = standard

# (list) Android additional libraries to copy into libs/armeabi
#android.add_libs_armeabi = libs/android/*.so
#android.add_libs_armeabi_v7a = libs/android-v7/*.so
#android.add_libs_arm64_v8a = libs/android-v8/*.so
#android.add_libs_x86 = libs/android-x86/*.so
#android.add_libs_mips = libs/android-mips/*.so

# (bool) Indicate whether the screen should stay on
# Don't forget to add the WAKE_LOCK permission if you set this to True
#android.wakelock = False

# (list) Android application meta-data to set (key=value format)
#android.meta_data =

# (list) Android library project to add (will be added in the
# project.properties automatically.)
#android.library_references =

# (list) Android shared libraries which will be added to AndroidManifest.xml using <uses-library> tag
#android.uses_library =

# (str) Android logcat filters to use
#android.logcat_filters = *:S python:D

# (bool) Copy library instead of making a libpymodules.so
#android.copy_libs = 1

# (str) The Android arch to build for, choices: armeabi-v7a, arm64-v8a, x86, x86_64
android.arch = armeabi-v7a

# (int) overrides automatic versionCode computation (used in build.gradle)
# this is not the same as app version and should only be edited if you know what you're doing
# android.numeric_version = 1

#
# Python for android (p4a) specific
#

# (str) python-for-android fork to use, defaults to upstream (kivy)
#p4a.fork = kivy

# (str) python-for-android branch to use, defaults to master
#p4a.branch = master

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#p4a.source_dir =

# (str) The directory in which python-for-android should look for your own build recipes (if any)
#p4a.local_recipes =

# (str) Filename to the hook for p4a
#p4a.hook =

# (str) Bootstrap to use for android builds
# p4a.bootstrap = sdl2

# (int) port number to specify an explicit --port= p4a argument (eg for bootstrap flask)
#p4a.port =


#
# iOS specific
#

# (str) Path to a custom kivy-ios folder
#ios.kivy_ios_dir = ../kivy-ios
# Alternately, specify the URL and branch of a git checkout:
#ios.kivy_ios_url = https://github.com/kivy/kivy-ios
#ios.kivy_ios_branch = master

# Another platform dependency: ios-deploy
# Uncomment to use a custom checkout
#ios.ios_deploy_dir = ../ios_deploy
# Or specify URL and branch
#ios.ios_deploy_url = https://github.com/phonegap/ios-deploy
#ios.ios_deploy_branch = 1.7.0

# (str) Name of the certificate to use for signing the debug version
# Get a list of available identities: buildozer ios list_identities
#ios.codesign.debug = "iPhone Developer: <lastname> <firstname> (<hexstring>)"

# (str) Name of the certificate to use for signing the release version
#ios.codesign.release = %(ios.codesign.debug)s


[buildozer]

# (int) Log level (0 = error only, 1 = info, 2 = debug (with command output))
log_level = 2

# (int) Display warning if buildozer is run as root (0 = False, 1 = True)
warn_on_root = 1

# (str) Path to build artifact storage, absolute or relative to spec file
# build_dir = ./.buildozer

# (str) Path to build output (i.e. .apk, .ipa) storage
# bin_dir = ./bin

#    -----------------------------------------------------------------------------
#    List as sections
#
#    You can define all the "list" as [section:key].
#    Each line will be considered as a option to the list.
#    Let's take [app] / source.exclude_patterns.
#    Instead of doing:
#
#[app]
#source.exclude_patterns = license,data/audio/*.wav,data/images/original/*
#
#    This can be translated into:
#
#[app:source.exclude_patterns]
#license
#data/audio/*.wav
#data/images/original/*
#


#    -----------------------------------------------------------------------------
#    Profiles
#
#    You can extend section / key with a profile
#    For example, you want to deploy a demo version of your application without
#    HD content. You could first change the title to add "(demo)" in the name
#    and extend the excluded directories to remove the HD content.
#
#[app@demo]
#title = My Application (demo)
#
#[app:source.exclude_patterns@demo]
#images/hd/*
#
#    Then, invoke the command line with the "demo" profile:
#
#buildozer --profile demo android debug

Logs

Clean, Build and Deploy

https://drive.google.com/file/d/1o4lmjPBsVChc0LNxhS006GZ7aJxEgtfo/view?usp=sharing

Run

https://drive.google.com/file/d/1Kn4xAbKFld1vbE3yL54wT4l_Z29mRp5_/view?usp=sharing

[RobertFlatt]

I think most would agree with you that the process is not push button. Improvement would be excellent.

PRs welcome.

Conceptually simple, the devil is in the details. As shown by this https://github.com/Android-for-Python/Android-for-Python-Users#requirements which is a list of examples and not a set of rules. That list is definitely not exhaustive. Recipes create some special cases, and false positives result in apk bloat. In addition abstraction requires that any fails are reported meaningfully.

So a PR that addresses these would be, I think, great.

[RobertFlatt]

Did I miss something very basic?

yes

p4a.branch = fix/pip_no_deps

And it builds, I don’t expect an exact size match.

-rw-r--r-- 1 bobf bobf 23516365 Jan  3 14:27 pyrebase-0.1-arm64-v8a-debug.apk
-rw-r--r-- 1 bobf bobf 25656823 Dec 18 10:16 ref.apk

Ran the app, my very basic test is to log into firebase, and that worked. 😃

But the set of packages used is a little different. I can sort of explain why.

The good news is I didn’t find a corner case. The bad news is I’m a skeptic (which is about me not about the code). But I’d recommend others try it, some success stories would help.

RESULTS

FOUND

From the Buildozer log:

Usual defaults:

hostpython3, libffi, openssl, sdl2_image, sdl2_mixer, sdl2_ttf, sqlite3, python3, sdl2, setuptools, pycparser, six, cffi, pyjnius, android, kivy,

Generated for pyrebase:

pycryptodome, pyparsing, gcloud, pyrebase, requests, googleapis-common-protos, requests-toolbelt, python-jwt, jws, protobuf, rsa, oauth2client, pyasn1-modules, certifi, httplib2, pyasn1

EXPECTED

pyrebase,requests,urllib3,chardet,idna,gcloud,oauth2client,requests-toolbelt,protobuf_cpp,python-jwt,pycryptodome,httplib2,pyparsing,pyasn1,pyasn1_modules,rsa,jwcrypto,cryptography, deprecated, wrapt

DIFFERENCES

EXPECTED FOUND:

pyrebase,requests,gcloud,oauth2client,requests-toolbelt, python-jwt, pycryptodome,httplib2, pyparsing, pyasn1,pyasn1_modules,rsa

EXPECTED NOT FOUND:

urllib3, chardet, protobuf_cpp, jwcrypto, cryptography, deprecated, wrapt

UNEXPECTED:

googleapis-common-protos, jws, protobuf