Strange Firestore Installation Issue in Compute Engine

I encountered the following strange behavior on a Google Compute Engine node (in a Python 3 virtualenv sandbox).

I installed the firebase-admin module which has a dependency on google-cloud-firestore:

$ pip install firebase-admin
Collecting firebase-admin
  Downloading firebase_admin-2.9.0-py2.py3-none-any.whl (54kB)
    100% |████████████████████████████████| 61kB 2.4MB/s 
Collecting google-cloud-storage>=1.2.0 (from firebase-admin)
  Downloading google_cloud_storage-1.8.0-py2.py3-none-any.whl (53kB)
    100% |████████████████████████████████| 61kB 4.3MB/s 
Requirement already satisfied: six>=1.6.1 in /home/hkj/py3/lib/python3.5/site-packages (from firebase-admin)
Collecting requests>=2.13.0 (from firebase-admin)
  Downloading requests-2.18.4-py2.py3-none-any.whl (88kB)
    100% |████████████████████████████████| 92kB 3.7MB/s 
Collecting google-auth>=1.3.0 (from firebase-admin)
  Downloading google_auth-1.4.1-py2.py3-none-any.whl (65kB)
    100% |████████████████████████████████| 71kB 4.6MB/s 
Collecting google-cloud-firestore>=0.27.0 (from firebase-admin)
  Downloading google_cloud_firestore-0.29.0-py2.py3-none-any.whl (105kB)
    100% |████████████████████████████████| 112kB 4.1MB/s 
Collecting google-resumable-media>=0.3.1 (from google-cloud-storage>=1.2.0->firebase-admin)
  Downloading google_resumable_media-0.3.1-py2.py3-none-any.whl
Collecting google-cloud-core<0.29dev,>=0.28.0 (from google-cloud-storage>=1.2.0->firebase-admin)
  Downloading google_cloud_core-0.28.1-py2.py3-none-any.whl
Collecting google-api-core<2.0.0dev,>=0.1.1 (from google-cloud-storage>=1.2.0->firebase-admin)
  Downloading google_api_core-1.0.0-py2.py3-none-any.whl (51kB)
    100% |████████████████████████████████| 61kB 5.4MB/s 
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /home/hkj/py3/lib/python3.5/site-packages (from requests>=2.13.0->firebase-admin)
Collecting idna<2.7,>=2.5 (from requests>=2.13.0->firebase-admin)
  Downloading idna-2.6-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 5.1MB/s 
Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.13.0->firebase-admin)
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB)
    100% |████████████████████████████████| 143kB 4.0MB/s 
Collecting certifi>=2017.4.17 (from requests>=2.13.0->firebase-admin)
  Downloading certifi-2018.1.18-py2.py3-none-any.whl (151kB)
    100% |████████████████████████████████| 153kB 4.4MB/s 
Collecting rsa>=3.1.4 (from google-auth>=1.3.0->firebase-admin)
  Downloading rsa-3.4.2-py2.py3-none-any.whl (46kB)
    100% |████████████████████████████████| 51kB 8.1MB/s 
Collecting cachetools>=2.0.0 (from google-auth>=1.3.0->firebase-admin)
  Downloading cachetools-2.0.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1 (from google-auth>=1.3.0->firebase-admin)
  Downloading pyasn1_modules-0.2.1-py2.py3-none-any.whl (60kB)
    100% |████████████████████████████████| 61kB 6.7MB/s 
Collecting googleapis-common-protos<2.0dev,>=1.5.3 (from google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage>=1.2.0->firebase-admin)
  Downloading googleapis-common-protos-1.5.3.tar.gz
Collecting protobuf>=3.0.0 (from google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage>=1.2.0->firebase-admin)
  Downloading protobuf-3.5.2-cp35-cp35m-manylinux1_x86_64.whl (6.4MB)
    100% |████████████████████████████████| 6.4MB 218kB/s 
Collecting pytz (from google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage>=1.2.0->firebase-admin)
  Downloading pytz-2018.3-py2.py3-none-any.whl (509kB)
    100% |████████████████████████████████| 512kB 2.6MB/s 
Requirement already satisfied: setuptools>=34.0.0 in /home/hkj/py3/lib/python3.5/site-packages (from google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage>=1.2.0->firebase-admin)
Collecting pyasn1>=0.1.3 (from rsa>=3.1.4->google-auth>=1.3.0->firebase-admin)
  Downloading pyasn1-0.4.2-py2.py3-none-any.whl (71kB)
    100% |████████████████████████████████| 71kB 10.0MB/s 
Building wheels for collected packages: googleapis-common-protos
  Running setup.py bdist_wheel for googleapis-common-protos ... done
  Stored in directory: /home/hkj/.cache/pip/wheels/93/e0/cb/b06866f012310b96fba79c37f199aaf973a2e97a42ca7ef696
Successfully built googleapis-common-protos
Installing collected packages: google-resumable-media, protobuf, googleapis-common-protos, pytz, idna, chardet, certifi, requests, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, google-api-core, google-cloud-core, google-cloud-storage, google-cloud-firestore, firebase-admin
Successfully installed cachetools-2.0.1 certifi-2018.1.18 chardet-3.0.4 firebase-admin-2.9.0 google-api-core-1.0.0 google-auth-1.4.1 google-cloud-core-0.28.1 google-cloud-firestore-0.29.0 google-cloud-storage-1.8.0 google-resumable-media-0.3.1 googleapis-common-protos-1.5.3 idna-2.6 protobuf-3.5.2 pyasn1-0.4.2 pyasn1-modules-0.2.1 pytz-2018.3 requests-2.18.4 rsa-3.4.2

As you can see at the end it has installed google-cloud-firestore-0.29.0 package. But then I got the following error, trying to use it:

>>> import firebase_admin
>>> firebase_admin.initialize_app()
<firebase_admin.App object at 0x7f2f77fb95f8>
>>> from firebase_admin import firestore
Traceback (most recent call last):
  File "/home/hkj/py3/lib/python3.5/site-packages/firebase_admin/firestore.py", line 22, in <module>
    from google.cloud import firestore # pylint: disable=import-error,no-name-in-module
  File "/home/hkj/py3/lib/python3.5/site-packages/google/cloud/firestore.py", line 18, in <module>
    from google.cloud.firestore_v1beta1 import __version__
  File "/home/hkj/py3/lib/python3.5/site-packages/google/cloud/firestore_v1beta1/__init__.py", line 21, in <module>
    from google.cloud.firestore_v1beta1._helpers import GeoPoint
  File "/home/hkj/py3/lib/python3.5/site-packages/google/cloud/firestore_v1beta1/_helpers.py", line 24, in <module>
    import grpc
ImportError: No module named 'grpc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/hkj/py3/lib/python3.5/site-packages/firebase_admin/firestore.py", line 28, in <module>
    raise ImportError('Failed to import the Cloud Firestore library for Python. Make sure '
ImportError: Failed to import the Cloud Firestore library for Python. Make sure to install the "google-cloud-firestore" module.

Installing google-cloud-firestore separately resolved the issue:

$ pip install google-cloud-firestore
Requirement already satisfied: google-cloud-firestore in /home/hkj/py3/lib/python3.5/site-packages
Requirement already satisfied: google-cloud-core<0.29dev,>=0.28.0 in /home/hkj/py3/lib/python3.5/site-packages (from google-cloud-firestore)
Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=0.1.1 in /home/hkj/py3/lib/python3.5/site-packages (from google-cloud-firestore)
Requirement already satisfied: six>=1.10.0 in /home/hkj/py3/lib/python3.5/site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /home/hkj/py3/lib/python3.5/site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.3 in /home/hkj/py3/lib/python3.5/site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: setuptools>=34.0.0 in /home/hkj/py3/lib/python3.5/site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: pytz in /home/hkj/py3/lib/python3.5/site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/hkj/py3/lib/python3.5/site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: protobuf>=3.0.0 in /home/hkj/py3/lib/python3.5/site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Collecting grpcio>=1.8.2; extra == "grpc" (from google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
  Downloading grpcio-1.10.0-cp35-cp35m-manylinux1_x86_64.whl (7.5MB)
    100% |████████████████████████████████| 7.5MB 186kB/s 
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/hkj/py3/lib/python3.5/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /home/hkj/py3/lib/python3.5/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: idna<2.7,>=2.5 in /home/hkj/py3/lib/python3.5/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: certifi>=2017.4.17 in /home/hkj/py3/lib/python3.5/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/hkj/py3/lib/python3.5/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: rsa>=3.1.4 in /home/hkj/py3/lib/python3.5/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: cachetools>=2.0.0 in /home/hkj/py3/lib/python3.5/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.1 in /home/hkj/py3/lib/python3.5/site-packages (from pyasn1-modules>=0.2.1->google-auth<2.0.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=0.1.1->google-cloud-firestore)
Installing collected packages: grpcio
Successfully installed grpcio-1.10.0

Any idea why grpcio did not install in the first attempt?

[tseaver]

@constantinos07 Until the pip bug is fixed, or firebase-admin-python adds the explicit pin needed to work around it, you can work around this by creating an explicit requirements.txt which pins the needed versions, E.g.:

grpcio==1.13.0rc3
google-cloud-firestore==0.29.0
google-cloud-storage==1.10.0

[hiranya911]

We define it as >= 0.27.0. So it ends up installing 0.29.0. This issue also seems to occur randomly. For instance I don’t see it happening in our CI builds: https://travis-ci.org/firebase/firebase-admin-python/jobs/410999811