Bigquery: KeyError: 'configuration'

We are using google-cloud-bigquery==0.32.0, and have an hourly script that monitors the status of async queries. The code basically looks like this:

from google.cloud import bigquery

bq_client = bigquery.Client(project=GCLOUD_PROJECT_ID_BIGQUERY)
all_jobs = bq_client.list_jobs(all_users=True)
for job in all_jobs:
	analyse_job(job)

Excerpt of installed google cloud packages:

google-api-core==1.1.1
google-api-python-client==1.6.6
google-auth==1.4.1
google-auth-httplib2==0.0.3
google-cloud-bigquery==0.32.0
google-cloud-core==0.28.1
google-cloud-monitoring==0.28.1
google-cloud-storage==1.8.0
google-resumable-media==0.3.1
googleapis-common-protos==1.5.3

Starting from 20/04/2018 08:00 AM UTC the script started to fail with the following stacktrace:

Traceback (most recent call last):
14:26:19			  File "/home/script_repos/bigquery/monitor_jobs.py", line 214, in <module>
14:26:19			    main()
14:26:19			  File "/home/script_repos/bigquery/monitor_jobs.py", line 175, in main
14:26:19			    for idx, job in enumerate(all_jobs):
14:26:19			  File "/home/script_repos/bigquery/lib/python2.7/site-packages/google/api_core/page_iterator.py", line 200, in _items_iter
14:26:19			    for item in page:
14:26:19			  File "/home/script_repos/bigquery/lib/python2.7/site-packages/google/api_core/page_iterator.py", line 125, in next
14:26:19			    result = self._item_to_value(self._parent, item)
14:26:19			  File "/home/script_repos/bigquery/lib/python2.7/site-packages/google/cloud/bigquery/client.py", line 1396, in _item_to_job
14:26:19			    return iterator.client.job_from_resource(resource)
14:26:19			  File "/home/script_repos/bigquery/lib/python2.7/site-packages/google/cloud/bigquery/client.py", line 554, in job_from_resource
14:26:19			    config = resource['configuration']
14:26:19			KeyError: 'configuration'

Strange thing is that with the exact same virtual environment on my local computer, I don’t get this error. My local dev computer is a macOS High Sierra 10.13.3, while the server is running CentOS Linux 7. Both use Python 2.7.12. Do you guys have any idea what can possibly cause the error and why I am not seeing it locally?

[Maerig]

Yes, it happens when calling list_jobs with all_users=True while not having the bigquery.jobs.listAll permission.

[tseaver]

@shollyman ISTM that the back-end could reasonably just omit listing jobs for which the requester doesn’t have access.