Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error while enabling GSuite integration

See original GitHub issue

Issue: The documentation for setting up the GSuite integration is incomplete and leads towards a 400 error

Description: Following the README, I:

  1. Enabled Google API access (instructions)
  2. Created a new G Suite user account and accept the Terms of Service (this account will be used as the domain-wide delegated access).
  3. Performed G Suite Domain-Wide Delegation of Authority, as explained here
  4. Downloaded the service account’s credentials
  5. Setup env vars for cartography
GSUITE_GOOGLE_APPLICATION_CREDENTIALS - location of the credentials file.
GSUITE_DELEGATED_ADMIN - email address that you created in step 2

After this, Cartography crashes with the following output:

cartography --neo4j-uri bolt://localhost:7687 --neo4j-user neo4j --neo4j-password-env-var NEO4J_PASSWORD_ENV_VAR
INFO:cartography.sync:Starting sync with update tag '1580915718'
INFO:cartography.sync:Starting sync stage 'create-indexes'
INFO:cartography.intel.create_indexes:Creating indexes for cartography node types.
INFO:cartography.sync:Finishing sync stage 'create-indexes'
INFO:cartography.sync:Starting sync stage 'aws'
...
INFO:cartography.sync:Finishing sync stage 'aws'
INFO:cartography.sync:Starting sync stage 'gcp'
...
INFO:cartography.sync:Finishing sync stage 'gcp'
INFO:cartography.sync:Starting sync stage 'gsuite'
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.client:Refreshing access_token
ERROR:cartography.sync:Unhandled exception during sync stage 'gsuite'
Traceback (most recent call last):
  File "/app/cartography/sync.py", line 69, in run
    stage_func(neo4j_session, config)
  File "/app/cartography/intel/gsuite/__init__.py", line 79, in start_gsuite_ingestion
    api.sync_gsuite_users(session, resources.admin, config.update_tag, common_job_parameters)
  File "/app/cartography/intel/gsuite/api.py", line 230, in sync_gsuite_users
    resp_objs = get_all_users(admin)
  File "/app/cartography/intel/gsuite/api.py", line 109, in get_all_users
    resp = request.execute(num_retries=GOOGLE_API_NUM_RETRIES)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 856, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/admin/directory/v1/users?customer=my_customer&maxResults=500&orderBy=email&alt=json returned "Invalid Input">
Traceback (most recent call last):
  File "/usr/local/bin/cartography", line 11, in <module>
    load_entry_point('cartography', 'console_scripts', 'cartography')()
  File "/app/cartography/cli.py", line 240, in main
    return CLI(default_sync, prog='cartography').main(argv)
  File "/app/cartography/cli.py", line 220, in main
    return cartography.sync.run_with_config(self.sync, config)
  File "/app/cartography/sync.py", line 135, in run_with_config
    return sync.run(neo4j_driver, config)
  File "/app/cartography/sync.py", line 69, in run
    stage_func(neo4j_session, config)
  File "/app/cartography/intel/gsuite/__init__.py", line 79, in start_gsuite_ingestion
    api.sync_gsuite_users(session, resources.admin, config.update_tag, common_job_parameters)
  File "/app/cartography/intel/gsuite/api.py", line 230, in sync_gsuite_users
    resp_objs = get_all_users(admin)
  File "/app/cartography/intel/gsuite/api.py", line 109, in get_all_users
    resp = request.execute(num_retries=GOOGLE_API_NUM_RETRIES)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 856, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/admin/directory/v1/users?customer=my_customer&maxResults=500&orderBy=email&alt=json returned "Invalid Input">

I started troubleshooting this and I found the string my_customer is hardcoded in cartography/intel/gsuite/api.py:

request = admin.users().list(customer='my_customer', maxResults=500, orderBy='email')

I decided to replace it with the customerId of my GSuite org, and then I faced a 403 - Not Authorized error:

cartography --neo4j-uri bolt://localhost:7687 --neo4j-user neo4j --neo4j-password-env-var NEO4J_PASSWORD_ENV_VAR
INFO:cartography.sync:Starting sync with update tag '1580916464'
INFO:cartography.sync:Starting sync stage 'create-indexes'
INFO:cartography.intel.create_indexes:Creating indexes for cartography node types.
INFO:cartography.sync:Finishing sync stage 'create-indexes'
INFO:cartography.sync:Starting sync stage 'aws'
...
INFO:cartography.sync:Finishing sync stage 'aws'
INFO:cartography.sync:Starting sync stage 'gcp'
...
INFO:cartography.sync:Finishing sync stage 'gcp'
INFO:cartography.sync:Starting sync stage 'gsuite'
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.client:Refreshing access_token
WARNING:googleapiclient.http:Encountered 403 Forbidden with reason "forbidden"
ERROR:cartography.sync:Unhandled exception during sync stage 'gsuite'
Traceback (most recent call last):
  File "/app/cartography/sync.py", line 69, in run
    stage_func(neo4j_session, config)
  File "/app/cartography/intel/gsuite/__init__.py", line 79, in start_gsuite_ingestion
    api.sync_gsuite_users(session, resources.admin, config.update_tag, common_job_parameters)
  File "/app/cartography/intel/gsuite/api.py", line 230, in sync_gsuite_users
    resp_objs = get_all_users(admin)
  File "/app/cartography/intel/gsuite/api.py", line 109, in get_all_users
    resp = request.execute(num_retries=GOOGLE_API_NUM_RETRIES)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 856, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/admin/directory/v1/users?customer=<REDACTED>&maxResults=500&orderBy=email&alt=json returned "Not Authorized to access this resource/api">
Traceback (most recent call last):
  File "/usr/local/bin/cartography", line 11, in <module>
    load_entry_point('cartography', 'console_scripts', 'cartography')()
  File "/app/cartography/cli.py", line 240, in main
    return CLI(default_sync, prog='cartography').main(argv)
  File "/app/cartography/cli.py", line 220, in main
    return cartography.sync.run_with_config(self.sync, config)
  File "/app/cartography/sync.py", line 135, in run_with_config
    return sync.run(neo4j_driver, config)
  File "/app/cartography/sync.py", line 69, in run
    stage_func(neo4j_session, config)
  File "/app/cartography/intel/gsuite/__init__.py", line 79, in start_gsuite_ingestion
    api.sync_gsuite_users(session, resources.admin, config.update_tag, common_job_parameters)
  File "/app/cartography/intel/gsuite/api.py", line 230, in sync_gsuite_users
    resp_objs = get_all_users(admin)
  File "/app/cartography/intel/gsuite/api.py", line 109, in get_all_users
    resp = request.execute(num_retries=GOOGLE_API_NUM_RETRIES)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 856, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/admin/directory/v1/users?customer=<REDACTED>&maxResults=500&orderBy=email&alt=json returned "Not Authorized to access this resource/api">

I tried to add more scopes to the service account (listed below), but I still get Not Authorized:

  1. https://www.googleapis.com/auth/admin.directory.domain.readonly
  2. https://www.googleapis.com/auth/admin.directory.group.member.readonly
  3. https://www.googleapis.com/auth/admin.directory.user.readonly

Please complete the following information::

  • Cartography release version or commit hash: 801bb58
  • Python version: 3.7.5
  • OS: python:3.7 docker image

Issue Analytics

  • State:open
  • Created 4 years ago
  • Comments:16 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
zamirTo1commented, Sep 15, 2020

Hi @achantavy I do get the same error as @marco-lancini, I’ve also tried to build the GSuite section by myself according to this guide: https://developers.google.com/admin-sdk/directory/v1/guides/delegation same result 😦

0reactions
marco-lancinicommented, Sep 16, 2020

I’ve documented the approach I took, step-by-step, here: https://www.marcolancini.it/2020/blog-gsuite-domain-delegation/ Hope this helps!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Troubleshoot single sign-on (SSO) - Google Support
This document provides steps to resolve common error messages encountered during the integration or use of SAML-based single sign-on (SSO) with Google ...
Read more >
Common Issues in Google Workspace (G Suite) Integration
While integrating Google Workspace (G Suite) with the Hexnode console, an error message Invalid Input appears. Invalid Input Error. Possible Cause.
Read more >
Troubleshooting the Google integrations: Common problems ...
Go to Gmail and click the Gear icon on the top right and click, Settings. · Make sure Enable IMAP is selected ;...
Read more >
G Suite push group failed with error: Not Authorized to access ...
Resolution · Ensure the G Suite admin used for Provisioning has Super admin permission in G Suite · Enable API access in G...
Read more >
Troubleshooting Google Workspace “Error 400
When you attempt to authorize the Google Workspace Directory integration using a Super Administrator account, you can receive an “Error 400: ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Add support for AWS Transit Gateway

Next page

Reorganize documentation so that available modules are more easily visible