bigquery iam permissions not updatable

I have an existing environment which I’ve created and managed via Deployment Manager, including a BigQuery dataset and several tables.

A requirement came up where I needed a new service account which needed permissions on the BigQuery dataset, so I added the service account and adjusted the dataset’s accessControl policy in my jinja template.

resources:
#...
- name: my_dataset
  type: bigquery.v2.dataset
  accessControl:
    gcpIamPolicy:
      bindings:
      - role: 'roles/bigquery.dataOwner'
        members:
        - "serviceAccount:polyglot-sync@{{ properties["project"] }}.iam.gserviceaccount.com"
      - role: 'roles/bigquery.jobUser'
        members:
        - "serviceAccount:polyglot-sync@{{ properties["project"] }}.iam.gserviceaccount.com"
#...

The service account created successfully, but I received an error when DM tried to deploy the new accessControl policy:

- code: NO_METHOD_TO_UPDATE_ACCESS_CONTROL
  location: /deployments/sre-staging-17/resources/my_dataset
  message: No method found to update access control on resource 'my_dataset' of type
    'bigquery.v2.dataset'.

Surely accessControl policies will always be subject to change? Having them non-updatable seems unbelievable. Have I just misunderstood something, or is there another way to achieve this please?

[ocsig]

I’ve tested your scenario with the gcp-types/bigquery-v2:datasets type and update or permissions works fine.

I would also highlight the Cloud Foundation Toolkit templates for Bigquery Dataset and Table: https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit/tree/master/dm/templates/bigquery

Usinf the CFT Dataset template:

imports:
  - path: ../templates/bigquery/bigquery_dataset.py
    name: bigquery_dataset.py

resources:
  - name: test-bq-dataset
    type: bigquery_dataset.py
    properties:
      name: test_bq_dataset
      location: US
      access:
        - role: OWNER
          userByEmail: [e-mail]

Please let me know if this solved your issue!

[tommed]

The lack of email verification makes sense, thanks for the explanation!

The docs do say:

The following legacy mappings will be applied: OWNER <=> roles/bigquery.dataOwner WRITER <=> roles/bigquery.dataEditor READER <=> roles/bigquery.dataViewer This field will accept any of the above formats, but will return only the legacy format. For example, if you set this field to “roles/bigquery.dataOwner”, it will be returned back as “OWNER”.

I interpreted this to mean you can pass in valid roles such as roles/bigquery.jobUser, but just don’t expect it to return this in the same output. I am puzzled why the IAM here would only support what is referred to as “legacy” permissions and not actual role names; but assume this will change as DM evolves?

I made a change to reduce my access entry to just:

access:
  - role: WRITER
    userByEmail: iamMember: 'serviceAccount:my-bqsync-user@{{ env["project"] }}.iam.gserviceaccount.com'

However bq show still shows only projectReaders, projectWriters and projectOwners, suggesting it still hasn’t assigned the permissions correctly.

I do appreciate you saying the python examples work, but would really like to get the vanilla yaml working here before adding further complexity into our approach.