Permission denied on `submitting ML job through api client`

Here is my code that I am trying run for submitting a ML training job through API client. I have used this before (roughly a year ago) and worked.

from googleapiclient import discovery
from googleapiclient import http
from google.oauth2 import service_account

def submit_training_job(training_inputs, gcp_project, job_name, gcp_ser_acc):
    """
    submit training job
    @params {
    training_inputs :  a json containing all inputs for submitting a training job
    gcp_project: name of the project in google cloud platform
    job_name: a job name
    gcp_ser_acc: GCP service account
    }
    @returns {
    response: a json object
    }
    """
    credentials = service_account.Credentials.from_service_account_info(gcp_ser_acc)
    project_id = 'projects/{}'.format(gcp_project)
    cloudml = discovery.build(
        'ml',
        'v1',
        requestBuilder=http.HttpRequest,
        credentials=credentials)
    job_spec = {'jobId': job_name, 'trainingInput': training_inputs}
    request = cloudml.projects().jobs().create(body=job_spec, parent=project_id)
    print('request = ', request)
    response = request.execute()
    return response

However, I can’t get it worked and getting the following errors:

Traceback (most recent call last):
  File ".\deploy-cloud.py", line 132, in <module>
    response = submit_training_job(training_inputs, parsed.gcp_proj, job_name, config['GCP_SERVICE_ACC'])
  File ".\deploy-cloud.py", line 52, in submit_training_job
    response = request.execute()
  File "C:\Users\UserName\Anaconda3\lib\site-packages\googleapiclient\_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "C:\Users\UserName\Anaconda3\lib\site-packages\googleapiclient\http.py", line 856, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://ml.googleapis.com/v1/projects/GCP_PROJECT_NAME/jobs?alt=json returned "Permission denied on resource project GCP_PROJECT_NAME.". Details: "[{'@type': 'type.googleapis.com/google.rpc.Help', 'links': [{'description': 'Google developer console API key', 'url': 'https://console.developers.google.com/project/GCP_PROJECT_NAME/apiui/credential'}]}]">

I have created the service account with full rights. So why does it complain about the permission?

Can anyone points out what is happening?

[hafizurmeldcx]

All good . It is working now. The issue was the wrong spelling of my project name.

Thanks @busunkim96 and @SurferJeffAtGoogle for stopping by.

[SurferJeffAtGoogle]

What is the value of the gcp_project parameter passed to your function?