job.environment causes jobs to not start around ~50% of time but especially on first creation of environment

Description

We heavily use the job.environment part of actions and deploy many environments per day.

A sample of our deploy job might look like

deploy-feature:
    name: Deploy Feature Environment
    needs: build-image
    runs-on: ubuntu-latest
    environment:
      name: ${{ needs.build-image.outputs.branch }}
      url: https://${{ env.GITHUB_REF_SLUG_URL }}.mydomain.com
    steps:
      - etc etc

• Approximately 90% of the time when environment has not been created before (dynamic PR environments like the example above) this job will fail to even get created on the first execution. The result is simply “this check failed” after 0 seconds.
• If we then re-run all jobs (which is another story… when can we rerun individual jobs??? 😃 ) then it will always succeed.
• For other environments that are more permanent/static like “staging” or “production” it is far less of an issue though does fail in the same way occasionally.
• This applies to all of our jobs that use job.environment in all of our projects (20+)
• Have verified that removing job.environment removes the problem - but also removes all the related functionality of course 😃

Feels like some pre-job API/Webhook to create / check environment on the runner side fails to execute in time and thus crashed the whole process of initialising the job.

Virtual environments affected

• Ubuntu 16.04
• Ubuntu 18.04
• Ubuntu 20.04
• macOS 10.15
• macOS 11
• Windows Server 2016
• Windows Server 2019

Image version and build link

github.com runners with ubuntu-latest

Is it regression?

No response

Expected behavior

Jobs should always be successfully get initialised if they use job.environment

Actual behavior

No response

Repro steps

Using the example I already posted to generate new/unique environment for the first time.

Might succeed occasionally

[aiqiaoy]

Taking a look

[ruvceskistefan]

Hey all,

This issue does not seem to be a problem with the runner application, it concerns the GitHub actions platform more generally. If this is still happening, could you please post your feedback on the GitHub Community Support Forum which is actively monitored. Using the forum ensures that we route your problem to the correct team. 😃