Next Steps for Fully Functioning Composite Actions

NOTE:

For those who are not aware, I’m a college student who interned at GitHub for Summer 2020. Thank you all for all the thoughtful comments and feedback on this thread but since I’m not working at GitHub currently, I will not be responding to any new comments below in this thread.

================================================

Hi there 👋

We just released a new feature called composite run steps actions(https://docs.github.com/en/actions/creating-actions/about-actions#composite-run-steps-actions).

This issue thread will serve as a point of reference for people who are curious about the next steps after this feature and what we plan to work on next. There are probably quite a few questions about when we are going to support in the future like “when are you going to support ‘uses’ steps in a composite action”? In this Issue thread, I’ll talk about the technical problems that we are planning to solve to accomplish this. Throughout the next week, I’ll update it with any additional updates.

tl;dr in this issue, I’ll go over what we currently support for composite run steps and how we plan on developing fully functioning composite actions.

Composite run steps background

What is a composite run steps action?

Many of you have been asking us to build a feature that enables them to nest actions within actions (ex: https://github.com/actions/runner/issues/438).

An important first step for supporting nesting within action is to first start supporting the execution of multiple run steps in an action.

For some developers, they may want to execute a variety of scripts and commands in different languages and shells in a single action and map those outputs to the workflow. We’ve also heard from our community that many developers want to reuse parts of their workflows into other workflows. This new action type, composite run steps, can help developers do just that! For example, developers can abstract parts of their workflows as composite run steps actions and reuse them for other workflows.

Thus, we created this feature called composite run steps which allows users to run multiple run steps in an action!

What does composite run steps currently support?

For each run step in a composite action, we support:

• name
• id
• run
• env
• shell
• working-directory

In addition, we support mapping input and outputs throughout the action.

See docs for more info.

What does Composite Run Steps Not Support

We don’t support setting conditionals, continue-on-error, timeout-minutes, “uses”, and secrets on individual steps within a composite action right now.

(Note: we do support these attributes being set in workflows for a step that uses a composite run steps action)

Examples of how you can use composite run steps actions

• Outputs + Inputs:
  • Workflow: https://github.com/ethanchewy/testing-actions/actions/runs/184531939/workflow
  • Action: https://github.com/ethanchewy/test-composite/blob/outputs4/action.yml
  • Results: https://github.com/ethanchewy/testing-actions/runs/915607926?check_suite_focus=true
• Step-level shell and working_Dir:
  • Workflow: https://github.com/ethanchewy/testing-actions/actions/runs/184577987/workflow
  • Action: https://github.com/ethanchewy/test-composite/blob/shellDir2/action.yml
  • Results: https://github.com/ethanchewy/testing-actions/actions/runs/184577987?check_suite_focus=true
• Timeout-error:
  • Workflow: https://github.com/ethanchewy/testing-actions/actions/runs/184580073/workflow
  • Action: https://github.com/ethanchewy/test-composite/blob/timeout2/action.yml
  • Results: https://github.com/ethanchewy/testing-actions/actions/runs/184580073?check_suite_focus=true
• Step-level env:
  • Workflow: https://github.com/ethanchewy/testing-actions/actions/runs/184585257/workflow
  • Action: https://github.com/ethanchewy/test-composite/blob/env7/action.yml
  • Results: https://github.com/ethanchewy/testing-actions/actions/runs/184585257?check_suite_focus=true
• JSON Error Messaging:
  • Workflow: https://github.com/ethanchewy/testing-actions/actions/runs/184588237/workflow
  • Action: https://github.com/ethanchewy/testing-composite-errors/blob/json/action.yml
  • Results: https://github.com/ethanchewy/testing-actions/actions/runs/184588237?check_suite_focus=true
• Scripts:
  • Workflow: https://github.com/ethanchewy/testing-actions/actions/runs/184591667/workflow
  • Action: https://github.com/ethanchewy/test-script/blob/v18/action.yml
  • Results: https://github.com/ethanchewy/testing-actions/actions/runs/184591667

Next Steps for Developing Fully Functioning Composite Run Steps

In this section, I’ll describe some current features that we are currently working on or are planning to work on. You can see more details about each of these in https://github.com/actions/runner/blob/users/ethanchewy/compositeADR/docs/adrs/0549-composite-run-steps.md.

If conditionals

see: https://github.com/actions/runner/blob/main/docs/adrs/0549-composite-run-steps.md#if-condition

For if conditionals, we treat the composite action as a completely new slate. Each proceeding step depends on the proceeding step’s states (failure, cancelled, or success).

To keep track of these states, we can create a new ExecutionContext for composite actions that keep track of the attributes we need such as the current state of a composite action step before evaluating. When we support nesting, we need to follow some sort of inheritance model (maybe something like state = (parent state && child state)?

Uses

Related questions that we need to answer first before we can support uses:

• How are we going to handle post and pre steps? Should we aggregate them into one individual step for each type?
• When should we download the repos for nested actions

Current experimentation for this: https://github.com/actions/runner/pull/612

timeout-minutes

It makes a ton of sense creating a separate execution context for composite actions especially in the cases for keeping track of the conditional status of the step. This would make it easier to some sort of “time element” to easily get the “correct” condition of the step in log time (something like this algorithm: https://maksimdan.gitbook.io/interview-practice-problems/leetcode_sessions/design/permission-management-for-a-general-tree)

For the nested timeout-minutes support, couldn’t we just link all children tokens to their parents? The time left will trickle down to the children. For example,

|A (30 min.)|
  [B (15 min.)|
    |B2|
  |C (None)|
    |C2|

Let’s say the A composite action whole step has a timeout-minutes fo 30 minutes and the step B (which is a step inside the composite action) has a timeout-minutes of 15 minutes and the step B exceeds that time, we can easily fail that thread and differentiate that between a cancellation. Then, we would move onto the next step C which doesn’t have a timeout-minutes set and since it has ~15 minutes left, it will still start running. If the next step C takes more than 15 minutes, then we fail the whole action since the whole action has taken more than 30 minutes (the step C will automatically fail as well as the whole composite action step since they are linked)

Areas that we are also thinking about

There are many more layers of complexity for fully functioning composite actions.

For example at a high level,

• How would we use multiple Docker actions in a composite action effectively?
• How do we propagate and resolve errors effectively with nested actions?
• Should we support the shell attribute for a uses step? (probably not)

Runner specific questions:

• Should we create a separate, slimmed-down version of ExecutionContext for Composite Actions?
  • Yes, because this would reduce confusion and create an easier way to add onto it + easily hook this up with the .Global ExecutionContext.
• How do we map our Scopes in nested actions?
  • We’ve already come up with a pretty clever solution for generating a context name (i.e. {context.ScopeName}.{context.ContextName})
  • Our dictionary for resolving outputs is in the nested dictionary StepsContext[Step_Name][Scope_Name][“outputs”]. Could we flatten the dictionary to reduce lookup time?

TODOs

• Handle Continue-on-error (we need to change how we handle failures in the composite action handler [i.e. don’t call Complete(TaskResult.Failed))
• Handle private actions
• Outcomes?
• Revisit defaults? Does it make sense to add defaults once we have fully functional composite actions.

Extra Information

How we built it

• ADR: https://github.com/actions/runner/blob/users/ethanchewy/compositeADR/docs/adrs/0549-composite-run-steps.md
• Composite Actions related code: https://github.com/actions/runner/commits?author=ethanchewy
• PRs: ADR (https://github.com/actions/runner/pull/554), https://github.com/actions/runner/pull/549, https://github.com/actions/runner/pull/557, https://github.com/actions/runner/pull/564, https://github.com/actions/runner/pull/568, https://github.com/actions/runner/pull/578, https://github.com/actions/runner/pull/591, https://github.com/actions/runner/pull/610, https://github.com/actions/runner/pull/609, https://github.com/actions/runner/pull/607, https://github.com/actions/runner/pull/605, https://github.com/actions/runner/pull/599

[thboop]

The uses keyword is now live on Github.com for composite actions 🎉 🎉 . GHES and GHAE support will come in future releases.

Docs and the changelog will be live in the near future, but I know a lot of you have been really excited for this feature, so I wanted to give you a little bit of time to preview it before then!

You can now specify containers, other composite actions (up to a depth of 9) and node actions in additional to the previously available run steps.

For example, maybe you want to reuse steps for your node project:

action.yaml

name: 'Composite Basic Node'
description: 'Composite Basic Node '
inputs:
  version: # determines what node version to install
    required: false
    description: 'input description here'
    default: '12'
outputs: {}
runs:
    using: "composite"
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: ${{inputs.version}}
      - run: npm test
         shell: bash

Or maybe you want to see the full list of ways you can reference actions and steps inside an action.yaml

runs:
  using: "composite"
  steps:
    # Reference a specific commit
    - uses: actions/checkout@a81bbbf8298c0fa03ea29cdc473d45769f953675
    # Reference the major version of a release
    - uses: actions/checkout@v2
    # Reference a specific version
    - uses: actions/checkout@v2.2.0
    # Reference a branch
    - uses: actions/checkout@main
    # References a subdirectory in a public GitHub repository at a specific branch, ref, or SHA
    - uses: actions/aws/ec2@main
    # References a local action
    - uses: ./.github/actions/my-action
    
    # linux only
    # References a docker public registry action
    - uses: docker://gcr.io/cloud-builders/gradle
    # Reference a docker image published on docker hub
    - uses: docker://alpine:3.8
 
    # and run steps!
    - run: echo "hello world"
      shell: bash

We have some feedback items already that will be going live in a runner release in the near future:

• Show more information about what step is currently being run in composite actions
• Bug: Failure() condition is not respected for composite actions

We also have some enhancements that we want to get to in the next stage of work:

• Composite Actions - Support the if conditional

If you find a bug, have a question, or want to provide feedback for a new enhancement related to composite actions, please open an issue in this repo and tag me!

[thboop]

I can’t give any information about timelines, but we are currently working on composite actions! 🚀

We plan on enabling the uses keyword, which allows us to support:

• Javascript actions (like @actions/checkout)
• Container actions
• Nested composite actions (up to a recursion limit)
• The pre and post steps these actions generate

I’ll post periodic updates in this issue as we make progress.

We have an ADR discussing more technical details as well. We welcome any feedback you wish to provide, particularly around use cases/features that would be helpful to you if they aren’t covered above!

Please post that feedback in the ADR!