Compare commits

...

95 Commits

Author SHA1 Message Date
Henry Mercer
b96794f015 Merge pull request #4131 from github/update-v4.38.0-7e08580a9
Merge main into releases/v4
2026-09-09 15:02:30 +01:00
github-actions[bot]
02d5093871 Update changelog for v4.38.0 2026-09-09 12:33:07 +00:00
Henry Mercer
7e08580a93 Merge pull request #4130 from github/henrymercer/workflow-runner-sizing
Run slow macOS checks on larger runners
2026-09-09 12:16:50 +00:00
Henry Mercer
bfcc52b4f5 Run slow macOS checks on larger runners
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-09 12:57:40 +01:00
Henry Mercer
8c251e757c Merge pull request #4129 from github/update-bundle/codeql-bundle-v2.27.0
Update default bundle to 2.27.0
2026-09-09 11:48:16 +00:00
github-actions[bot]
0b7ca400df Add changelog note 2026-09-09 11:33:54 +00:00
github-actions[bot]
40484b3395 Update default bundle to codeql-bundle-v2.27.0 2026-09-09 11:33:48 +00:00
Henry Mercer
977e6ceaea Merge pull request #4124 from github/henrymercer/toolcache-bundle-cleanup
Delete unused CodeQL bundles from the toolcache before downloading
2026-09-08 17:04:25 +00:00
Henry Mercer
40a6b38247 Address toolcache cleanup review feedback
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-08 15:55:41 +01:00
Henry Mercer
deece8f852 Apply suggestion from @henrymercer 2026-09-07 14:08:36 +01:00
copilot-swe-agent[bot]
034db721dd Merge remote-tracking branch 'origin/main' into henrymercer/toolcache-bundle-cleanup
Co-authored-by: henrymercer <14129055+henrymercer@users.noreply.github.com>
2026-09-07 12:57:51 +00:00
Henry Mercer
313a0b9922 Merge pull request #4122 from github/henrymercer/friendly-potato
Don't record an overlay status when the job was cancelled
2026-09-04 16:55:52 +00:00
Henry Mercer
31da345c07 Address review comments
Delete each version directory individually so that a symlinked one is skipped rather than removed, take an `ActionState` so the environment is read through `ReadOnlyEnv` rather than the deprecated `getOptionalEnvVar`, let `deleteToolcacheBundles` report its own failure to locate the toolcache instead of having the caller catch it, quote paths in log messages, and rename `HAS_OBTAINED_CODEQL_TOOLS` to `HAS_SET_UP_CODEQL`, which is also set when we find the tools in the toolcache rather than downloading them.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 17:43:40 +01:00
Henry Mercer
46dfb14111 Merge pull request #4123 from github/redsun82-address-pr-4072-comments
Analyze all supported languages in the Linux Arm64 check
2026-09-04 16:39:50 +00:00
Henry Mercer
e06b60f75f Merge pull request #4125 from github/henrymercer/merge-queue-sarif-upload-flake
Don't upload the ESLint SARIF from merge queue refs
2026-09-04 16:37:26 +00:00
Henry Mercer
a48f2d3077 Record an overlay status only for conclusive job statuses
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 17:33:02 +01:00
Henry Mercer
657964c39f Read the job status from a ReadOnlyEnv in recordOverlayStatus
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 17:31:24 +01:00
Henry Mercer
d2fe508a6d Don't upload the ESLint SARIF from merge queue refs
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 17:22:52 +01:00
Paolo Tranquilli
16da74769e Log found databases in Linux Arm64 check
Log a success line for each language database found in the Linux Arm64 PR check.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 12:27:23 +02:00
Henry Mercer
e13c3dc834 Only clean up a toolcache on the workspace filesystem
Some runner images keep the toolcache on a different volume to the workspace, so deleting the tools there frees up disk space that the analysis cannot use, and costs a later step that wanted them in the toolcache a download. Windows runners are laid out this way, with the toolcache on `C:` and the workspace on `D:`.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 11:00:56 +01:00
Henry Mercer
1331773b9a Address Copilot review feedback
Run the cleanup even when the download will not be cached in the toolcache, since the toolcache shares a filesystem with the directory we extract to, so freeing it helps either way, and report an error other than the toolcache being absent as a failure rather than as an empty toolcache.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:26:09 +01:00
Henry Mercer
2681b03bd6 Add a changelog note
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:18:43 +01:00
Henry Mercer
a7a90f3ffb Enable toolcache cleanup in the multi-language PR check
The check installs the tools from a URL, so most versions in its matrix are downloaded rather than found in the toolcache, which is when the cleanup runs, and it then builds and analyses seven languages, so a bundle we damaged on the way in would show up.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:12:11 +01:00
Henry Mercer
4a0b22ec25 Delete unused CodeQL bundles from the toolcache before downloading
When we download a bundle the toolcache often already holds a different one that the job will not use, and on GitHub-hosted runners it shares a filesystem with the workspace, so it takes space away from the analysis. Empty the toolcache before downloading, which also frees space for the archive during extraction, and which is safe because getting as far as a download means the tools were not resolved from the toolcache. Skip this once a step has obtained the tools, since a later step may run a path it was given, and gate it on the runner being GitHub-hosted and on a feature flag that is off by default.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:12:10 +01:00
Henry Mercer
762a5ed7f7 Add a helper to delete the CodeQL tools from the toolcache
`deleteToolcacheBundles` removes `$RUNNER_TOOL_CACHE/CodeQL` and reports which versions were there. It refuses to follow a symlinked CodeQL directory so that it can only ever delete paths that are really inside the toolcache, and reports failures rather than throwing. Not called yet.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:12:10 +01:00
Henry Mercer
ceb85f25b5 Distinguish GitHub-hosted runners from ones that look hosted
`isHostedRunner` infers hostedness from the runner name and the toolcache path, so it also matches self-hosted runners that are configured to resemble hosted ones. Rename it to `looksLikeHostedRunner` so callers can see they are getting a heuristic, and add `isGitHubHostedRunner`, which reads the `RUNNER_ENVIRONMENT` value the Actions service reports. The existing callers keep the heuristic, so there is no behaviour change.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:12:09 +01:00
Henry Mercer
9fddc16f0d Merge pull request #4088 from github/dependabot/npm_and_yarn/octokit/plugin-rest-endpoint-methods-18.0.0
Bump @octokit/plugin-rest-endpoint-methods from 17.0.0 to 18.0.0
2026-09-04 07:23:47 +00:00
Paolo Tranquilli
36cbf13b5e Address review comments
- Analyze all languages CodeQL supports on Linux Arm64 (except Swift,
  which is macOS only) in the Linux Arm64 check, building the compiled
  ones via the shared multi-language-repo build.sh
- De-duplicate the check's language list via a shared LANGUAGES env var
  used by both the init input and the assert loop
- Fix CHANGELOG tense: "download" -> "downloads"

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 08:30:10 +02:00
Henry Mercer
38dd4a088a Don't record an overlay status when the job was cancelled
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-03 18:21:46 +01:00
Henry Mercer
2f3c1c964a Add an internal job-status input to the init Action
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-03 18:21:14 +01:00
Michael B. Gale
5914b031b1 Merge pull request #4111 from github/dependabot/github_actions/dot-github/workflows/actions/setup-java-6.0.0
Bump actions/setup-java from 5.7.0 to 6.0.0 in /.github/workflows
2026-09-03 16:50:30 +00:00
Michael B. Gale
102499482f Merge pull request #4119 from github/dependabot/npm_and_yarn/npm-minor-21ac56865f
Bump the npm-minor group across 1 directory with 2 updates
2026-09-03 16:22:14 +00:00
Michael B. Gale
9401a335a3 Merge pull request #4118 from github/dependabot/npm_and_yarn/browserslist-4.28.8
Bump browserslist from 4.24.2 to 4.28.8
2026-09-03 16:21:07 +00:00
Michael B. Gale
3ee8a9398b Merge branch 'main' into dependabot/npm_and_yarn/browserslist-4.28.8 2026-09-03 17:03:16 +01:00
Michael B. Gale
25da1b1495 Merge branch 'main' into dependabot/npm_and_yarn/npm-minor-21ac56865f 2026-09-03 17:00:00 +01:00
Michael B. Gale
9509fd0822 Merge pull request #4120 from github/dependabot/npm_and_yarn/eslint-plugin-jsdoc-64.2.1
Bump eslint-plugin-jsdoc from 62.9.0 to 64.2.1
2026-09-03 15:58:16 +00:00
Michael B. Gale
20ec60c2d9 Merge branch 'main' into dependabot/npm_and_yarn/eslint-plugin-jsdoc-64.2.1 2026-09-03 15:12:29 +01:00
Paolo Tranquilli
4cccb3aa86 Merge pull request #4072 from github/redsun82-linux-arm64-support
Add support for Linux Arm64 runners
2026-09-03 13:58:15 +00:00
github-actions[bot]
b724a86493 Rebuild 2026-09-02 17:56:48 +00:00
dependabot[bot]
2a03009e3f Bump eslint-plugin-jsdoc from 62.9.0 to 64.2.1
Bumps [eslint-plugin-jsdoc](https://github.com/gajus/eslint-plugin-jsdoc) from 62.9.0 to 64.2.1.
- [Release notes](https://github.com/gajus/eslint-plugin-jsdoc/releases)
- [Commits](https://github.com/gajus/eslint-plugin-jsdoc/compare/v62.9.0...v64.2.1)

---
updated-dependencies:
- dependency-name: eslint-plugin-jsdoc
  dependency-version: 64.2.1
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-02 17:55:01 +00:00
dependabot[bot]
da21771e2e Bump the npm-minor group across 1 directory with 2 updates
Bumps the npm-minor group with 2 updates in the / directory: [js-yaml](https://github.com/nodeca/js-yaml) and [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint).


Updates `js-yaml` from 5.3.0 to 5.4.0
- [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/5.3.0...5.4.0)

Updates `typescript-eslint` from 8.67.0 to 8.68.0
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.68.0/packages/typescript-eslint)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 5.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: npm-minor
- dependency-name: typescript-eslint
  dependency-version: 8.68.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: npm-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-02 17:54:50 +00:00
dependabot[bot]
bb36f8049c Bump browserslist from 4.24.2 to 4.28.8
Bumps [browserslist](https://github.com/browserslist/browserslist) from 4.24.2 to 4.28.8.
- [Release notes](https://github.com/browserslist/browserslist/releases)
- [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md)
- [Commits](https://github.com/browserslist/browserslist/compare/4.24.2...4.28.8)

---
updated-dependencies:
- dependency-name: browserslist
  dependency-version: 4.28.8
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-02 13:55:34 +00:00
Michael B. Gale
276e2ce25d Merge pull request #4117 from github/mbg/deps/1sep
Update dependencies
2026-09-02 13:39:31 +00:00
Michael B. Gale
ccd5275aa4 Merge branch 'main' into dependabot/github_actions/dot-github/workflows/actions/setup-java-6.0.0 2026-09-02 14:36:24 +01:00
Mario Campos
a0c73122a6 Merge pull request #4096 from github/mario-campos/use-json-module
Use `json` module for JSON validation in `output-cache`
2026-09-01 15:17:48 +00:00
Michael B. Gale
50e244824e Update dependencies 2026-09-01 12:16:07 +01:00
Paolo Tranquilli
025009006c Address review comments
- Stub process.platform/arch in the supported-platform test and loop
  over all supported pairs (including linux/arm64) so it no longer
  depends on the host
- Run the default queries in the Linux Arm64 PR check so the databases
  are finalized end-to-end

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fb7e2d12-6620-4a67-9c1c-4e82f4a5e8d9
2026-08-28 17:28:29 +02:00
Mario Campos
6f530319d8 Merge pull request #4113 from github/mario-campos/add-changenote-script
Create `changetool` script for validating change-notes
2026-08-27 18:09:46 +00:00
Henry Mercer
1bcdd0c019 Derive the API client type from @actions/github
`ApiClient` was hand-composed from `Octokit`, `Api` and `PaginateInterface`, imported directly from separately versioned Octokit packages. That asserted a shape matching what `@actions/github` actually returns, which held only while the versions happened to agree.

`@octokit/plugin-rest-endpoint-methods` v18 adds twelve Actions cache-limit methods. `@actions/github` still depends on `^17.0.0`, so the client it constructs no longer satisfies the v18-derived alias and the build fails.

Deriving the type from the constructor removes the assumption, so the alias tracks whatever `@actions/github` returns.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-27 18:56:47 +01:00
Mario Campos
f049ecb9ce Delete unused Node.js flag --experimental-test-module-mocks
Co-authored-by: Henry Mercer <henrymercer@github.com>
2026-08-27 11:22:56 -05:00
dependabot[bot]
25694c08ea Bump @octokit/plugin-rest-endpoint-methods from 17.0.0 to 18.0.0
Bumps [@octokit/plugin-rest-endpoint-methods](https://github.com/octokit/plugin-rest-endpoint-methods.js) from 17.0.0 to 18.0.0.
- [Release notes](https://github.com/octokit/plugin-rest-endpoint-methods.js/releases)
- [Commits](https://github.com/octokit/plugin-rest-endpoint-methods.js/compare/v17.0.0...v18.0.0)

---
updated-dependencies:
- dependency-name: "@octokit/plugin-rest-endpoint-methods"
  dependency-version: 18.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-27 11:01:07 +00:00
Henry Mercer
0963041ab4 Merge pull request #4090 from github/dependabot/npm_and_yarn/octokit/plugin-paginate-rest-15.0.0
Bump @octokit/plugin-paginate-rest from 14.0.0 to 15.0.0
2026-08-27 10:45:55 +00:00
github-actions[bot]
1a8ddd7325 Rebuild 2026-08-27 10:32:49 +00:00
Henry Mercer
abc579a511 Merge pull request #4112 from github/dependabot/npm_and_yarn/npm-minor-62a3d68ea9
Bump the npm-minor group across 1 directory with 2 updates
2026-08-27 10:28:51 +00:00
dependabot[bot]
4d1d53ec73 Bump @octokit/plugin-paginate-rest from 14.0.0 to 15.0.0
Bumps [@octokit/plugin-paginate-rest](https://github.com/octokit/plugin-paginate-rest.js) from 14.0.0 to 15.0.0.
- [Release notes](https://github.com/octokit/plugin-paginate-rest.js/releases)
- [Commits](https://github.com/octokit/plugin-paginate-rest.js/compare/v14.0.0...v15.0.0)

---
updated-dependencies:
- dependency-name: "@octokit/plugin-paginate-rest"
  dependency-version: 15.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-27 10:08:06 +00:00
Henry Mercer
6d4d5dfcd6 Merge pull request #4089 from github/dependabot/npm_and_yarn/octokit/types-17.0.0
Bump @octokit/types from 16.0.0 to 17.0.0
2026-08-27 09:37:27 +00:00
Paolo Tranquilli
7acc225a21 Merge pull request #4110 from github/redsun82-release-merge-instructions
Document merge-commit requirement for release PRs
2026-08-27 08:26:42 +00:00
Paolo Tranquilli
76ba03be44 Merge branch 'main' into redsun82-release-merge-instructions 2026-08-27 10:10:14 +02:00
Paolo Tranquilli
6441b26a49 Merge remote-tracking branch 'origin/main' into redsun82-linux-arm64-support 2026-08-27 10:09:22 +02:00
Mario Campos
cacb7b37ed Delete unused script/changetool NPM scripts 2026-08-26 23:39:11 -05:00
Mario Campos
e2cd31e32e Refactor tsconfig.json to extend base configuration and simplify options 2026-08-26 23:32:24 -05:00
Mario Campos
be555cf0a3 Extend scripts/changetool as a NPM workspace 2026-08-26 23:25:06 -05:00
Mario Campos
0447ab23b5 Enhance changenote validation to support nested bullet lists 2026-08-26 23:06:47 -05:00
Mario Campos
50ec5a5c18 Use Object.hasOwn instead of in operator
Apparently, the `in` operator will check inherited properties too.
2026-08-26 22:23:00 -05:00
Mario Campos
5aea817801 Replace Array.prototype.every() with for loop
`.every()` short-circuits on the first `false`
2026-08-26 22:20:25 -05:00
Mario Campos
580da893f6 Handle file errors as false 2026-08-26 22:15:34 -05:00
Mario Campos
426c10420b Create changetool script for validating change-notes 2026-08-26 17:13:40 -05:00
github-actions[bot]
dc5bc53050 Rebuild 2026-08-26 18:07:30 +00:00
github-actions[bot]
d75af13705 Rebuild 2026-08-26 18:03:17 +00:00
dependabot[bot]
0006c77502 Bump the npm-minor group across 1 directory with 2 updates
Bumps the npm-minor group with 2 updates in the / directory: [js-yaml](https://github.com/nodeca/js-yaml) and [uuid](https://github.com/uuidjs/uuid).


Updates `js-yaml` from 5.2.3 to 5.3.0
- [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/5.2.3...5.3.0)

Updates `uuid` from 14.0.1 to 14.0.2
- [Release notes](https://github.com/uuidjs/uuid/releases)
- [Changelog](https://github.com/uuidjs/uuid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/uuidjs/uuid/compare/v14.0.1...v14.0.2)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 5.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: npm-minor
- dependency-name: uuid
  dependency-version: 14.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: npm-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-26 18:01:20 +00:00
dependabot[bot]
ec1bd1999f Bump actions/setup-java from 5.7.0 to 6.0.0 in /.github/workflows
Bumps [actions/setup-java](https://github.com/actions/setup-java) from 5.7.0 to 6.0.0.
- [Release notes](https://github.com/actions/setup-java/releases)
- [Commits](b6effb05e4...dd06d9cba3)

---
updated-dependencies:
- dependency-name: actions/setup-java
  dependency-version: 6.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-26 17:59:42 +00:00
Henry Mercer
4d40c93072 Merge pull request #4108 from github/mergeback/v4.37.9-to-main-cdf488f5
Mergeback v4.37.9 refs/heads/releases/v4 into main
2026-08-26 17:19:52 +00:00
Henry Mercer
3e93618b99 Merge pull request #4105 from github/henrymercer/tools-download-telemetry
Improve CodeQL tools download time telemetry
2026-08-26 17:01:23 +00:00
Paolo Tranquilli
42c2ea9ef7 Document merge-commit requirement for release PRs
Release, mergeback, and backport PRs must be merged with a merge commit
so the branch linkage the release automation depends on is preserved
(the mergeback tags the release using the merge commit of the
"Merge main into releases/vN" PR, so squashing or rebasing breaks
tagging).

Add a path-scoped Copilot instructions file scoped to the files that
reliably change in these PRs (CHANGELOG.md, src/defaults.json,
lib/defaults.json, src/api-compatibility.json), and add an explicit note
to the Releasing runbook in CONTRIBUTING.md so the rule is enforced
regardless of who merges.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0dae6e42-7fc6-4fb1-b1ea-15dc17e402a2
2026-08-26 17:13:12 +02:00
github-actions[bot]
f37565646f Rebuild 2026-08-26 14:41:38 +00:00
github-actions[bot]
0f2e2bde5c Update changelog and version after v4.37.9 2026-08-26 14:41:24 +00:00
Paolo Tranquilli
b5383aad77 Merge remote-tracking branch 'origin/redsun82-linux-arm64-support' into redsun82-linux-arm64-support 2026-08-25 16:08:34 +02:00
Henry Mercer
9d89e2d1d6 Rebuild
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 628ce334-991a-4578-9c1b-93d2e96bbddb
2026-08-24 18:32:11 +01:00
Henry Mercer
bee82de8ba Add tools download durations to the init status report
Surface `tools_extraction_duration_ms` and `tools_total_duration_ms` from
both the `init` and `setup-codeql` actions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 628ce334-991a-4578-9c1b-93d2e96bbddb
2026-08-24 18:32:08 +01:00
Henry Mercer
c05e445696 Report tools download durations on both download paths
The streaming path reported no timings at all, so we have no data for the
path that most runs take. It now reports a total duration, which is also
populated on the download-then-extract path.

That path additionally reports the extraction duration, which was
previously computed but only logged.

`downloadDurationMs` keeps its existing meaning of time spent downloading
alone, so existing telemetry stays comparable.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 628ce334-991a-4578-9c1b-93d2e96bbddb
2026-08-24 18:32:04 +01:00
Paolo Tranquilli
a952c0706c Merge branch 'main' into redsun82-linux-arm64-support 2026-08-24 10:55:44 +02:00
Paolo Tranquilli
97b7459e09 Merge remote-tracking branch 'origin/main' into redsun82-linux-arm64-support 2026-08-21 17:43:09 +02:00
Mario Campos
fcd8d74cda Refactor output-caching functions to accept file path dependency
This makes it easier to test. Credit to @mbg.
2026-08-14 14:41:04 -05:00
Mario Campos
b5d34388b7 Refactor tests in output-cache.test.ts to use named imports from output-cache 2026-08-14 14:28:17 -05:00
Mario Campos
1f46830a1f Fix VersionInfo derived type to pass linter 2026-08-14 14:24:37 -05:00
Mario Campos
b60777a215 Delete unnecessary statement from JSDoc comments 2026-08-14 14:05:42 -05:00
Mario Campos
f3deecb42d Refactor OutputCache with json module 2026-08-14 11:38:38 -05:00
Mario Campos
c8ba2d2a05 Refactor isVersionInfo with json module 2026-08-14 11:35:14 -05:00
Mario Campos
d7d901a925 Refactor tests to use getCommandCacheFilePath
As well as fixup a few other problems.
2026-08-14 10:56:07 -05:00
Mario Campos
cb9d39fcc1 Export getCommandCacheFilePath function for use in tests 2026-08-14 10:55:33 -05:00
Mario Campos
93d793c5fe Use json module for JSON validation in output-cache 2026-08-13 23:19:37 -05:00
dependabot[bot]
9875827ada Bump @octokit/types from 16.0.0 to 17.0.0
Bumps [@octokit/types](https://github.com/octokit/types.ts) from 16.0.0 to 17.0.0.
- [Release notes](https://github.com/octokit/types.ts/releases)
- [Commits](https://github.com/octokit/types.ts/compare/v16.0.0...v17.0.0)

---
updated-dependencies:
- dependency-name: "@octokit/types"
  dependency-version: 17.0.0
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-13 10:32:23 +00:00
Paolo Tranquilli
b4c390c362 Merge remote-tracking branch 'origin/main' into redsun82-linux-arm64-support 2026-08-10 09:51:23 +02:00
Paolo Tranquilli
43ae5e70b0 Address review comments
- Add darwin/x64 case to linked bundle tests and disambiguate the
  parametrized test title by platform/arch
- Reword CHANGELOG entry to clarify the CLI supports Linux Arm64
- Add a dedicated Linux Arm64 end-to-end PR check on ubuntu-24.04-arm

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 308c78bb-53a3-4877-a373-70fab87543f1
2026-08-05 15:51:01 +02:00
Paolo Tranquilli
4c96123771 Add support for linux/arm64
Recognize the linux/arm64 platform so the CodeQL CLI is accepted on
Arm64 Linux runners and the native per-platform bundle
codeql-bundle-linux-arm64.tar.zst is resolved, rather than falling
through to the combined all-platforms gzip fallback (which deliberately
omits arm64).

Add ["linux", "arm64"] to the supported platforms and make the linux
bundle name arch-aware. Keep the download mock faithful and cover the
new arch branch with tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-04 15:32:12 +02:00
54 changed files with 4424 additions and 1370 deletions

View File

@@ -0,0 +1,16 @@
---
applyTo: "CHANGELOG.md,src/defaults.json,lib/defaults.json,src/api-compatibility.json"
---
# Merging release, mergeback, and backport PRs
The release process creates a cascade of PRs (`main` → `releases/vN`, then
`releases/vN` → `main` mergeback, then `releases/vN` → `releases/v(N-1)`
backport). These PRs reliably touch `CHANGELOG.md`, `src/defaults.json` /
`lib/defaults.json` (bundle/CLI version bump), and `src/api-compatibility.json`.
Such PRs **must be merged with a merge commit**. Never squash or rebase, as
that breaks the branch linkage the release automation relies on.
When arming auto-merge on these PRs, use `--merge` (e.g. `gh pr merge --merge`),
not `--squash` or `--rebase`.

View File

@@ -56,7 +56,7 @@ jobs:
include:
- os: ubuntu-latest
version: nightly-latest
- os: macos-latest
- os: macos-latest-xlarge
version: nightly-latest
- os: windows-latest
version: nightly-latest

View File

@@ -63,7 +63,7 @@ jobs:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Java
uses: actions/setup-java@b6effb05e454b25005698d916606bdc6ffcbf961 # v5.7.0
uses: actions/setup-java@dd06d9cba3e5552c54d9f8ea23572deb30010f7c # v6.0.0
with:
java-version: ${{ inputs.java-version || '17' }}
distribution: temurin

View File

@@ -63,7 +63,7 @@ jobs:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Java
uses: actions/setup-java@b6effb05e454b25005698d916606bdc6ffcbf961 # v5.7.0
uses: actions/setup-java@dd06d9cba3e5552c54d9f8ea23572deb30010f7c # v6.0.0
with:
java-version: ${{ inputs.java-version || '17' }}
distribution: temurin

106
.github/workflows/__linux-arm64.yml generated vendored Normal file
View File

@@ -0,0 +1,106 @@
# Warning: This file is generated automatically, and should not be modified.
# Instead, please modify the template in the pr-checks directory and run:
# pr-checks/sync.sh
# to regenerate this file.
name: PR Check - Linux Arm64
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GO111MODULE: auto
on:
push:
branches:
- main
- releases/v*
pull_request: {}
merge_group:
types:
- checks_requested
schedule:
- cron: '0 5 * * *'
workflow_dispatch:
inputs:
dotnet-version:
type: string
description: The version of .NET to install
required: false
default: 9.x
go-version:
type: string
description: The version of Go to install
required: false
default: '>=1.21.0'
workflow_call:
inputs:
dotnet-version:
type: string
description: The version of .NET to install
required: false
default: 9.x
go-version:
type: string
description: The version of Go to install
required: false
default: '>=1.21.0'
defaults:
run:
shell: bash
concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' || false }}
group: linux-arm64-${{github.ref}}-${{inputs.dotnet-version}}-${{inputs.go-version}}
jobs:
linux-arm64:
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04-arm
version: nightly-latest
name: Linux Arm64
if: github.triggering_actor != 'dependabot[bot]'
permissions:
contents: read
security-events: read
timeout-minutes: 45
runs-on: ${{ matrix.os }}
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install .NET
uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0
with:
dotnet-version: ${{ inputs.dotnet-version || '9.x' }}
- name: Install Go
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version: ${{ inputs.go-version || '>=1.21.0' }}
cache: false
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
with:
version: ${{ matrix.version }}
use-all-platform-bundle: 'false'
setup-kotlin: 'true'
- uses: ./../action/init
with:
languages: ${{ env.LANGUAGES }}
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Build code
run: ./build.sh
- uses: ./../action/analyze
with:
upload-database: false
- name: Assert databases exist
run: |
cd "$RUNNER_TEMP/codeql_databases"
for lang in ${LANGUAGES//,/ }; do
if [[ ! -d "$lang" ]]; then
echo "Did not find a database for $lang"
exit 1
fi
echo "Found database for $lang"
done
env:
LANGUAGES: cpp,csharp,go,java,javascript,python,ruby
CODEQL_ACTION_TEST_MODE: true

View File

@@ -191,5 +191,6 @@ jobs:
exit 1
fi
env:
CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES: true
CODEQL_ACTION_RESOLVE_SUPPORTED_LANGUAGES_USING_CLI: true
CODEQL_ACTION_TEST_MODE: true

View File

@@ -54,11 +54,11 @@ jobs:
fail-fast: false
matrix:
include:
- os: macos-latest
- os: macos-latest-xlarge
version: linked
- os: macos-latest
- os: macos-latest-xlarge
version: default
- os: macos-latest
- os: macos-latest-xlarge
version: nightly-latest
name: Swift analysis using a custom build command
if: github.triggering_actor != 'dependabot[bot]'

View File

@@ -67,11 +67,43 @@ jobs:
- name: Upload sarif
uses: ./upload-sarif
if: matrix.os == 'ubuntu-latest' && matrix.node-version == 24
# The merge queue deletes its `gh-readonly-queue` ref as soon as the queue entry resolves,
# so uploading against it races with that deletion. Both the `merge_group` run and the
# paired `push` run that the queue branch creates use that ref, so gate on the ref itself
# rather than the event. The same results are uploaded by the `pull_request` run and again
# by the `push` run on `main`.
if: matrix.os == 'ubuntu-latest' && matrix.node-version == 24 && !startsWith(github.ref, 'refs/heads/gh-readonly-queue/')
with:
sarif_file: eslint.sarif
category: eslint
changetool-tests:
name: changetool unit tests
permissions:
contents: read
runs-on: ubuntu-slim
timeout-minutes: 10
concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' || false }}
group: pr-checks-changetool-tests-${{ github.ref }}-${{ github.event_name }}
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 24
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run changetool unit tests
run: npm --workspace changetool test
# These checks do not need to be run as part of the same matrix that we use for the `unit-tests`
# job.
other-checks:

View File

@@ -2,6 +2,12 @@
See the [releases page](https://github.com/github/codeql-action/releases) for the relevant changes to the CodeQL CLI and language packs.
## 4.38.0 - 09 Sept 2026
- On GitHub-hosted runners, the CodeQL Action now deletes unused CodeQL bundles from the toolcache before downloading a different bundle, which frees up disk space for the analysis. We expect to roll this change out to everyone in September. [#4124](https://github.com/github/codeql-action/pull/4124)
- The CodeQL Action now supports CodeQL releases that are compatible with Linux Arm64 and downloads the native `linux-arm64` CodeQL bundle when available. [#4072](https://github.com/github/codeql-action/pull/4072)
- Update default CodeQL bundle version to [2.27.0](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.27.0). [#4129](https://github.com/github/codeql-action/pull/4129)
## 4.37.9 - 26 Aug 2026
- Update default CodeQL bundle version to [2.26.4](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.26.4). [#4106](https://github.com/github/codeql-action/pull/4106)

View File

@@ -60,10 +60,13 @@ Here are a few things you can do that will increase the likelihood of your pull
This workflow goes through the pull requests that have been merged to `main` since the last release, creates a changelog, then opens a pull request to merge the changes since the last release into the `releases/v3` release branch.
You can start a release by triggering this workflow via [workflow dispatch](https://github.com/github/codeql-action/actions/workflows/update-release-branch.yml).
1. The workflow run will open a pull request titled "Merge main into releases/v3". Follow the steps on the checklist in the pull request. Once you've checked off all but the last two of these, approve the PR and automerge it.
1. The workflow run will open a pull request titled "Merge main into releases/v3". Follow the steps on the checklist in the pull request. Once you've checked off all but the last two of these, approve the PR and automerge it **with a merge commit** (`gh pr merge --merge`).
1. When the "Merge main into releases/v3" pull request is merged into the `releases/v3` branch, a mergeback pull request to `main` will be automatically created. This mergeback pull request incorporates the changelog updates into `main`, tags the release using the merge commit of the "Merge main into releases/v3" pull request, and bumps the patch version of the CodeQL Action.
1. If a backport to an older major version is required, a pull request targeting that version's branch will also be automatically created.
1. Approve the mergeback and backport pull request (if applicable) and automerge them.
1. Approve the mergeback and backport pull request (if applicable) and automerge them **with a merge commit** (`gh pr merge --merge`).
> [!NOTE]
> The release, mergeback, and backport pull requests must always be merged with a merge commit — **never squash or rebase**. The mergeback tags the release using the merge commit of the "Merge main into releases/v3" pull request, so squashing or rebasing breaks tagging and the branch linkage the release automation relies on.
Once the mergeback and backport pull request have been merged, the release is complete.

View File

@@ -209,4 +209,18 @@ export default [
],
},
},
{
files: ["scripts/changetool/**/*.ts"],
languageOptions: {
parserOptions: {
project: "./scripts/changetool/tsconfig.json",
},
},
rules: {
"no-console": "off",
"import/extensions": "off",
},
},
];

View File

@@ -164,6 +164,13 @@ inputs:
[Internal] The ID of the check run, as provided by the Actions runtime environment. Do not set this value manually.
default: ${{ job.check_run_id }}
required: false
job-status:
description: >-
[Internal] The status of the job, as provided by the Actions runtime environment. This is how the
post step learns whether the job as a whole succeeded, failed, or was cancelled. Do not set this
value manually.
default: ${{ job.status }}
required: false
outputs:
codeql-path:
description: The path of the CodeQL binary used for analysis

View File

@@ -1,6 +1,6 @@
{
"bundleVersion": "codeql-bundle-v2.26.4",
"cliVersion": "2.26.4",
"priorBundleVersion": "codeql-bundle-v2.26.3",
"priorCliVersion": "2.26.3"
"bundleVersion": "codeql-bundle-v2.27.0",
"cliVersion": "2.27.0",
"priorBundleVersion": "codeql-bundle-v2.26.4",
"priorCliVersion": "2.26.4"
}

2190
lib/entry-points.js generated

File diff suppressed because it is too large Load Diff

1517
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
{
"name": "codeql",
"version": "4.37.9",
"version": "4.38.0",
"private": true,
"description": "CodeQL action",
"scripts": {
@@ -17,7 +17,8 @@
},
"license": "MIT",
"workspaces": [
"pr-checks"
"pr-checks",
"scripts/changetool"
],
"dependencies": {
"@actions/artifact": "^5.0.3",
@@ -31,27 +32,27 @@
"@actions/io": "^2.0.0",
"@actions/tool-cache": "^3.0.1",
"@octokit/core": "^7.0.7",
"@octokit/plugin-paginate-rest": "^14.0.0",
"@octokit/plugin-rest-endpoint-methods": "^17.0.0",
"@octokit/plugin-paginate-rest": "^15.0.0",
"@octokit/plugin-rest-endpoint-methods": "^18.0.0",
"@octokit/plugin-retry": "^8.1.1",
"archiver": "^8.0.0",
"fast-deep-equal": "^3.1.3",
"follow-redirects": "^1.16.0",
"get-folder-size": "^5.0.0",
"https-proxy-agent": "^7.0.6",
"js-yaml": "^5.2.3",
"js-yaml": "^5.4.0",
"jsonschema": "1.5.0",
"long": "^5.3.2",
"node-forge": "^1.4.0",
"semver": "^7.8.5",
"uuid": "^14.0.1",
"uuid": "^14.0.2",
"undici": "^6.28.0"
},
"devDependencies": {
"@ava/typescript": "6.0.0",
"@eslint/compat": "^2.1.0",
"@microsoft/eslint-formatter-sarif": "^3.1.0",
"@octokit/types": "^16.0.0",
"@octokit/types": "^17.0.0",
"@types/archiver": "^8.0.0",
"@types/follow-redirects": "^1.14.4",
"@types/js-yaml": "^4.0.9",
@@ -66,14 +67,14 @@
"eslint-import-resolver-typescript": "^4.4.5",
"eslint-plugin-github": "^6.1.2",
"eslint-plugin-import-x": "^4.17.1",
"eslint-plugin-jsdoc": "^62.9.0",
"eslint-plugin-jsdoc": "^64.2.1",
"eslint-plugin-no-async-foreach": "^0.1.1",
"glob": "^13.0.6",
"globals": "^17.11.0",
"nock": "^14.0.17",
"sinon": "^22.1.0",
"typescript": "^6.0.3",
"typescript-eslint": "^8.67.0"
"typescript-eslint": "^8.68.0"
},
"overrides": {
"@actions/tool-cache": {

View File

@@ -1,10 +1,7 @@
import * as githubUtils from "@actions/github/lib/utils";
import { type Octokit } from "@octokit/core";
import { type PaginateInterface } from "@octokit/plugin-paginate-rest";
import { type Api } from "@octokit/plugin-rest-endpoint-methods";
/** The type of the Octokit client. */
export type ApiClient = Octokit & Api & { paginate: PaginateInterface };
export type ApiClient = InstanceType<typeof githubUtils.GitHub>;
/** Constructs an `ApiClient` using `token` for authentication. */
export function getApiClient(token: string): ApiClient {

View File

@@ -2,7 +2,8 @@ name: "All-platform bundle"
description: "Tests using an all-platform CodeQL Bundle"
operatingSystems:
- ubuntu
- macos
- os: macos
runner-image: macos-latest-xlarge
- windows
versions:
- nightly-latest

View File

@@ -0,0 +1,35 @@
name: "Linux Arm64"
description: "An end-to-end integration test running on a Linux Arm64 runner, checking that the native linux-arm64 CodeQL bundle is downloaded and can analyze interpreted and compiled code"
operatingSystems:
- os: ubuntu
runner-image: ubuntu-24.04-arm
# The native linux-arm64 CodeQL bundle is only available in recent CLI releases, so we restrict this
# check to `nightly-latest`, which is guaranteed to ship it. Older stable versions do not have an
# arm64 asset, and `prepare-test` would resolve an x64 bundle URL for them on this runner.
versions:
- nightly-latest
installGo: true
installDotNet: true
# The set of languages CodeQL supports on this platform, excluding Swift (macOS only).
env:
LANGUAGES: cpp,csharp,go,java,javascript,python,ruby
steps:
- uses: ./../action/init
with:
languages: ${{ env.LANGUAGES }}
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Build code
run: ./build.sh
- uses: ./../action/analyze
with:
upload-database: false
- name: Assert databases exist
run: |
cd "$RUNNER_TEMP/codeql_databases"
for lang in ${LANGUAGES//,/ }; do
if [[ ! -d "$lang" ]]; then
echo "Did not find a database for $lang"
exit 1
fi
echo "Found database for $lang"
done

View File

@@ -15,6 +15,7 @@ operatingSystems:
- stable-v2.21.4
- stable-v2.22.4
env:
CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES: true
CODEQL_ACTION_RESOLVE_SUPPORTED_LANGUAGES_USING_CLI: true
installGo: true
installDotNet: true

View File

@@ -5,7 +5,8 @@ versions:
- default
- nightly-latest
operatingSystems:
- macos
- os: macos
runner-image: macos-latest-xlarge
installGo: true
installDotNet: true
env:

View File

@@ -5,8 +5,8 @@
"@actions/core": "^2.0.3",
"@actions/github": "^8.0.1",
"@octokit/core": "^7.0.7",
"@octokit/plugin-paginate-rest": ">=9.2.2",
"@octokit/plugin-rest-endpoint-methods": "^17.0.0",
"@octokit/plugin-paginate-rest": ">=15.0.0",
"@octokit/plugin-rest-endpoint-methods": "^18.0.0",
"semver": "^7.8.5",
"yaml": "^2.9.0"
},

View File

@@ -253,8 +253,8 @@ const languageSetups: LanguageSetups = {
name: "Install Java",
uses: pinnedUses(
"actions/setup-java",
"b6effb05e454b25005698d916606bdc6ffcbf961",
"v5.7.0",
"dd06d9cba3e5552c54d9f8ea23572deb30010f7c",
"v6.0.0",
),
with: {
"java-version": `\${{ inputs.java-version || '${defaultLanguageVersions.java}' }}`,

View File

@@ -0,0 +1,200 @@
import assert from "node:assert/strict";
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import { describe, it } from "node:test";
import {
isValidChangenoteContent,
isValidChangenoteFile,
isValidChangenoteFilename,
hasValidChangenoteCategory,
VALID_CHANGE_NOTE_CATEGORIES,
} from "./validate.ts";
async function withTmpFile<T>(
baseFileName: string,
contents: string,
body: (filePath: string) => Promise<T>,
): Promise<T> {
const tmpDir = fs.mkdtempSync(
path.join(os.tmpdir(), "changetool-validate-test-"),
);
try {
const filePath = path.join(tmpDir, baseFileName);
fs.writeFileSync(filePath, contents);
return await body(filePath);
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
}
await describe("isValidChangenoteContent", async () => {
await it("recognizes an unordered Markdown list", async () => {
const inputs = [
"- One changenote entry",
"- First item\n- Second item",
"\n\n\n\n- Fixed a bug\n- Added a feature",
];
for (const input of inputs) {
assert.equal(isValidChangenoteContent(input), true);
}
});
await it("does not recognize non-Markdown text", async () => {
const inputs = [
"This is not a list.",
'["this", "is", "JSON"]',
"---",
"***",
"___",
"paragraph",
];
for (const input of inputs) {
assert.equal(isValidChangenoteContent(input), false);
}
});
await it("does not recognize ordered Markdown lists", async () => {
const inputs = [
"1. First item\n2. Second item",
"\n\n\n1. First item\n1. Second item",
];
for (const input of inputs) {
assert.equal(isValidChangenoteContent(input), false);
}
});
await it("requires all list items to use a hyphen bullet", async () => {
const inputs = [
"* Fixed a bug\n* Added feature",
"+ Fixed a bug\n+ Added feature",
"- Fixed a bug\n* Added feature",
"- Fixed a bug\n+ Added feature",
"- Fixed a bug\n * Added feature\n + Updated docs",
"\n\n\n* Fixed a bug",
"\n\n\n+ Fixed a bug",
"---\n* Fixed a bug\n* Added feature",
] as const;
for (const input of inputs) {
assert.equal(isValidChangenoteContent(input), false);
}
});
await it("does not contain other Markdown elements", async () => {
const inputs = [
"- Fixed a bug\n\nParagraph of text",
"- Fixed a bug\n\n* Added a feature",
"# Header\n- Fixed a bug",
"- Fixed a bug\n## Subheader",
];
for (const input of inputs) {
assert.equal(isValidChangenoteContent(input), false);
}
});
});
await describe("isValidChangenoteFilename", async () => {
await it("accepts valid filenames", async () => {
const inputs = [
"2023-01-01-fix-bug.md",
"2023-12-31-add-feature.md",
"2023-06-15-update-docs.md",
];
for (const input of inputs) {
assert.equal(isValidChangenoteFilename(input), true);
}
});
await it("rejects invalid filenames", async () => {
const inputs = [
"missing-date-from-filename.md",
"2021-01-01.md",
"2026-12-19-wrong-file-name-extension.txt",
];
for (const input of inputs) {
assert.equal(isValidChangenoteFilename(input), false);
}
});
});
await describe("hasValidChangenoteCategory", async () => {
await it("accepts valid categories", async () => {
for (const category of Object.keys(VALID_CHANGE_NOTE_CATEGORIES)) {
const frontmatter = { category };
assert.equal(hasValidChangenoteCategory(frontmatter), true);
}
});
await it("rejects invalid categories", async () => {
const inputs = [
"",
"invalid-category",
"bug-fix",
"new-feature",
"security-patch",
"miscellaneous",
"documentation",
];
for (const category of inputs) {
const frontmatter = { category };
assert.equal(hasValidChangenoteCategory(frontmatter), false);
}
});
await it("reject missing category", async () => {
assert.equal(hasValidChangenoteCategory({}), false);
assert.equal(hasValidChangenoteCategory({ category: null }), false);
assert.equal(hasValidChangenoteCategory({ category: undefined }), false);
});
});
await describe("isValidChangenoteFile", async () => {
await it("accepts a valid change-note file", async () => {
await withTmpFile(
"2026-01-01-fix-bug.md",
"---\ncategory: fix\n---\n- Fixed a bug\n",
async (filePath) => {
assert.equal(isValidChangenoteFile(filePath), true);
},
);
});
await it("rejects invalid filename", async () => {
await withTmpFile(
"fix-bug.md",
"---\ncategory: fix\n---\n- Fixed a bug\n",
async (filePath) => {
assert.equal(isValidChangenoteFile(filePath), false);
},
);
});
await it("rejects missing frontmatter", async () => {
await withTmpFile(
"2026-01-01-fix-bug.md",
"- Fixed a bug\n",
async (filePath) => {
assert.equal(isValidChangenoteFile(filePath), false);
},
);
});
await it("rejects invalid Markdown", async () => {
await withTmpFile(
"2026-01-01-fix-bug.md",
"---\ncategory: fix\n---\n* Fixed a bug\n",
async (filePath) => {
assert.equal(isValidChangenoteFile(filePath), false);
},
);
});
});

View File

@@ -0,0 +1,121 @@
import * as fs from "node:fs";
import * as path from "node:path";
import { matter } from "lite-matter";
import type { List, ListItem } from "mdast";
import { fromMarkdown } from "mdast-util-from-markdown";
// Regex for filename: YYYY-MM-DD-id.md
const VALID_CHANGE_NOTE_FILENAME_PATTERN =
/^(\d{4})-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])-([a-z0-9]+(?:-[a-z0-9]+)*)\.md$/;
export const VALID_CHANGE_NOTE_CATEGORIES = {
breaking: "Breaking Changes",
feature: "New Features",
improvement: "Improvements",
securityFix: "Security Fixes",
fix: "Bug Fixes",
unship: "Removed Features",
deprecation: "Deprecations",
knownIssue: "Known Issues",
misc: "Miscellaneous",
};
/**
* Validates that the given Markdown string meets the criteria for a change-note, which is:
* - A single unordered list
* - Each list item must start with a hyphen (-)
* - No other Markdown elements are allowed
* @param content The Markdown string to validate
* @returns True if the string is a valid change-note, false otherwise
*/
export function isValidChangenoteContent(content: string): boolean {
const ast = fromMarkdown(content);
const lines = content.split("\n");
function listHasHyphenBullets(node: List | ListItem): boolean {
if (node.type === "list") {
return node.children.every(listHasHyphenBullets);
}
const line = lines[node.position!.start.line - 1].trim();
return (
line.startsWith("-") &&
node.children.every(
(child) => child.type !== "list" || listHasHyphenBullets(child),
)
);
}
return (
ast.children.length === 1 &&
ast.children[0].type === "list" &&
ast.children[0].ordered === false &&
listHasHyphenBullets(ast.children[0])
);
}
/**
* Validates that the given filename meets the criteria for a change-note filename.
* @param filename The name of the change-note file to validate.
* @returns True if the filename is valid, false otherwise.
*/
export function isValidChangenoteFilename(filename: string): boolean {
return filename.match(VALID_CHANGE_NOTE_FILENAME_PATTERN) !== null;
}
/**
* Validates that the given frontmatter has a valid change-note category.
* @param frontmatter The frontmatter object to validate.
* @returns True if the frontmatter has a valid category, false otherwise.
*/
export function hasValidChangenoteCategory(
frontmatter: Record<string, unknown>,
): boolean {
const category = frontmatter["category"];
return (
typeof category === "string" &&
Object.hasOwn(VALID_CHANGE_NOTE_CATEGORIES, category)
);
}
/**
* Validates that the given change-note file meets all of the criteria for a change-note.
* @param filename The name of the change-note file to validate.
* @returns True if the file is a valid change-note, false otherwise.
*/
export function isValidChangenoteFile(filename: string): boolean {
let isValid: boolean = true;
let fileData: string | undefined;
try {
fileData = fs.readFileSync(filename, "utf8");
} catch (error) {
console.error(`${filename}: failed to read file`, error);
return false;
}
const { data: frontmatter, content } = matter(fileData);
if (!isValidChangenoteFilename(path.basename(filename))) {
isValid = false;
console.error(
`${filename}: invalid filename; must match pattern YYYY-MM-DD-id.md`,
);
}
if (!hasValidChangenoteCategory(frontmatter)) {
isValid = false;
const categories = Object.keys(VALID_CHANGE_NOTE_CATEGORIES).join(", ");
console.error(
`${filename}: invalid category; must be one of: ${categories}`,
);
}
if (!isValidChangenoteContent(content)) {
isValid = false;
console.error(
`${filename}: invalid Markdown; content must be a single unordered list with hyphen bullets and no other Markdown elements`,
);
}
return isValid;
}

View File

@@ -0,0 +1,51 @@
import { pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import { isValidChangenoteFile } from "./cli/validate.ts";
const entryPoint = process.argv[1];
if (entryPoint && import.meta.url === pathToFileURL(entryPoint).href) {
try {
process.exit(main());
} catch (error) {
console.error(error);
process.exit(1);
}
}
function main(): number {
const { positionals } = parseArgs({
allowPositionals: true,
strict: true,
});
const [command, ...paths] = positionals;
switch (command) {
case undefined:
case "help":
return usage();
case "validate":
return validate(paths);
default:
console.error(`Unknown command: ${command}`);
return 1;
}
}
function usage(): number {
console.log("Usage: changetool validate <path> [<path> ...]");
return 0;
}
function validate(paths: string[]): number {
let valid = true;
if (paths.length === 0) {
console.error("error: no paths provided (see 'help' command for usage)");
return 1;
}
for (const path of paths) {
if (!isValidChangenoteFile(path)) {
valid = false;
}
}
return valid ? 0 : 1;
}

View File

@@ -0,0 +1,21 @@
{
"name": "changetool",
"version": "1.0.0",
"private": true,
"description": "Validates change-notes and merges them into CHANGELOG.md",
"license": "MIT",
"type": "module",
"scripts": {
"start": "tsx index.ts",
"test": "node --test --experimental-strip-types cli/*.test.ts"
},
"devDependencies": {
"@types/node": "^26.2.0",
"tsx": "^4.23.12",
"typescript": "^7.0.2"
},
"dependencies": {
"lite-matter": "^0.1.2",
"mdast-util-from-markdown": "^2.0.3"
}
}

View File

@@ -0,0 +1,11 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"module": "preserve",
"allowImportingTsExtensions": true,
"rootDir": ".",
"sourceMap": false
},
"include": ["./**/*.ts"],
"exclude": ["node_modules"]
}

View File

@@ -283,6 +283,19 @@ export function isSelfHostedRunner(env: Env = getEnv()) {
return env.getOptional(ActionsEnvVars.RUNNER_ENVIRONMENT) === "self-hosted";
}
/**
* Whether the job is running on a runner that GitHub hosts, and whose toolcache is therefore thrown
* away once the job has finished.
*
* Unlike `looksLikeHostedRunner`, this is based on what the service reports for the job rather than
* on how the runner's filesystem happens to be laid out, so it does not match self-hosted runners
* that are configured to resemble hosted ones, such as those that mount a persistent volume at
* `/opt/hostedtoolcache`.
*/
export function isGitHubHostedRunner(env: Env = getEnv()) {
return env.getOptional(ActionsEnvVars.RUNNER_ENVIRONMENT) === "github-hosted";
}
/** Determines whether the workflow trigger is `dynamic`. */
export function isDynamicWorkflow(env: Env = getEnv()): boolean {
return getWorkflowEventName(env) === "dynamic";

View File

@@ -1,8 +1,5 @@
import * as core from "@actions/core";
import * as githubUtils from "@actions/github/lib/utils";
import { type Octokit } from "@octokit/core";
import { type PaginateInterface } from "@octokit/plugin-paginate-rest";
import { type Api } from "@octokit/plugin-rest-endpoint-methods";
import * as retry from "@octokit/plugin-retry";
import { RequestRequestOptions } from "@octokit/types";
import {
@@ -128,7 +125,7 @@ export function makeProxyRequestOptions(
}
/** The type of GitHub API client we use. */
export type ApiClient = Octokit & Api & { paginate: PaginateInterface };
export type ApiClient = InstanceType<typeof githubUtils.GitHub>;
/** Options for `createApiClientWithDetails`. */
interface CreateApiClientOptions {

View File

@@ -5,7 +5,7 @@ import * as core from "@actions/core";
import { getOptionalInput, isDefaultSetup } from "./actions-util";
import { EnvVar } from "./environment";
import { Logger } from "./logging";
import { isHostedRunner, tryGetFolderBytes } from "./util";
import { looksLikeHostedRunner, tryGetFolderBytes } from "./util";
/**
* Returns the total size of all the specified paths.
@@ -109,7 +109,7 @@ export function getDependencyCachingEnabled(): CachingKind {
if (dependencyCaching !== undefined) return getCachingKind(dependencyCaching);
// On self-hosted runners which may have dependencies installed centrally, disable caching by default
if (!isHostedRunner()) return CachingKind.None;
if (!looksLikeHostedRunner()) return CachingKind.None;
// Disable in advanced workflows by default.
if (!isDefaultSetup()) return CachingKind.None;

View File

@@ -128,7 +128,6 @@ test("CliError constructor with empty stderr", (t) => {
for (const [platform, arch] of [
["weird_plat", "x64"],
["linux", "arm64"],
["win32", "arm64"],
]) {
test.serial(
@@ -157,20 +156,34 @@ for (const [platform, arch] of [
);
}
test("wrapCliConfigurationError - supported platform", (t) => {
const commandError = new CommandInvocationError(
"codeql",
["version"],
1,
"Some error",
for (const [platform, arch] of [
["linux", "x64"],
["linux", "arm64"],
["win32", "x64"],
["darwin", "x64"],
["darwin", "arm64"],
]) {
test.serial(
`wrapCliConfigurationError - ${platform}/${arch} supported`,
(t) => {
sinon.stub(process, "platform").value(platform);
sinon.stub(process, "arch").value(arch);
const commandError = new CommandInvocationError(
"codeql",
["version"],
1,
"Some error",
);
const cliError = new CliError(commandError);
const wrappedError = wrapCliConfigurationError(cliError);
// Should return the original error since the platform is supported, rather
// than replacing it with the unsupported-platform ConfigurationError.
t.is(wrappedError, cliError);
},
);
const cliError = new CliError(commandError);
const wrappedError = wrapCliConfigurationError(cliError);
// Should return the original error since platform is supported
t.is(wrappedError, cliError);
});
}
test("wrapCliConfigurationError - autobuild error", (t) => {
const commandError = new CommandInvocationError(

View File

@@ -8,6 +8,7 @@ import { ConfigurationError } from "./util";
const SUPPORTED_PLATFORMS = [
["linux", "x64"],
["linux", "arm64"],
["win32", "x64"],
["darwin", "x64"],
["darwin", "arm64"],

View File

@@ -3,12 +3,11 @@ import path from "path";
import test from "ava";
import { EnvVar } from "../environment";
import { getRunnerLogger } from "../logging";
import { getTestEnv, setupTests } from "../testing-utils";
import { setupTests } from "../testing-utils";
import * as util from "../util";
import * as outputCache from "./output-cache";
import { getCachedCodeQlVersion } from "./output-cache";
setupTests(test);
@@ -18,18 +17,18 @@ test.serial(
"getCachedCodeQlVersion reuses a version persisted by an earlier step",
async (t) => {
await util.withTmpDir(async (tmpDir: string) => {
const cacheFile = path.join(tmpDir, "codeql-action-command-cache.json");
const cacheFilePath = path.join(tmpDir, "cache.json");
fs.writeFileSync(
cacheFile,
cacheFilePath,
JSON.stringify({
cmd: "/path/to/codeql",
entries: { version: { version: "2.20.0" } },
}),
"utf8",
);
const env = getTestEnv({ [EnvVar.TEMP]: tmpDir });
t.deepEqual(
outputCache.getCachedCodeQlVersion(logger, env, "/path/to/codeql"),
getCachedCodeQlVersion(logger, cacheFilePath, "/path/to/codeql"),
{
version: "2.20.0",
},
@@ -42,18 +41,17 @@ test.serial(
"getCachedCodeQlVersion ignores a persisted version from a different CLI",
async (t) => {
await util.withTmpDir(async (tmpDir: string) => {
const cacheFile = path.join(tmpDir, "version.json");
const cacheFilePath = path.join(tmpDir, "cache.json");
fs.writeFileSync(
cacheFile,
cacheFilePath,
JSON.stringify({
cmd: "/path/to/other-codeql",
version: { version: "2.20.0" },
entries: { version: { version: "2.20.0" } },
}),
"utf8",
);
const env = getTestEnv({ [EnvVar.TEMP]: tmpDir });
t.is(
outputCache.getCachedCodeQlVersion(logger, env, "/path/to/codeql"),
getCachedCodeQlVersion(logger, cacheFilePath, "/path/to/codeql"),
undefined,
);
});
@@ -64,11 +62,10 @@ test.serial(
"getCachedCodeQlVersion ignores a malformed persisted value",
async (t) => {
await util.withTmpDir(async (tmpDir: string) => {
const cacheFile = path.join(tmpDir, "version.json");
fs.writeFileSync(cacheFile, "not valid json", "utf8");
const env = getTestEnv({ [EnvVar.TEMP]: tmpDir });
const cacheFilePath = path.join(tmpDir, "cache.json");
fs.writeFileSync(cacheFilePath, "not valid json", "utf8");
t.is(
outputCache.getCachedCodeQlVersion(logger, env, "/path/to/codeql"),
getCachedCodeQlVersion(logger, cacheFilePath, "/path/to/codeql"),
undefined,
);
});
@@ -79,9 +76,7 @@ test.serial(
"getCachedCodeQlVersion ignores a persisted value with the wrong structure",
async (t) => {
await util.withTmpDir(async (tmpDir: string) => {
const cacheFile = path.join(tmpDir, "version.json");
const env = getTestEnv({ [EnvVar.TEMP]: tmpDir });
const cacheFilePath = path.join(tmpDir, "cache.json");
const testValues = [
{ cmd: "/path/to/codeql" },
{ entries: { version: { version: "2.20.0" } } },
@@ -104,9 +99,9 @@ test.serial(
].map((v) => JSON.stringify(v));
for (const value of testValues) {
fs.writeFileSync(cacheFile, value, "utf8");
fs.writeFileSync(cacheFilePath, value, "utf8");
t.is(
outputCache.getCachedCodeQlVersion(logger, env, "/path/to/codeql"),
getCachedCodeQlVersion(logger, cacheFilePath, "/path/to/codeql"),
undefined,
value,
);
@@ -117,10 +112,10 @@ test.serial(
test.serial("getCachedCodeQlVersion ignores non-existent file", async (t) => {
await util.withTmpDir(async (tmpDir: string) => {
const env = getTestEnv({ [EnvVar.TEMP]: tmpDir });
const cacheFilePath = path.join(tmpDir, "cache.json");
t.notThrows(() => {
t.is(
outputCache.getCachedCodeQlVersion(logger, env, "/path/to/codeql"),
getCachedCodeQlVersion(logger, cacheFilePath, "/path/to/codeql"),
undefined,
);
});

View File

@@ -3,9 +3,10 @@ import path from "path";
import { getTemporaryDirectory } from "../actions-util";
import { Env } from "../environment";
import * as json from "../json";
import { Logger } from "../logging";
import type { VersionInfo } from "./types";
import { VersionInfo, versionInfoBaseSchema } from "./types";
/**
* The keys of the command cache. Each key corresponds to a command whose output we cache.
@@ -13,12 +14,19 @@ import type { VersionInfo } from "./types";
export type CommandCacheKey = string;
/**
* The type of the command cache that is persisted to disk.
* The JSON schema of the command cache that is persisted to disk.
*/
export interface OutputCache {
cmd: string;
entries: Record<CommandCacheKey, unknown>;
}
const outputCacheSchema = {
cmd: json.string,
entries: json.object({}),
} as const satisfies json.Schema;
/**
* The type that describes the command cache that is persisted to disk.
*/
export type OutputCache = json.FromSchema<typeof outputCacheSchema> & {
entries: { version: VersionInfo };
};
/**
* The name of the temporary file that backs the on-disk cache of
@@ -43,18 +51,18 @@ export function resetCachedCodeQlVersion(): void {
* Returns the path to the temporary file that backs the
* on-disk cache of CLI responses between workflow steps.
*/
function getCommandCacheFilePath(env: Env): string {
export function getCommandCacheFilePath(env: Env): string {
return path.join(getTemporaryDirectory(env), COMMAND_CACHE_FILENAME);
}
/**
* Caches the CodeQL CLI version both in-memory and on disk.
* @param env The environment variables to use.
* @param cacheFilePath The path to the cache file.
* @param cmd The path to the CodeQL CLI.
* @param version The version information to cache.
*/
export function cacheCodeQlVersion(
env: Env,
cacheFilePath: string,
cmd: string,
version: VersionInfo,
): void {
@@ -70,22 +78,18 @@ export function cacheCodeQlVersion(
// processes, can reuse it rather than invoking `codeql version` again. We
// record the CLI path so that a different step using a different CodeQL bundle
// doesn't pick up a stale version.
fs.writeFileSync(
getCommandCacheFilePath(env),
JSON.stringify(outputCache),
"utf8",
);
fs.writeFileSync(cacheFilePath, JSON.stringify(outputCache), "utf8");
}
/**
* Returns the cached CodeQL CLI version, if any.
* @param logger The logger to use for logging messages.
* @param env The environment variables to use.
* @param cacheFilePath The path to the cache file.
* @param cmd The path to the CodeQL CLI.
*/
export function getCachedCodeQlVersion(
logger: Logger,
env: Env,
cacheFilePath: string,
cmd?: string,
): undefined | VersionInfo {
if (cachedCodeQlVersion !== undefined) {
@@ -96,11 +100,9 @@ export function getCachedCodeQlVersion(
// invokes `codeql version` instead.
let serialized: string;
try {
serialized = fs.readFileSync(getCommandCacheFilePath(env), "utf8");
serialized = fs.readFileSync(cacheFilePath, "utf8");
} catch (e) {
logger.debug(
`Cannot read CLI-cache file ${getCommandCacheFilePath(env)}: ${e}`,
);
logger.debug(`Cannot read CLI-cache file ${cacheFilePath}: ${e}`);
return undefined;
}
let persisted: unknown;
@@ -127,17 +129,7 @@ export function getCachedCodeQlVersion(
* @param x The value to test
*/
function isVersionInfo(x: unknown): x is VersionInfo {
const candidate = x as Partial<VersionInfo> | null;
return (
typeof candidate === "object" &&
candidate !== null &&
typeof candidate.version === "string" &&
(candidate.features === undefined ||
(typeof candidate.features === "object" &&
candidate.features !== null)) &&
(candidate.overlayVersion === undefined ||
typeof candidate.overlayVersion === "number")
);
return json.isObject(x) && json.validateSchema(versionInfoBaseSchema, x);
}
/**
@@ -145,12 +137,10 @@ function isVersionInfo(x: unknown): x is VersionInfo {
* @param x The value to test
*/
function isOutputCache(x: unknown): x is OutputCache {
const candidate = x as Partial<OutputCache> | null;
return (
typeof candidate === "object" &&
candidate !== null &&
typeof candidate.cmd === "string" &&
candidate.entries !== undefined &&
isVersionInfo(candidate.entries.version)
json.isObject(x) &&
json.validateSchema(outputCacheSchema, x) &&
json.isObject<{ version: unknown }>(x.entries) &&
isVersionInfo(x.entries.version)
);
}

View File

@@ -1,6 +1,11 @@
export interface VersionInfo {
version: string;
features?: { [name: string]: boolean };
import * as json from "../json";
/**
* The JSON schema of the expected output of the `codeql version` command.
*/
export const versionInfoBaseSchema = {
version: json.string,
features: json.optional(json.object({})),
/**
* The overlay version helps deal with backward incompatible changes for
* overlay analysis. When a precompiled query pack reports the same overlay
@@ -9,5 +14,17 @@ export interface VersionInfo {
* or if either the pack or the CLI does not report an overlay version,
* we need to revert to non-overlay analysis.
*/
overlayVersion?: number;
}
overlayVersion: json.optional(json.number),
} as const satisfies json.Schema;
/**
* The base type that describes the expected output of the `codeql version` command.
*/
export type VersionInfoBase = json.FromSchema<typeof versionInfoBaseSchema>;
/**
* The full type that describes the expected output of the `codeql version` command.
*/
export type VersionInfo = Omit<VersionInfoBase, "features"> & {
features?: { [name: string]: boolean };
};

View File

@@ -510,7 +510,12 @@ async function getCodeQLForCmd(
return cmd;
},
async getVersion() {
let result = outputCache.getCachedCodeQlVersion(logger, getEnv(), cmd);
const cacheFilePath = outputCache.getCommandCacheFilePath(getEnv());
let result = outputCache.getCachedCodeQlVersion(
logger,
cacheFilePath,
cmd,
);
if (result === undefined) {
result = await runCliJson<VersionInfo>(
cmd,
@@ -519,7 +524,7 @@ async function getCodeQLForCmd(
noStreamStdout: true,
},
);
outputCache.cacheCodeQlVersion(getEnv(), cmd, result);
outputCache.cacheCodeQlVersion(cacheFilePath, cmd, result);
}
return result;
},

View File

@@ -90,9 +90,8 @@ import {
Result,
Success,
Failure,
isHostedRunner,
looksLikeHostedRunner,
} from "./util";
export { type Config } from "./config/action-config";
/**
@@ -938,7 +937,7 @@ export async function isTrapCachingEnabled(
if (trapCaching !== undefined) return trapCaching === "true";
// On self-hosted runners which may have slow network access, disable TRAP caching by default.
if (!isHostedRunner()) return false;
if (!looksLikeHostedRunner()) return false;
// If overlay analysis is enabled, then disable TRAP caching since overlay analysis supersedes it.
// This change is gated behind a feature flag.

View File

@@ -1,6 +1,6 @@
{
"bundleVersion": "codeql-bundle-v2.26.4",
"cliVersion": "2.26.4",
"priorBundleVersion": "codeql-bundle-v2.26.3",
"priorCliVersion": "2.26.3"
"bundleVersion": "codeql-bundle-v2.27.0",
"cliVersion": "2.27.0",
"priorBundleVersion": "codeql-bundle-v2.26.4",
"priorCliVersion": "2.26.4"
}

View File

@@ -63,6 +63,12 @@ export enum EnvVar {
/** Whether the CodeQL Action has already warned the user about low disk space. */
HAS_WARNED_ABOUT_DISK_SPACE = "CODEQL_ACTION_HAS_WARNED_ABOUT_DISK_SPACE",
/**
* Whether a step in this job has already set up CodeQL. Steps that run afterwards may be holding
* a path into the toolcache, so we must not delete anything from it.
*/
HAS_SET_UP_CODEQL = "CODEQL_ACTION_HAS_SET_UP_CODEQL",
/** Whether the `setup-codeql` action has been run. */
SETUP_CODEQL_ACTION_HAS_RUN = "CODEQL_ACTION_SETUP_CODEQL_HAS_RUN",

View File

@@ -74,6 +74,11 @@ export enum Feature {
AllowMergeConfigFiles = "allow_merge_config_files",
/** Controls whether we allow multiple values for the `analysis-kinds` input. */
AllowMultipleAnalysisKinds = "allow_multiple_analysis_kinds",
/**
* Controls whether we delete CodeQL bundles that we are not going to use from the toolcache
* before downloading a different bundle, in order to reclaim disk space.
*/
CleanupToolcacheBundles = "cleanup_toolcache_bundles",
CleanupTrapCaches = "cleanup_trap_caches",
/** Whether to allow the `config-file` input to be specified via a repository property. */
ConfigFileRepositoryProperty = "config_file_repository_property",
@@ -211,6 +216,11 @@ export const featureConfig = {
envVar: "CODEQL_ACTION_ALLOW_MULTIPLE_ANALYSIS_KINDS",
minimumVersion: undefined,
},
[Feature.CleanupToolcacheBundles]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES",
minimumVersion: undefined,
},
[Feature.CleanupTrapCaches]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CLEANUP_TRAP_CACHES",

View File

@@ -15,10 +15,12 @@ import { getRunnerLogger } from "./logging";
import { OverlayDatabaseMode } from "./overlay/overlay-database-mode";
import * as overlayStatus from "./overlay/status";
import { parseRepositoryNwo } from "./repository";
import { JobStatus } from "./status-report";
import {
createFeatures,
createTestConfig,
DEFAULT_ACTIONS_VARS,
getTestEnv,
makeMacro,
makeVersionInfo,
RecordingLogger,
@@ -58,6 +60,8 @@ test.serial("init-post action with debug mode off", async (t) => {
createTestConfig({ debugMode: false }),
parseRepositoryNwo("github/codeql-action"),
createFeatures([]),
"success",
getTestEnv(),
getRunnerLogger(true),
);
@@ -80,6 +84,8 @@ test.serial("init-post action with debug mode on", async (t) => {
createTestConfig({ debugMode: true }),
parseRepositoryNwo("github/codeql-action"),
createFeatures([]),
"success",
getTestEnv(),
getRunnerLogger(true),
);
@@ -375,6 +381,8 @@ test.serial(
}),
parseRepositoryNwo("github/codeql-action"),
createFeatures([Feature.OverlayAnalysisStatusSave]),
"success",
getTestEnv(),
getRunnerLogger(true),
);
@@ -443,6 +451,8 @@ test.serial(
}),
parseRepositoryNwo("github/codeql-action"),
createFeatures([]),
"success",
getTestEnv(),
getRunnerLogger(true),
);
@@ -457,8 +467,13 @@ test.serial(
test.serial("does not save overlay status when build successful", async (t) => {
return await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
// Mark analyze as having completed successfully.
// Mark analyze as having completed successfully. `tryUploadSarifIfRunFailed` reads this from
// the process environment, while `recordOverlayStatus` reads it from the environment it is
// given.
process.env[EnvVar.ANALYZE_DID_COMPLETE_SUCCESSFULLY] = "true";
const env = getTestEnv({
[EnvVar.ANALYZE_DID_COMPLETE_SUCCESSFULLY]: "true",
});
sinon.stub(util, "checkDiskUsage").resolves({
numAvailableBytes: 100 * NUM_BYTES_PER_GIB,
@@ -480,6 +495,8 @@ test.serial("does not save overlay status when build successful", async (t) => {
}),
parseRepositoryNwo("github/codeql-action"),
createFeatures([Feature.OverlayAnalysisStatusSave]),
"success",
env,
getRunnerLogger(true),
);
@@ -517,6 +534,8 @@ test.serial(
}),
parseRepositoryNwo("github/codeql-action"),
createFeatures([]),
"success",
getTestEnv(),
getRunnerLogger(true),
);
@@ -528,6 +547,137 @@ test.serial(
},
);
/**
* Runs `uploadFailureInfo` for an overlay-base job that did not complete successfully, with the
* given job status from the Actions runtime environment.
*/
async function runOverlayPostStep({
jobStatus,
codeQlReportedError = false,
}: {
jobStatus: string | undefined;
codeQlReportedError?: boolean;
}) {
return await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
delete process.env[EnvVar.ANALYZE_DID_COMPLETE_SUCCESSFULLY];
const env = getTestEnv(
codeQlReportedError
? { [EnvVar.JOB_STATUS]: JobStatus.FailureStatus }
: {},
);
sinon.stub(util, "checkDiskUsage").resolves({
numAvailableBytes: 100 * NUM_BYTES_PER_GIB,
numTotalBytes: 200 * NUM_BYTES_PER_GIB,
});
const saveOverlayStatusStub = sinon
.stub(overlayStatus, "saveOverlayStatus")
.resolves(true);
await initActionPostHelper.uploadFailureInfo(
sinon.spy(),
sinon.spy(),
codeql.createStubCodeQL({}),
createTestConfig({
debugMode: false,
languages: ["javascript"],
overlayDatabaseMode: OverlayDatabaseMode.OverlayBase,
}),
parseRepositoryNwo("github/codeql-action"),
createFeatures([Feature.OverlayAnalysisStatusSave]),
jobStatus,
env,
getRunnerLogger(true),
);
return { saveOverlayStatusStub };
});
}
test.serial(
"does not save overlay status when the job was cancelled",
async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: "cancelled",
});
t.true(
saveOverlayStatusStub.notCalled,
"a cancellation tells us nothing about whether the analysis would have succeeded",
);
},
);
test.serial(
"does not save overlay status when the job status is not recognised",
async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: "some-new-status",
});
t.true(
saveOverlayStatusStub.notCalled,
"a status we do not recognise tells us nothing about whether the analysis would have succeeded",
);
},
);
test.serial(
"does not save overlay status when the job status is unavailable",
async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: undefined,
});
t.true(
saveOverlayStatusStub.notCalled,
"without a job status we cannot tell whether the analysis would have succeeded",
);
},
);
test.serial(
"saves overlay status when the job failed rather than being cancelled",
async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: "failure",
});
t.true(
saveOverlayStatusStub.calledOnce,
"a failed job indicates that the analysis itself failed",
);
},
);
test.serial("saves overlay status when the job succeeded", async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: "success",
});
t.true(
saveOverlayStatusStub.calledOnce,
"the analysis did not complete successfully even though the job as a whole succeeded",
);
});
test.serial(
"saves overlay status when a CodeQL Action reported an error before the run was cancelled",
async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: "cancelled",
codeQlReportedError: true,
});
t.true(
saveOverlayStatusStub.calledOnce,
"the analysis genuinely failed, even though the run was later cancelled",
);
},
);
function createTestWorkflow(
steps: workflow.WorkflowJobStep[],
): workflow.Workflow {

View File

@@ -18,7 +18,7 @@ import {
sanitizeArtifactName,
} from "./debug-artifacts";
import * as dependencyCaching from "./dependency-caching";
import { EnvVar } from "./environment";
import { EnvVar, ReadOnlyEnv } from "./environment";
import { Feature, FeatureEnablement } from "./feature-flags";
import { Logger } from "./logging";
import { OverlayDatabaseMode } from "./overlay/overlay-database-mode";
@@ -316,6 +316,8 @@ export async function tryUploadSarifIfRunFailed(
* @param config The CodeQL Action configuration.
* @param repositoryNwo The name and owner of the repository.
* @param features Information about enabled features.
* @param jobStatus The status of the job, as reported by the Actions runtime environment.
* @param env The environment to read variables from.
* @param logger The logger to use.
* @returns The results of uploading the SARIF file for the failure.
*/
@@ -331,9 +333,11 @@ export async function uploadFailureInfo(
config: Config,
repositoryNwo: RepositoryNwo,
features: FeatureEnablement,
jobStatus: string | undefined,
env: ReadOnlyEnv,
logger: Logger,
): Promise<UploadFailedSarifResult> {
await recordOverlayStatus(codeql, config, features, logger);
await recordOverlayStatus(codeql, config, features, jobStatus, env, logger);
const uploadFailedSarifResult = await tryUploadSarifIfRunFailed(
config,
@@ -412,6 +416,37 @@ export async function uploadFailureInfo(
return uploadFailedSarifResult;
}
/**
* Whether one of the CodeQL Actions reported an error for this job, which means the analysis
* genuinely failed.
*
* Note that the converse does not hold: an Action that is terminated abruptly, or that fails before
* it can gather telemetry, does not get to report anything.
*/
function didCodeQlReportError(env: ReadOnlyEnv): boolean {
const jobStatus = env.getOptional(EnvVar.JOB_STATUS);
return (
jobStatus === JobStatus.FailureStatus ||
jobStatus === JobStatus.ConfigErrorStatus
);
}
/**
* Whether the job status tells us anything about whether the analysis itself would have succeeded.
*
* We check for the statuses we know to be meaningful rather than excluding the ones that are not,
* so that a status we do not recognise is treated as inconclusive.
*/
function isConclusiveJobStatus(jobStatus: string | undefined): boolean {
switch (jobStatus?.trim().toLowerCase()) {
case "failure":
case "success":
return true;
default:
return false;
}
}
/**
* If overlay base database creation was attempted but the analysis did not complete
* successfully, save the failure status to the Actions cache so that subsequent runs
@@ -421,16 +456,30 @@ async function recordOverlayStatus(
codeql: CodeQL,
config: Config,
features: FeatureEnablement,
jobStatus: string | undefined,
env: ReadOnlyEnv,
logger: Logger,
) {
if (
config.overlayDatabaseMode !== OverlayDatabaseMode.OverlayBase ||
process.env[EnvVar.ANALYZE_DID_COMPLETE_SUCCESSFULLY] === "true" ||
env.getOptional(EnvVar.ANALYZE_DID_COMPLETE_SUCCESSFULLY) === "true" ||
!(await features.getValue(Feature.OverlayAnalysisStatusSave))
) {
return;
}
// Only record a failure when the job outcome tells us something about the analysis. A cancelled
// job, or a status we do not recognise, says nothing about whether the analysis would have
// succeeded, so recording a failure would disable overlay analysis needlessly. We still record
// one if a CodeQL Action reported an error before the job ended.
if (!isConclusiveJobStatus(jobStatus) && !didCodeQlReportError(env)) {
logger.info(
"Not recording an improved incremental analysis failure for this job because the job " +
`status (${jobStatus ?? "unset"}) does not tell us whether the analysis itself failed.`,
);
return;
}
const checkRunIdInput = actionsUtil.getOptionalInput("check-run-id");
const checkRunId =
checkRunIdInput !== undefined ? parseInt(checkRunIdInput, 10) : undefined;

View File

@@ -8,6 +8,7 @@ import * as core from "@actions/core";
import {
restoreInputs,
getOptionalInput,
getTemporaryDirectory,
printDebugLogs,
} from "./actions-util";
@@ -20,7 +21,7 @@ import {
DependencyCachingUsageReport,
getDependencyCacheUsage,
} from "./dependency-caching";
import { EnvVar } from "./environment";
import { EnvVar, getEnv } from "./environment";
import { initFeatures } from "./feature-flags";
import * as gitUtils from "./git-utils";
import * as initActionPostHelper from "./init-action-post-helper";
@@ -55,6 +56,11 @@ async function run(startedAt: Date) {
| undefined;
let dependencyCachingUsage: DependencyCachingUsageReport | undefined;
try {
// Read the job status before restoring inputs, since it is provided by the Actions runtime
// environment for this step and would otherwise be overwritten by the value that the `init`
// Action saw, which is always a success.
const jobStatus = getOptionalInput("job-status");
// Restore inputs from `init` Action.
restoreInputs();
@@ -84,6 +90,8 @@ async function run(startedAt: Date) {
config,
repositoryNwo,
features,
jobStatus,
getEnv(),
logger,
);

View File

@@ -174,6 +174,14 @@ async function sendCompletedStatusReport(
initToolsDownloadFields.tools_download_duration_ms =
toolsDownloadStatusReport.downloadDurationMs;
}
if (toolsDownloadStatusReport?.extractionDurationMs !== undefined) {
initToolsDownloadFields.tools_extraction_duration_ms =
toolsDownloadStatusReport.extractionDurationMs;
}
if (toolsDownloadStatusReport?.totalDurationMs !== undefined) {
initToolsDownloadFields.tools_total_duration_ms =
toolsDownloadStatusReport.totalDurationMs;
}
if (toolsFeatureFlagsValid !== undefined) {
initToolsDownloadFields.tools_feature_flags_valid = toolsFeatureFlagsValid;
}

View File

@@ -85,6 +85,14 @@ async function sendCompletedStatusReport(
initToolsDownloadFields.tools_download_duration_ms =
toolsDownloadStatusReport.downloadDurationMs;
}
if (toolsDownloadStatusReport?.extractionDurationMs !== undefined) {
initToolsDownloadFields.tools_extraction_duration_ms =
toolsDownloadStatusReport.extractionDurationMs;
}
if (toolsDownloadStatusReport?.totalDurationMs !== undefined) {
initToolsDownloadFields.tools_total_duration_ms =
toolsDownloadStatusReport.totalDurationMs;
}
if (toolsFeatureFlagsValid !== undefined) {
initToolsDownloadFields.tools_feature_flags_valid = toolsFeatureFlagsValid;
}

View File

@@ -1,3 +1,5 @@
import * as fs from "fs";
import * as os from "os";
import * as path from "path";
import * as github from "@actions/github";
@@ -7,7 +9,8 @@ import * as sinon from "sinon";
import * as actionsUtil from "./actions-util";
import * as api from "./api-client";
import { EnvVar } from "./environment";
import * as diagnostics from "./diagnostics";
import { ActionsEnvVars, EnvVar, ReadOnlyEnv } from "./environment";
import { Feature } from "./feature-flags";
import { getRunnerLogger } from "./logging";
import { getCacheRestoreKeyPrefix } from "./overlay/caching";
@@ -27,6 +30,7 @@ import {
setupActionsVars,
setupTests,
} from "./testing-utils";
import * as toolsDownload from "./tools-download";
import {
getErrorMessage,
GitHubVariant,
@@ -120,24 +124,42 @@ test.serial(
const LINKED_BUNDLE_TEST_CASES = [
{
platform: "linux",
arch: "x64",
tarSupportsZstd: true,
expectedBundleName: "codeql-bundle-linux64.tar.zst",
expectedCompressionMethod: "zstd",
},
{
platform: "linux",
arch: "arm64",
tarSupportsZstd: true,
expectedBundleName: "codeql-bundle-linux-arm64.tar.zst",
expectedCompressionMethod: "zstd",
},
{
platform: "darwin",
arch: "arm64",
tarSupportsZstd: true,
expectedBundleName: "codeql-bundle-osx64.tar.zst",
expectedCompressionMethod: "zstd",
},
{
platform: "darwin",
arch: "x64",
tarSupportsZstd: true,
expectedBundleName: "codeql-bundle-osx64.tar.zst",
expectedCompressionMethod: "zstd",
},
{
platform: "win32",
arch: "x64",
tarSupportsZstd: true,
expectedBundleName: "codeql-bundle-win64.tar.gz",
expectedCompressionMethod: "gzip",
},
{
platform: "linux",
arch: "x64",
tarSupportsZstd: false,
expectedBundleName: "codeql-bundle-linux64.tar.gz",
expectedCompressionMethod: "gzip",
@@ -146,15 +168,17 @@ const LINKED_BUNDLE_TEST_CASES = [
for (const {
platform,
arch,
tarSupportsZstd,
expectedBundleName,
expectedCompressionMethod,
} of LINKED_BUNDLE_TEST_CASES) {
test.serial(
`getCodeQLSource selects ${expectedBundleName} for linked tools`,
`getCodeQLSource selects ${expectedBundleName} for linked tools on ${platform}/${arch}`,
async (t) => {
const features = createFeatures([]);
sinon.stub(process, "platform").value(platform);
sinon.stub(process, "arch").value(arch);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
@@ -234,6 +258,7 @@ test.serial(
codeqlFolder: "codeql",
statusReport: {
downloadDurationMs: 200,
totalDurationMs: 300,
},
toolsVersion: LINKED_CLI_VERSION.cliVersion,
});
@@ -286,6 +311,7 @@ test.serial(
codeqlFolder: "codeql",
statusReport: {
downloadDurationMs: 200,
totalDurationMs: 300,
},
toolsVersion: expectedVersion,
});
@@ -917,3 +943,616 @@ test.serial(
]);
},
);
/** The CLI version that the toolcache cleanup tests download. */
const CLEANUP_CLI_VERSION = "2.21.0";
/** The bundle version that the toolcache cleanup tests download. */
const CLEANUP_BUNDLE_VERSION = "20240101";
/** A version of the CodeQL tools that is already in the toolcache but that we are not going to use. */
const CLEANUP_STALE_VERSION = "2.20.0";
/** Creates a directory in the toolcache that looks like a tool that `tool-cache` has cached. */
function createToolcacheEntry(
toolcacheRoot: string,
tool: string,
version: string,
): string {
const versionDirectory = path.join(toolcacheRoot, tool, version);
const archDirectory = path.join(versionDirectory, os.arch());
fs.mkdirSync(archDirectory, { recursive: true });
fs.writeFileSync(path.join(archDirectory, "contents"), "x".repeat(1024));
fs.writeFileSync(`${archDirectory}.complete`, "");
return versionDirectory;
}
/**
* Stubs out the download and the diagnostic sink, then downloads the CodeQL tools.
*
* @returns the extraction directory and the toolcache cleanup diagnostic, if emitted.
*/
async function runDownloadCodeQL(
toolcacheRoot: string,
features: Feature[],
bundleVersion: string | undefined,
): Promise<{
codeqlFolder: string;
cleanupDiagnostic: toolsDownload.ToolcacheCleanupResult | undefined;
}> {
sinon
.stub(toolsDownload, "downloadAndExtract")
.callsFake(async (_url, _compressionMethod, dest) => {
// The real implementation creates the destination directory, which matters here because the
// cleanup deletes it first and `writeToolcacheMarkerFile` writes into its parent afterwards.
fs.mkdirSync(dest, { recursive: true });
return { totalDurationMs: 1 };
});
const addDiagnostic = sinon.stub(diagnostics, "addNoLanguageDiagnostic");
const { codeqlFolder } = await setupCodeql.downloadCodeQL(
"https://example.com/codeql-bundle.tar.gz",
"gzip",
bundleVersion,
CLEANUP_CLI_VERSION,
SAMPLE_DOTCOM_API_DETAILS,
undefined, // tarVersion
toolcacheRoot, // tempDir
createFeatures(features),
getRunnerLogger(true),
);
const diagnostic = addDiagnostic
.getCalls()
.map((call) => call.args[1])
.find((d) => d.source?.id === "codeql-action/toolcache-bundle-cleanup");
return {
codeqlFolder,
cleanupDiagnostic: diagnostic?.attributes as
| toolsDownload.ToolcacheCleanupResult
| undefined,
};
}
/**
* Sets up a toolcache containing the version of the CodeQL tools that we are about to download, a
* different version of the CodeQL tools, and an unrelated tool, then downloads the CodeQL tools.
*/
async function testToolcacheCleanup(
t: ExecutionContext<unknown>,
{
features,
runnerEnvironment,
setUp,
}: {
features: Feature[];
runnerEnvironment: string | undefined;
setUp?: () => void;
},
check: (context: {
cleanupDiagnostic: toolsDownload.ToolcacheCleanupResult | undefined;
destinationDirectory: string;
staleDirectory: string;
}) => void,
) {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
if (runnerEnvironment === undefined) {
delete process.env[ActionsEnvVars.RUNNER_ENVIRONMENT];
} else {
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = runnerEnvironment;
}
setUp?.();
// The extraction of the bundle would normally create this directory.
const destinationDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_CLI_VERSION,
);
const staleDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const otherToolDirectory = createToolcacheEntry(tmpDir, "Node", "20.0.0");
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
features,
CLEANUP_BUNDLE_VERSION,
);
t.true(
fs.existsSync(otherToolDirectory),
"Should never delete other tools from the toolcache.",
);
check({ cleanupDiagnostic, destinationDirectory, staleDirectory });
});
}
test.serial(
"downloadCodeQL does not clean up the toolcache when the feature flag is disabled",
async (t) => {
await testToolcacheCleanup(
t,
{ features: [], runnerEnvironment: "github-hosted" },
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.true(fs.existsSync(staleDirectory));
t.true(fs.existsSync(destinationDirectory));
t.is(cleanupDiagnostic, undefined);
},
);
},
);
test.serial(
"downloadCodeQL does not clean up the toolcache when the runner is not GitHub-hosted",
async (t) => {
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "self-hosted",
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.true(fs.existsSync(staleDirectory));
t.true(fs.existsSync(destinationDirectory));
t.is(cleanupDiagnostic, undefined);
},
);
},
);
test.serial(
"downloadCodeQL does not clean up the toolcache when the runner environment is unknown",
async (t) => {
// A runner that doesn't report its environment must be treated as not GitHub-hosted, since its
// toolcache may well outlive the job.
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: undefined,
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.true(fs.existsSync(staleDirectory));
t.true(fs.existsSync(destinationDirectory));
t.is(cleanupDiagnostic, undefined);
},
);
},
);
test.serial(
"downloadCodeQL deletes other CodeQL bundles from the toolcache when enabled on a GitHub-hosted runner",
async (t) => {
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "github-hosted",
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.false(
fs.existsSync(staleDirectory),
"Should delete the version directory, including the `tool-cache` marker file it contains.",
);
t.false(
fs.existsSync(path.join(destinationDirectory, os.arch(), "contents")),
"Should also delete a partial entry for the version we are about to download, rather " +
"than extracting over it.",
);
t.deepEqual(cleanupDiagnostic, {
deletedVersions: [CLEANUP_STALE_VERSION, CLEANUP_CLI_VERSION].sort(),
failed: false,
});
},
);
},
);
test.serial(
"downloadCodeQL reports no deleted versions when the toolcache has no CodeQL bundles",
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
// A toolcache with other tools in it, but no CodeQL.
const otherToolDirectory = createToolcacheEntry(tmpDir, "Node", "20.0.0");
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
t.true(fs.existsSync(otherToolDirectory));
t.deepEqual(cleanupDiagnostic, { deletedVersions: [], failed: false });
});
},
);
test.serial(
"downloadCodeQL continues when deleting a CodeQL bundle from the toolcache fails",
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
createToolcacheEntry(tmpDir, "CodeQL", CLEANUP_CLI_VERSION);
const staleDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const rmStub = sinon
.stub(fs.promises, "rm")
.rejects(new Error("EACCES: permission denied"));
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
// Restore before `withTmpDir` cleans up after itself.
rmStub.restore();
t.true(fs.existsSync(staleDirectory));
t.deepEqual(
cleanupDiagnostic,
{ deletedVersions: [], failed: true },
"Should not report versions that we failed to delete.",
);
});
},
);
test.serial(
"deleteToolcacheBundles reports a failure when the toolcache location is unknown",
async (t) => {
const messages: LoggedMessage[] = [];
const result = await toolsDownload.deleteToolcacheBundles({
env: new ReadOnlyEnv({}),
logger: getRecordingLogger(messages),
});
t.deepEqual(
result,
{ deletedVersions: [], failed: true },
"Should report a failure rather than throwing, so the download can continue.",
);
checkExpectedLogMessages(t, messages, [
"Unable to determine toolcache directory: RUNNER_TOOL_CACHE environment variable must be set",
]);
},
);
test.serial(
"deleteToolcacheBundles uses the supplied environment rather than process.env",
async (t) => {
await withTmpDir(async (tmpDir) => {
const ambientRoot = path.join(tmpDir, "ambient");
const injectedRoot = path.join(tmpDir, "injected");
setupActionsVars(tmpDir, ambientRoot);
const ambientVersion = createToolcacheEntry(
ambientRoot,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const injectedVersion = createToolcacheEntry(
injectedRoot,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const fileEntry = path.join(injectedRoot, "CodeQL", "not-a-directory");
fs.writeFileSync(fileEntry, "keep");
const result = await toolsDownload.deleteToolcacheBundles({
env: new ReadOnlyEnv({
[ActionsEnvVars.RUNNER_TOOL_CACHE]: injectedRoot,
}),
logger: getRunnerLogger(true),
});
t.deepEqual(result, {
deletedVersions: [CLEANUP_STALE_VERSION],
failed: false,
});
t.false(fs.existsSync(injectedVersion));
t.true(fs.existsSync(ambientVersion));
t.is(fs.readFileSync(fileEntry, "utf8"), "keep");
});
},
);
test.serial(
"deleteToolcacheBundles reports a failure when the toolcache directory cannot be read",
async (t) => {
await withTmpDir(async (tmpDir) => {
const versionDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const messages: LoggedMessage[] = [];
const readdir = sinon
.stub(fs.promises, "readdir")
.rejects(new Error("permission denied"));
try {
const result = await toolsDownload.deleteToolcacheBundles({
env: new ReadOnlyEnv({
[ActionsEnvVars.RUNNER_TOOL_CACHE]: tmpDir,
}),
logger: getRecordingLogger(messages),
});
t.deepEqual(result, { deletedVersions: [], failed: true });
t.true(fs.existsSync(versionDirectory));
checkExpectedLogMessages(t, messages, [
`Failed to clean up the CodeQL toolcache at '${path.join(tmpDir, "CodeQL")}': permission denied`,
]);
} finally {
readdir.restore();
}
});
},
);
test.serial(
"downloadCodeQL does not follow a symlinked CodeQL toolcache directory",
async (t) => {
await withTmpDir(async (tmpDir) => {
const toolcacheRoot = path.join(tmpDir, "toolcache");
setupActionsVars(tmpDir, toolcacheRoot);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
// Somewhere the toolcache cleanup must never reach.
const outsideDirectory = path.join(tmpDir, "outside");
createToolcacheEntry(outsideDirectory, "CodeQL", CLEANUP_STALE_VERSION);
createToolcacheEntry(outsideDirectory, "CodeQL", CLEANUP_CLI_VERSION);
fs.mkdirSync(toolcacheRoot, { recursive: true });
fs.symlinkSync(
path.join(outsideDirectory, "CodeQL"),
path.join(toolcacheRoot, "CodeQL"),
);
const { cleanupDiagnostic } = await runDownloadCodeQL(
toolcacheRoot,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
t.true(
fs.existsSync(
path.join(outsideDirectory, "CodeQL", CLEANUP_STALE_VERSION),
),
"Should not delete anything through a symlinked CodeQL directory.",
);
t.deepEqual(cleanupDiagnostic, { deletedVersions: [], failed: true });
});
},
);
test.serial(
"downloadCodeQL does not clean up the toolcache once a step has already set up CodeQL",
async (t) => {
// `.github/workflows/codeql.yml` sets up CodeQL twice and then runs both returned paths. If the
// second setup downloads, it must not delete the bundle the first one handed out.
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "github-hosted",
setUp: () => {
process.env[EnvVar.HAS_SET_UP_CODEQL] = "true";
},
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.true(fs.existsSync(staleDirectory));
t.true(fs.existsSync(destinationDirectory));
t.is(cleanupDiagnostic, undefined);
},
);
},
);
test.serial(
"setupCodeQLBundle records that this job has set up CodeQL",
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
delete process.env[EnvVar.HAS_SET_UP_CODEQL];
sinon.stub(setupCodeql, "downloadCodeQL").resolves({
codeqlFolder: "codeql",
statusReport: { totalDurationMs: 1 },
toolsVersion: LINKED_CLI_VERSION.cliVersion,
});
await setupCodeql.setupCodeQLBundle(
"linked",
SAMPLE_DOTCOM_API_DETAILS,
tmpDir,
GitHubVariant.DOTCOM,
SAMPLE_DEFAULT_CLI_VERSION,
undefined, // rawLanguages
false, // useOverlayAwareDefaultCliVersion
createFeatures([]),
getRunnerLogger(true),
);
t.is(
process.env[EnvVar.HAS_SET_UP_CODEQL],
"true",
"A later step must be able to tell that the toolcache is in use.",
);
});
},
);
test.serial(
"downloadCodeQL cleans up the toolcache even when the download will not be cached",
async (t) => {
// A `tools` URL we can't derive a bundle version from is extracted to a temporary directory
// rather than the toolcache, but the toolcache is on the same filesystem, so emptying it still
// frees up space for the analysis.
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
const staleDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const { codeqlFolder, cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
undefined, // bundleVersion
);
t.is(path.dirname(codeqlFolder), tmpDir);
t.not(codeqlFolder, path.join(tmpDir, "CodeQL"));
t.true(fs.existsSync(codeqlFolder));
t.false(fs.existsSync(`${codeqlFolder}.complete`));
t.false(fs.existsSync(staleDirectory));
t.deepEqual(cleanupDiagnostic, {
deletedVersions: [CLEANUP_STALE_VERSION],
failed: false,
});
});
},
);
test.serial(
"downloadCodeQL reports a failure when the toolcache cannot be inspected",
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
createToolcacheEntry(tmpDir, "CodeQL", CLEANUP_STALE_VERSION);
const lstatStub = sinon.stub(fs.promises, "lstat").rejects(
Object.assign(new Error("permission denied"), {
code: "EACCES",
}),
);
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
lstatStub.restore();
t.deepEqual(
cleanupDiagnostic,
{ deletedVersions: [], failed: true },
"An error other than the toolcache being absent must not be reported as success.",
);
});
},
);
test.serial(
"downloadCodeQL does not clean up a toolcache on a different filesystem to the workspace",
async (t) => {
// Some runner images keep the toolcache on a different volume to the workspace, in which case
// deleting the tools frees up disk space that the analysis cannot use.
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
const staleDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_STALE_VERSION,
);
sinon
.stub(toolsDownload, "isToolcacheOnWorkspaceFilesystem")
.returns(false);
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
t.true(fs.existsSync(staleDirectory));
t.is(cleanupDiagnostic, undefined);
});
},
);
test.serial(
"isToolcacheOnWorkspaceFilesystem assumes a different filesystem when it cannot tell",
async (t) => {
await withTmpDir(async (tmpDir) => {
const logger = getRunnerLogger(true);
setupActionsVars(tmpDir, tmpDir);
t.true(toolsDownload.isToolcacheOnWorkspaceFilesystem(logger));
// If we can't tell, we assume the toolcache is not somewhere we can reclaim space from.
process.env[ActionsEnvVars.RUNNER_TOOL_CACHE] = path.join(
tmpDir,
"does-not-exist",
);
t.false(toolsDownload.isToolcacheOnWorkspaceFilesystem(logger));
});
},
);
test.serial(
"downloadCodeQL does not delete through a symlinked version directory",
async (t) => {
await withTmpDir(async (tmpDir) => {
const toolcacheRoot = path.join(tmpDir, "toolcache");
setupActionsVars(tmpDir, toolcacheRoot);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
createToolcacheEntry(toolcacheRoot, "CodeQL", CLEANUP_STALE_VERSION);
// Somewhere outside the toolcache that a version directory points at.
const outsideDirectory = path.join(tmpDir, "outside");
fs.mkdirSync(outsideDirectory, { recursive: true });
fs.writeFileSync(path.join(outsideDirectory, "contents"), "x");
fs.symlinkSync(
outsideDirectory,
path.join(toolcacheRoot, "CodeQL", "9.9.9"),
);
const { cleanupDiagnostic } = await runDownloadCodeQL(
toolcacheRoot,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
t.true(
fs.existsSync(path.join(outsideDirectory, "contents")),
"Should not delete anything through a symlinked version directory.",
);
t.true(
fs
.lstatSync(path.join(toolcacheRoot, "CodeQL", "9.9.9"))
.isSymbolicLink(),
);
t.deepEqual(cleanupDiagnostic, {
deletedVersions: [CLEANUP_STALE_VERSION],
failed: false,
});
});
},
);

View File

@@ -2,14 +2,17 @@ import * as fs from "fs";
import { OutgoingHttpHeaders } from "http";
import * as path from "path";
import * as core from "@actions/core";
import * as toolcache from "@actions/tool-cache";
import { default as deepEqual } from "fast-deep-equal";
import * as semver from "semver";
import { v4 as uuidV4 } from "uuid";
import { ActionState } from "./action-common";
import {
isAnalyzingPullRequest,
isDynamicWorkflow,
isGitHubHostedRunner,
isRunningLocalAction,
} from "./actions-util";
import * as api from "./api-client";
@@ -19,6 +22,7 @@ import {
makeDiagnostic,
makeTelemetryDiagnostic,
} from "./diagnostics";
import { EnvVar, getEnv } from "./environment";
import {
CODEQL_VERSION_ZSTD_BUNDLE,
CodeQLDefaultVersionInfo,
@@ -30,8 +34,10 @@ import { Logger } from "./logging";
import { getCodeQlVersionsForOverlayBaseDatabases } from "./overlay/caching";
import * as tar from "./tar";
import {
deleteToolcacheBundles,
downloadAndExtract,
getToolcacheDirectory,
isToolcacheOnWorkspaceFilesystem,
ToolsDownloadStatusReport,
writeToolcacheMarkerFile,
} from "./tools-download";
@@ -75,7 +81,7 @@ export function getCodeQLBundleName(
if (process.platform === "win32") {
platform = "win64";
} else if (process.platform === "linux") {
platform = "linux64";
platform = process.arch === "arm64" ? "linux-arm64" : "linux64";
} else if (process.platform === "darwin") {
platform = "osx64";
} else {
@@ -784,6 +790,7 @@ export const downloadCodeQL = async function (
apiDetails: api.GitHubApiDetails,
tarVersion: tar.TarVersion | undefined,
tempDir: string,
features: FeatureEnablement,
logger: Logger,
): Promise<{
codeqlFolder: string;
@@ -817,6 +824,8 @@ export const downloadCodeQL = async function (
const extractedBundlePath =
toolcacheInfo?.path ?? getTempExtractionDir(tempDir);
await tryDeleteToolcacheBundles({ env: getEnv(), features, logger });
const statusReport = await downloadAndExtract(
codeqlURL,
compressionMethod,
@@ -869,6 +878,48 @@ function getToolcacheDestinationInfo(
return undefined;
}
/**
* Reclaims disk space by deleting the CodeQL tools from the toolcache, if enabled.
*
* On GitHub-hosted runners the toolcache shares a filesystem with the workspace, so tools left in
* the toolcache take up space that the analysis could use instead. This holds wherever we extract
* the tools we are obtaining, since the toolcache is on that filesystem either way.
*/
async function tryDeleteToolcacheBundles({
env,
features,
logger,
}: ActionState<["Logger", "ReadOnlyEnv", "FeatureFlags"]>): Promise<void> {
// A step that has already set up CodeQL may hand out a path into the toolcache that a later step
// runs, so only the first step to set it up can know that nothing else relies on the toolcache.
if (env.getOptional(EnvVar.HAS_SET_UP_CODEQL) !== undefined) {
logger.debug(
"Not deleting the CodeQL tools from the toolcache since a previous step in this job has " +
"already set up CodeQL.",
);
return;
}
if (
!isGitHubHostedRunner() ||
!isToolcacheOnWorkspaceFilesystem(logger) ||
!(await features.getValue(Feature.CleanupToolcacheBundles))
) {
return;
}
const result = await deleteToolcacheBundles({ env, logger });
addNoLanguageDiagnostic(
undefined,
makeTelemetryDiagnostic(
"codeql-action/toolcache-bundle-cleanup",
"Toolcache CodeQL bundle cleanup",
{ ...result },
),
);
}
export function getCodeQLURLVersion(url: string): string {
const match = url.match(/\/codeql-bundle-(.*)\//);
if (match === null || match.length < 2) {
@@ -978,6 +1029,7 @@ export async function setupCodeQLBundle(
apiDetails,
zstdAvailability.version,
tempDir,
features,
logger,
);
toolsVersion = result.toolsVersion;
@@ -989,6 +1041,11 @@ export async function setupCodeQLBundle(
default:
util.assertNever(source);
}
// Record that this job now has a copy of the CodeQL tools, so that a later step doesn't delete
// the toolcache out from under the path we are about to return.
core.exportVariable(EnvVar.HAS_SET_UP_CODEQL, "true");
return {
codeqlFolder,
toolsDownloadStatusReport,

View File

@@ -14,7 +14,10 @@ import {
isSelfHostedRunner,
} from "./actions-util";
import { getAnalysisKey, getApiClient } from "./api-client";
import { getCachedCodeQlVersion } from "./cli/output-cache";
import {
getCachedCodeQlVersion,
getCommandCacheFilePath,
} from "./cli/output-cache";
import type { Config } from "./config/action-config";
import type { ComputedInput, InputName } from "./config/inputs";
import { parseRegistriesWithoutCredentials } from "./config/pack-registries";
@@ -376,7 +379,10 @@ export async function createStatusReportBase(
core.exportVariable(EnvVar.WORKFLOW_STARTED_AT, workflowStartedAt);
}
const runnerOs = getRequiredEnvParam("RUNNER_OS");
const codeQlCliVersion = getCachedCodeQlVersion(logger, getEnv());
const codeQlCliVersion = getCachedCodeQlVersion(
logger,
getCommandCacheFilePath(getEnv()),
);
const actionRef = process.env["GITHUB_ACTION_REF"] || "";
const testingEnvironment = getTestingEnvironment();
// re-export the testing environment variable so that it is available to subsequent steps,
@@ -620,8 +626,21 @@ export interface InitWithConfigStatusReport extends InitStatusReport {
/** Fields of the init status report populated when the tools source is `download`. */
export interface InitToolsDownloadFields {
/** Time taken to download the bundle, in milliseconds. */
/**
* Time taken to download the bundle, in milliseconds. Not populated when the bundle is downloaded
* and extracted concurrently.
*/
tools_download_duration_ms?: number;
/**
* Time taken to extract the bundle, in milliseconds. Not populated when the bundle is downloaded
* and extracted concurrently.
*/
tools_extraction_duration_ms?: number;
/**
* Total time taken to make the bundle available on disk, in milliseconds. This includes any time
* spent on a streaming attempt that failed and fell back to downloading before extracting.
*/
tools_total_duration_ms?: number;
/**
* Whether the relevant tools dotcom feature flags have been misconfigured.
* Only populated if we attempt to determine the default version based on the dotcom feature flags. */

View File

@@ -937,7 +937,9 @@ export function mockBundleDownloadApi({
process.platform === "win32"
? "win64"
: process.platform === "linux"
? "linux64"
? process.arch === "arm64"
? "linux-arm64"
: "linux64"
: "osx64";
const baseUrl = apiDetails?.url ?? "https://example.com";

View File

@@ -15,7 +15,7 @@ import { withTmpDir } from "./util";
setupTests(test);
test.serial(
"downloadAndExtract reports the duration when downloading before extracting",
"downloadAndExtract reports the durations when downloading before extracting",
async (t) => {
await withTmpDir(async (tmpDir) => {
const archivePath = path.join(tmpDir, "codeql-bundle.tar.gz");
@@ -34,6 +34,8 @@ test.serial(
);
t.assert(Number.isInteger(statusReport.downloadDurationMs));
t.assert(Number.isInteger(statusReport.extractionDurationMs));
t.assert(Number.isInteger(statusReport.totalDurationMs));
});
},
);
@@ -67,6 +69,7 @@ test.serial(
);
t.assert(Number.isInteger(statusReport.downloadDurationMs));
t.assert(Number.isInteger(statusReport.totalDurationMs));
t.true(request.isDone());
t.false(extractTarZst.called);
t.true(downloadTool.calledOnce);
@@ -76,7 +79,7 @@ test.serial(
);
test.serial(
"downloadAndExtract omits the download duration when streaming extraction",
"downloadAndExtract reports only the total duration when streaming extraction",
async (t) => {
await withTmpDir(async (tmpDir) => {
sinon.stub(process, "platform").value("linux");
@@ -106,7 +109,9 @@ test.serial(
getRunnerLogger(true),
);
t.deepEqual(statusReport, {});
t.assert(Number.isInteger(statusReport.totalDurationMs));
t.is(statusReport.downloadDurationMs, undefined);
t.is(statusReport.extractionDurationMs, undefined);
t.false(downloadTool.called);
t.true(extractTarZst.calledOnce);
t.true(request.isDone());

View File

@@ -10,6 +10,8 @@ import * as toolcache from "@actions/tool-cache";
import { https } from "follow-redirects";
import * as semver from "semver";
import { ActionState } from "./action-common";
import { ActionsEnvVars, getEnv, ReadOnlyEnv } from "./environment";
import { formatDuration, Logger } from "./logging";
import * as tar from "./tar";
import { cleanUpPath, getErrorMessage, getRequiredEnvParam } from "./util";
@@ -31,7 +33,21 @@ const STREAMING_STALL_TIMEOUT_MS = 5 * 60 * 1000; // 5 minutes
const TOOLCACHE_TOOL_NAME = "CodeQL";
export type ToolsDownloadStatusReport = {
/**
* Time spent downloading the bundle, in milliseconds. Not populated when the bundle is downloaded
* and extracted concurrently, since the two cannot be told apart.
*/
downloadDurationMs?: number;
/**
* Time spent extracting the bundle, in milliseconds. Not populated when the bundle is downloaded
* and extracted concurrently, since the two cannot be told apart.
*/
extractionDurationMs?: number;
/**
* Total time taken to make the bundle available on disk, in milliseconds. This includes any time
* spent on a streaming attempt that failed and fell back to downloading before extracting.
*/
totalDurationMs: number;
};
export async function downloadAndExtract(
@@ -47,11 +63,12 @@ export async function downloadAndExtract(
`Downloading CodeQL tools from ${codeqlURL} . This may take a while.`,
);
const startTime = performance.now();
try {
if (compressionMethod === "zstd" && process.platform === "linux") {
logger.info(`Streaming the extraction of the CodeQL bundle.`);
const toolsInstallStart = performance.now();
await downloadAndExtractZstdWithStreaming(
codeqlURL,
dest,
@@ -61,16 +78,14 @@ export async function downloadAndExtract(
logger,
);
const combinedDurationMs = Math.round(
performance.now() - toolsInstallStart,
);
const totalDurationMs = Math.round(performance.now() - startTime);
logger.info(
`Finished downloading and extracting CodeQL bundle to ${dest} (${formatDuration(
combinedDurationMs,
totalDurationMs,
)}).`,
);
return {};
return { totalDurationMs };
}
} catch (e) {
core.warning(
@@ -98,7 +113,7 @@ export async function downloadAndExtract(
)}).`,
);
let extractionDurationMs: number;
let extractionDurationMs: number | undefined;
try {
logger.info("Extracting CodeQL bundle.");
@@ -120,7 +135,11 @@ export async function downloadAndExtract(
await cleanUpPath(archivedBundlePath, "CodeQL bundle archive", logger);
}
return { downloadDurationMs };
return {
downloadDurationMs,
extractionDurationMs,
totalDurationMs: Math.round(performance.now() - startTime),
};
}
async function downloadAndExtractZstdWithStreaming(
@@ -180,16 +199,154 @@ async function downloadAndExtractZstdWithStreaming(
await tar.extractTarZst(response, dest, tarVersion, logger);
}
/** Gets the path to the toolcache directory that holds all versions of the CodeQL tools. */
function getToolcacheToolDirectory(env: ReadOnlyEnv): string {
return path.join(
env.getRequired(ActionsEnvVars.RUNNER_TOOL_CACHE),
TOOLCACHE_TOOL_NAME,
);
}
/** Gets the name of the toolcache directory that holds the given version of the CodeQL tools. */
function getToolcacheVersionDirectoryName(version: string): string {
return semver.clean(version) || version;
}
/** Gets the path to the toolcache directory for the specified version of the CodeQL tools. */
export function getToolcacheDirectory(version: string): string {
return path.join(
getRequiredEnvParam("RUNNER_TOOL_CACHE"),
TOOLCACHE_TOOL_NAME,
semver.clean(version) || version,
getToolcacheToolDirectory(getEnv()),
getToolcacheVersionDirectoryName(version),
os.arch() || "",
);
}
/**
* Whether the toolcache is on the same filesystem as the workspace, and so whether deleting the
* tools frees up disk space that the analysis can use.
*
* These are separate volumes on some runner images. Windows runners, for example, keep the
* toolcache on `C:` while the workspace is on `D:`.
*/
export function isToolcacheOnWorkspaceFilesystem(logger: Logger): boolean {
try {
return (
fs.statSync(getRequiredEnvParam("RUNNER_TOOL_CACHE")).dev ===
fs.statSync(getRequiredEnvParam("GITHUB_WORKSPACE")).dev
);
} catch (e) {
logger.debug(
`Could not determine whether the toolcache is on the same filesystem as the workspace: ${getErrorMessage(e)}`,
);
return false;
}
}
/** The outcome of trying to reclaim disk space by deleting the CodeQL tools from the toolcache. */
export interface ToolcacheCleanupResult {
/** The versions of the CodeQL tools that were deleted. */
deletedVersions: string[];
/**
* Whether we hit an error while trying to delete the tools. Distinguishes a toolcache that had
* nothing to reclaim from one we failed to clean up.
*/
failed: boolean;
}
/**
* Deletes every version of the CodeQL tools from the toolcache.
*
* Only safe to call when we are about to download the tools, since that means we did not resolve
* them from the toolcache and so nothing in there is in use by this job.
*
* This only ever touches the CodeQL directory of the toolcache. Cleanup errors are logged and
* returned as `failed: true` rather than thrown.
*
* @returns the versions that were deleted, and whether we hit an error while trying.
*/
export async function deleteToolcacheBundles({
env,
logger,
}: ActionState<["Logger", "ReadOnlyEnv"]>): Promise<ToolcacheCleanupResult> {
let toolDirectory: string;
try {
toolDirectory = getToolcacheToolDirectory(env);
} catch (e) {
logger.info(
`Unable to determine toolcache directory: ${getErrorMessage(e)}`,
);
return { deletedVersions: [], failed: true };
}
try {
// Refuse to follow a symlinked CodeQL directory, so that we can only ever delete paths that are
// really inside the toolcache.
if ((await fs.promises.lstat(toolDirectory)).isSymbolicLink()) {
logger.info(
`Not deleting the CodeQL tools from the toolcache since '${toolDirectory}' is a symlink.`,
);
return { deletedVersions: [], failed: true };
}
} catch (e: any) {
if (e?.code === "ENOENT") {
logger.debug(
`There are no CodeQL tools at '${toolDirectory}' to delete from the toolcache.`,
);
return { deletedVersions: [], failed: false };
}
logger.info(
`Failed to inspect the CodeQL tools at '${toolDirectory}': ${getErrorMessage(e)}`,
);
return { deletedVersions: [], failed: true };
}
try {
const entries = await fs.promises.readdir(toolDirectory, {
withFileTypes: true,
});
const deletedVersions: string[] = [];
let failed = false;
for (const entry of entries) {
// `isDirectory` is false for a symlink, so we never delete a version directory that is
// really somewhere else.
if (!entry.isDirectory()) {
logger.debug(
`Not deleting '${entry.name}' from the CodeQL toolcache since it is not a directory.`,
);
continue;
}
const versionDirectory = path.join(toolDirectory, entry.name);
try {
await fs.promises.rm(versionDirectory, {
force: true,
recursive: true,
});
deletedVersions.push(entry.name);
logger.info(
`Deleted the CodeQL tools at '${versionDirectory}' from the toolcache to free up disk space.`,
);
} catch (e) {
failed = true;
logger.info(
`Failed to delete the CodeQL tools at '${versionDirectory}' from the toolcache: ${getErrorMessage(e)}`,
);
}
}
return { deletedVersions: deletedVersions.sort(), failed };
} catch (e) {
logger.info(
`Failed to clean up the CodeQL toolcache at '${toolDirectory}': ${getErrorMessage(e)}`,
);
return { deletedVersions: [], failed: true };
}
}
export function writeToolcacheMarkerFile(
extractedPath: string,
logger: Logger,

View File

@@ -842,9 +842,13 @@ export async function checkForTimeout() {
* directory with the name hostedtoolcache which is present on
* GitHub-hosted runners.
*
* @returns true iff the runner is hosted by GitHub
* Since this is a heuristic over how the runner happens to be named and laid out, it also matches
* self-hosted runners that are configured to resemble hosted ones. Prefer
* `isGitHubHostedRunner` when you need the answer the Actions service reports.
*
* @returns true iff the runner looks like it is hosted by GitHub
*/
export function isHostedRunner() {
export function looksLikeHostedRunner() {
return (
// Name of the runner on hosted Windows runners
process.env["RUNNER_NAME"]?.includes("Hosted Agent") ||

View File

@@ -37,5 +37,5 @@
"@octokit/core/dist-types/types": ["./node_modules/@octokit/core/dist-types/types.d.ts"]
},
},
"exclude": ["node_modules", "pr-checks"]
"exclude": ["node_modules", "pr-checks", "scripts/changetool"]
}