Nightlies are the first bundles to contain per-language bundles, so
allow a workflow that explicitly asks for a nightly to use one. Their
tags record the date they were built rather than a version, so there is
no minimum version to check against; a nightly is always at least as new
as the first release to publish per-language bundles.
Only do this when a nightly was asked for explicitly. Nightlies can also
be forced for analyses that did not ask for one, and those should keep
getting the bundle that contains every language while the feature is
still being tested.
Fall back to the combined bundle from the same nightly if it turns out
not to contain the language, since we chose the bundle ourselves rather
than being asked for it.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4c7b873-8278-4a36-b67c-6cc5ea3b2316
Swift autobuild does not finish within the timeout on a standard macOS
runner, which is why the generated Swift checks use a larger one.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4c7b873-8278-4a36-b67c-6cc5ea3b2316
The reported durations do not distinguish a streaming attempt that failed
from one that was never made, since a bundle that is not Zstandard skips
streaming altogether. Telemetry from self-hosted Linux runners, which take
the download-then-extract path consistently rather than intermittently,
shows this is not a hypothetical case.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4c7b873-8278-4a36-b67c-6cc5ea3b2316
The Actions QL pack depends on the JavaScript one, which is the only
dependency of its kind, so the Actions bundle carries the JavaScript
extractor as well as its own. Let each language declare the extractors
its bundle is expected to contain, so that the check still fails if a
bundle contains anything beyond what its language needs.
Fold the generated Actions-only check into this workflow, which now
covers every language, and with it the ability for a generated check to
run on additional branches, which nothing needs any more.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4c7b873-8278-4a36-b67c-6cc5ea3b2316
Checks that each per-language bundle is trimmed to the language it is
for, is kept out of the toolcache, and can build a database for that
language.
This is not generated from `pr-checks`, since each language needs its own
platform and build steps, which the generated checks cannot express.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4c7b873-8278-4a36-b67c-6cc5ea3b2316
The runner image ships with its own copy of CodeQL in the toolcache, so
checking whether the toolcache contains CodeQL at all does not tell us
anything about the bundle this test downloaded.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4c7b873-8278-4a36-b67c-6cc5ea3b2316
A bundle that contains a single language can also be requested directly
via the `tools` input, in which case we did not choose it but must still
keep it out of the toolcache, since a later job analyzing a different
language could otherwise pick up an installation that is missing the
extractor it needs.
Recognise such bundles by their name. When one was requested explicitly,
a missing bundle is an error rather than a reason to fall back, since
substituting a different bundle would ignore what was asked for.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4c7b873-8278-4a36-b67c-6cc5ea3b2316
Per-language bundles are not yet published as part of a release, so this
check points at a pre-release and needs exercising before it can run on
`main`. Allow a check to opt in to running on additional branches so that
it can be, without opening a pull request.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4c7b873-8278-4a36-b67c-6cc5ea3b2316
Checks that a bundle containing only a single language is both trimmed
and able to analyze that language. Uses the Actions bundle, since the
Actions QL pack is the only one that depends on the library pack of
another language, and so is the case most likely to be missing something
it needs.
The bundle is pinned to a specific pre-release, so this check will need
updating once per-language bundles are published as part of a release.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4c7b873-8278-4a36-b67c-6cc5ea3b2316
Download a bundle containing only the single language being analyzed,
rather than the combined bundle that contains every language, when that
is both safe and beneficial. Per-language bundles are substantially
smaller, so this saves download time and disk space on the runner.
Eligibility is decided in one place, since it is easy for these
conditions to drift apart. Per-language bundles are never added to the
toolcache, because a bundle for one language must not be reused for a
job that analyzes another.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4c7b873-8278-4a36-b67c-6cc5ea3b2316
Delete each version directory individually so that a symlinked one is skipped rather than removed, take an `ActionState` so the environment is read through `ReadOnlyEnv` rather than the deprecated `getOptionalEnvVar`, let `deleteToolcacheBundles` report its own failure to locate the toolcache instead of having the caller catch it, quote paths in log messages, and rename `HAS_OBTAINED_CODEQL_TOOLS` to `HAS_SET_UP_CODEQL`, which is also set when we find the tools in the toolcache rather than downloading them.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Some runner images keep the toolcache on a different volume to the workspace, so deleting the tools there frees up disk space that the analysis cannot use, and costs a later step that wanted them in the toolcache a download. Windows runners are laid out this way, with the toolcache on `C:` and the workspace on `D:`.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run the cleanup even when the download will not be cached in the toolcache, since the toolcache shares a filesystem with the directory we extract to, so freeing it helps either way, and report an error other than the toolcache being absent as a failure rather than as an empty toolcache.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The check installs the tools from a URL, so most versions in its matrix are downloaded rather than found in the toolcache, which is when the cleanup runs, and it then builds and analyses seven languages, so a bundle we damaged on the way in would show up.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When we download a bundle the toolcache often already holds a different one that the job will not use, and on GitHub-hosted runners it shares a filesystem with the workspace, so it takes space away from the analysis. Empty the toolcache before downloading, which also frees space for the archive during extraction, and which is safe because getting as far as a download means the tools were not resolved from the toolcache. Skip this once a step has obtained the tools, since a later step may run a path it was given, and gate it on the runner being GitHub-hosted and on a feature flag that is off by default.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`deleteToolcacheBundles` removes `$RUNNER_TOOL_CACHE/CodeQL` and reports which versions were there. It refuses to follow a symlinked CodeQL directory so that it can only ever delete paths that are really inside the toolcache, and reports failures rather than throwing. Not called yet.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`isHostedRunner` infers hostedness from the runner name and the toolcache path, so it also matches self-hosted runners that are configured to resemble hosted ones. Rename it to `looksLikeHostedRunner` so callers can see they are getting a heuristic, and add `isGitHubHostedRunner`, which reads the `RUNNER_ENVIRONMENT` value the Actions service reports. The existing callers keep the heuristic, so there is no behaviour change.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Analyze all languages CodeQL supports on Linux Arm64 (except Swift,
which is macOS only) in the Linux Arm64 check, building the compiled
ones via the shared multi-language-repo build.sh
- De-duplicate the check's language list via a shared LANGUAGES env var
used by both the init input and the assert loop
- Fix CHANGELOG tense: "download" -> "downloads"
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
