Compare commits

...

67 Commits

Author SHA1 Message Date
Michael B. Gale
8b185ebf3d Add e2e test 2026-10-16 23:02:59 +01:00
Michael B. Gale
afed03cada Don't perform check for non-traced analyses 2026-10-16 22:56:47 +01:00
Michael B. Gale
e5b2809f92 Refactor non-Swift-specific parts out into traced.ts 2026-10-16 18:19:13 +01:00
Michael B. Gale
f4ae223961 Rename feature to SkipUnsupportedTracedAnalysis 2026-10-16 17:52:53 +01:00
Michael B. Gale
b09f3c6b4d Move xcodeVersion to platform.ts 2026-10-16 17:50:07 +01:00
Michael B. Gale
61cb554113 Use ActionConfig in diagnostics.ts 2026-10-16 17:44:26 +01:00
Michael B. Gale
cf5d0598fd Fix test creating diagnostic locally 2026-10-16 17:28:16 +01:00
Michael B. Gale
cba494e0a6 Check if Xcode version is unsupported 2026-10-16 17:28:16 +01:00
Michael B. Gale
df26bdf81b Add function to determine Xcode version based on symlink 2026-10-16 17:28:16 +01:00
Michael B. Gale
e0c11175e5 Add FF to fail if macOS version is unsupported 2026-10-16 17:28:16 +01:00
Michael B. Gale
db45eabc68 Check if macOS version is unsupported 2026-10-16 17:28:16 +01:00
Michael B. Gale
094875d25b Skip Swift checks if tools feature is enabled 2026-10-16 17:28:15 +01:00
Michael B. Gale
6989a48f28 Add swiftSupportsAllPlatforms to ToolsFeature 2026-10-16 17:28:15 +01:00
Michael B. Gale
70550612a5 Use macOSVersion in isSwiftCompatible 2026-10-16 17:28:13 +01:00
Michael B. Gale
a361259e4a Refactor swift check out of init-action.ts 2026-10-16 17:28:12 +01:00
Michael B. Gale
13bca40cb4 Add macOSVersion helper 2026-10-16 17:28:08 +01:00
Michael B. Gale
4b331afa38 Make getCodeQLForCmd use Env 2026-10-16 17:24:41 +01:00
Michael B. Gale
0797511522 Add osRelease to BaseState 2026-10-16 17:24:41 +01:00
Michael B. Gale
f69da902e0 Add e2e test 2026-10-16 17:18:11 +01:00
Michael B. Gale
8dc9a9913b Omit --sarif-codescanning-config if the config file doesn't exist 2026-10-16 17:18:11 +01:00
Michael B. Gale
031b097718 Refactor writeTemporaryDiagnosticFile out of add[NoLanguage]Diagnostic 2026-10-16 17:18:11 +01:00
Michael B. Gale
a3e8512735 Use temporary diagnostic dir in generateFailedSarif 2026-10-16 17:18:10 +01:00
Michael B. Gale
78ebd22fd6 Make diagnostics less dependent on global state
Add limited test coverage
2026-10-16 17:18:10 +01:00
Michael B. Gale
39f74e9837 Write diagnostics to temporary location when database isn't ready 2026-10-16 17:18:09 +01:00
Michael B. Gale
11fd255a0b Refactor writeDiagnosticFile out of writeDiagnostic 2026-10-16 17:18:08 +01:00
Michael B. Gale
d6b8ed80f2 Ensure that rootDirectory is suitable for sarifFile in uploadSarifArtifact 2026-10-16 17:02:02 +01:00
Michael B. Gale
3a61940e40 Improve logging in uploadSarifArtifact 2026-10-16 17:02:02 +01:00
Michael B. Gale
850cc877c3 Upload failed SARIF as workflow artifact in test mode 2026-10-16 17:01:24 +01:00
Michael B. Gale
68eb797e87 Skip waiting for SARIF if nothing was uploaded 2026-10-16 16:54:37 +01:00
Michael B. Gale
2765b8f6f9 Propagate Config to combineSarifFilesUsingCLI 2026-10-16 16:54:37 +01:00
Michael B. Gale
b1c5d475ab Add and use shouldSkipGenerateFailedSarif 2026-10-16 16:54:37 +01:00
Michael B. Gale
e180f803c5 Allow to explicitly not skip the SARIF upload 2026-10-16 16:54:36 +01:00
Michael B. Gale
bf0e2ed66e Log when we didn't get a normal configuration file 2026-10-16 16:54:36 +01:00
Michael B. Gale
604377fd2d Move inline script from diagnostics-export.yml to a .ts file 2026-10-16 16:53:07 +01:00
Michael B. Gale
83e1837c6d Merge pull request #4247 from github/mbg/copilot/clarify-feature-flags
Add Copilot instructions for FFs, and update documentation more generally
2026-10-16 15:20:34 +00:00
Michael B. Gale
5ecf3b32e9 Suggest "Minor" prefix 2026-10-16 16:03:32 +01:00
Michael B. Gale
e461913b38 Drop "requests" again 2026-10-16 16:03:00 +01:00
Michael B. Gale
6992155901 Elaborate GHES details 2026-10-16 16:02:01 +01:00
Michael B. Gale
f3c93e3106 Add some guidelines for change notes 2026-10-16 15:32:50 +01:00
Michael B. Gale
2441e6e56e Update "Review instructions" based on recent reviews 2026-10-16 15:31:56 +01:00
Michael B. Gale
0d2baf6822 Update copilot-instructions.md to mention feature flags 2026-10-16 15:11:17 +01:00
Henry Mercer
ae098fa9fe Merge pull request #4242 from github/henrymercer/graduate-toolcache-cleanup
Enable cleaning up toolcache bundles by default
2026-10-16 12:43:00 +00:00
Michael B. Gale
0a458f10ac Merge pull request #4230 from github/mbg/auth/allow-unauthed
Make `auth` in `GitHubApiDetails` optional
2026-10-16 11:05:53 +00:00
Henry Mercer
4ebc48bf25 Remove the toolcache cleanup test override
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9bf883d4-ecd7-4428-970f-86895f760571
2026-10-15 14:54:30 +01:00
Henry Mercer
1ec6b03cf4 Enable toolcache bundle cleanup by default
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9bf883d4-ecd7-4428-970f-86895f760571
2026-10-15 14:54:30 +01:00
Michael B. Gale
7a6e2f798f Merge pull request #4240 from github/mergeback/v4.39.0-to-main-6f2ac81f
Mergeback v4.39.0 refs/heads/releases/v4 into main
2026-10-15 11:25:10 +00:00
github-actions[bot]
46d3daa879 Rebuild 2026-10-15 10:57:15 +00:00
github-actions[bot]
2edace40f1 Update changelog and version after v4.39.0 2026-10-15 10:57:06 +00:00
Michael B. Gale
e5b217aeb5 Merge pull request #4238 from github/dependabot/github_actions/dot-github/workflows/actions-minor-c673fd7efc
Bump the actions-minor group across 1 directory with 3 updates
2026-10-15 10:26:50 +00:00
Michael B. Gale
3ec43abae3 Merge branch 'main' into dependabot/github_actions/dot-github/workflows/actions-minor-c673fd7efc 2026-10-15 11:01:21 +01:00
github-actions[bot]
3afe9c01db Rebuild 2026-10-14 21:26:52 +00:00
dependabot[bot]
8844fb33e0 Bump the actions-minor group across 1 directory with 3 updates
Bumps the actions-minor group with 3 updates in the /.github/workflows directory: [actions/upload-artifact](https://github.com/actions/upload-artifact), [actions/setup-node](https://github.com/actions/setup-node) and [actions/download-artifact](https://github.com/actions/download-artifact).


Updates `actions/upload-artifact` from 7.0.1 to 7.0.2
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](043fb46d1a...cf430e030d)

Updates `actions/setup-node` from 7.0.0 to 7.1.0
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](8207627860...949feb2413)

Updates `actions/download-artifact` from 8.0.1 to 8.0.2
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](3e5f45b2cf...9000827ccb)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: 7.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions-minor
- dependency-name: actions/setup-node
  dependency-version: 7.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions-minor
- dependency-name: actions/download-artifact
  dependency-version: 8.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-14 21:24:28 +00:00
Michael B. Gale
c1641847bb Remove explicit authorization header from uploadBundledDatabase 2026-10-14 20:03:19 +01:00
Michael B. Gale
047c751d92 Add a test for uploadBundledDatabase 2026-10-14 20:02:27 +01:00
Michael B. Gale
f6e8dce502 Pass apiDetails to getApiClient in uploadBundledDatabase 2026-10-14 18:28:21 +01:00
Michael B. Gale
a08b587b61 Allow getApiClient to accept GitHubApiDetails 2026-10-14 18:27:15 +01:00
Michael B. Gale
bfe264b559 Add very basic getApiClientWithExternalAuth test 2026-10-14 18:19:52 +01:00
Michael B. Gale
f9b4185b99 Check token value in uploadBundledDatabase 2026-10-14 18:19:52 +01:00
Michael B. Gale
66c96fdc35 Check auth value in getAuthorizationHeaderFor
Also add some tests for it
2026-10-14 18:19:52 +01:00
Michael B. Gale
1d30a842cd Guard against blank auth value
This shouldn't be possible, but was previously handled by the library
2026-10-14 18:19:51 +01:00
Michael B. Gale
1b0a617641 Make externalRepoAuth check more consistent 2026-10-14 18:19:51 +01:00
Michael B. Gale
96501a33ac Use getCurrentInstance() 2026-10-14 18:19:51 +01:00
Michael B. Gale
a2438139ce Make auth property on GitHubApiDetails optional 2026-10-14 18:19:50 +01:00
Michael B. Gale
e0625309c3 Move GitHubApiDetails to github.ts 2026-10-14 18:19:50 +01:00
Michael B. Gale
61bfcffab9 Add functions to get the current and dotcom instance 2026-10-14 18:19:50 +01:00
Michael B. Gale
102e46e9d1 Move some definitions out of util.ts into a new github.ts module 2026-10-14 18:19:46 +01:00
Michael B. Gale
0735d7910f Improve GitHubApiDetails types and documentation 2026-10-14 18:14:48 +01:00
79 changed files with 3906 additions and 1318 deletions

View File

@@ -11,9 +11,12 @@ The CodeQL Action is used in GitHub Actions workflows to run CodeQL scans using
## Review instructions
- When wording review comments, be helpful and friendly. Assume that the PR author has written the code with the best of intentions. Word your comments constructively as suggestions for improvements. Do not word suggestions as commands.
- Prioritise identifying critical issues, such as logic bugs, over comments on documentation or typos.
- When wording review comments, be helpful and friendly. Assume that the PR author has written the code with the best of intentions. Never word suggestions as commands or using the imperative. Examples of what you should never write: "Preserve that non-fatal behaviour here" or "Please preserve that non-fatal behaviour here".
- When writing feedback, assume that the PR author is competent and is able to understand whether the point you are making is valid or not, why you are raising it, and how to best address it. While you can suggest how to address a point you are making, don't phrase it as a requirement or assume that it is the only solution.
- Always word your comments constructively as suggestions for improvements, for example: "It would make sense to ..." or "Could you add ...?" or "Would it make sense to ...?"
- If you want to comment on a change that you believe will fail a CI check, do not present the CI failure you expect as a fact. Instead, write that you think a change "may" lead to a failure in CI. Suggest that, if such a failure manifests, the changes you are commenting on may be the place responsible for the failure and are worth looking at.
- If a suggestion you make is suitable for a follow-up, such as a refactoring that doesn't change the behaviour or fixing a typo in a comment, mention that it can be addressed in a later PR rather than blocking this one.
- If a suggestion you make is suitable for a follow-up, such as a refactoring that doesn't change the behaviour or fixing a typo in a comment, mention that it can be addressed in a later PR rather than blocking this one or prefix your comment with "Minor: ".
- If a change is a net improvement, for example because it improves on an existing limitation of existing code, do not complain about pre-existing problems that remain. You may comment on them, but you should make it clear that the thing you are commenting on is not new by writing e.g. "Not new in this PR, but [..]" followed by your description of the issue and a suggestion that it could be improved at the same time with e.g. "Consider whether this is worth addressing as part of this PR as well."
## Generated code
@@ -22,6 +25,16 @@ The main codebase of the CodeQL Action is written in TypeScript and compiled to
GitHub Actions workflows in the `.github/workflows` directory whose filenames start with two underscores (e.g. `__all-platform-bundle.yml`) are automatically generated using the `pr-checks/sync.sh` script from template files in the `pr-checks/checks` directory. Therefore, you do not need to review files in the `.github/workflows` directory that start with two underscores. However, you should review changes to the `pr-checks` directory as well as workflows in the `.github/workflows` directory that do not start with underscores.
## Feature flags
Some functionality in the code may be gated behind feature flags, which are controlled server-side. If the functionality that is gated behind them is user-facing, then this should be documented in a change note at the time that the gated functionality is added. See `CHANGELOG.md` for existing notes that mention "experimental" for examples. These typically include information about when users may expect to see the feature rolled out.
When a feature flag that gates some functionality is removed, this does not need to be documented in a change note since the feature is already rolled out and the notes for the release which introduced the feature already describe it. The only exception is functionality which will start having an effect on GHES as a result of the feature flag removal:
- Most functionality gated behind feature flags does not affect GHES.
- We can make functionality available on GHES by setting `defaultValue` to `true` first and waiting for this to have had an effect on GHES. No change note is needed if a feature flag is removed where `defaultValue` was `true`.
- When removing a feature flag where `defaultValue` was `false` and the functionality starts to affect GHES, a change note may be appropriate.
## Release process
Changes that are merged to the `main` branch are not used in production straight away. Instead, changes from `main` must first be released using the release process, which has to be manually triggered by a staff member. Therefore, it is possible for further changes to be made to the code after some changes have been merged to `main` but before they are released. Once the release process automation is started, a series of PRs are automatically generated:

View File

@@ -87,7 +87,7 @@ jobs:
post-processed-sarif-path: '${{ runner.temp }}/post-processed'
- name: Upload SARIF files
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: |
analysis-kinds-${{ matrix.os }}-${{ matrix.version }}-${{ matrix.analysis-kinds }}
@@ -95,7 +95,7 @@ jobs:
retention-days: 7
- name: Upload post-processed SARIF
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: |
post-processed-${{ matrix.os }}-${{ matrix.version }}-${{ matrix.analysis-kinds }}

View File

@@ -65,7 +65,7 @@ jobs:
output: '${{ runner.temp }}/results'
upload-database: false
- name: Upload SARIF
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: config-export-${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: '${{ runner.temp }}/results/javascript.sarif'

View File

@@ -47,7 +47,7 @@ jobs:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm

View File

@@ -48,6 +48,15 @@ jobs:
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
@@ -76,61 +85,18 @@ jobs:
output: '${{ runner.temp }}/results'
upload-database: false
- name: Upload SARIF
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: diagnostics-export-${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: '${{ runner.temp }}/results/javascript.sarif'
retention-days: 7
- name: Check diagnostics appear in SARIF
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
env:
SARIF_PATH: '${{ runner.temp }}/results/javascript.sarif'
with:
script: |
const fs = require('fs');
function checkStatusPageNotification(n) {
const expectedMessage = 'Plaintext message';
if (n.message.text !== expectedMessage) {
core.setFailed(`Expected the status page diagnostic to have the message '${expectedMessage}', but found '${n.message.text}'.`);
}
if (n.locations.length !== 1) {
core.setFailed(`Expected the status page diagnostic to have exactly 1 location, but found ${n.locations.length}.`);
}
}
const sarif = JSON.parse(fs.readFileSync(process.env['SARIF_PATH'], 'utf8'));
const run = sarif.runs[0];
const toolExecutionNotifications = run.invocations[0].toolExecutionNotifications;
const statusPageNotifications = toolExecutionNotifications.filter(n =>
n.descriptor.id === 'lang/diagnostics/example' && n.properties?.visibility?.statusPage
);
if (statusPageNotifications.length !== 1) {
core.setFailed(
'Expected exactly one status page reporting descriptor for this diagnostic in the ' +
`'runs[].invocations[].toolExecutionNotifications[]' SARIF property, but found ` +
`${statusPageNotifications.length}. All notification reporting descriptors: ` +
`${JSON.stringify(toolExecutionNotifications)}.`
);
}
checkStatusPageNotification(statusPageNotifications[0]);
const notifications = run.tool.driver.notifications;
const diagnosticNotification = notifications.filter(n =>
n.id === 'lang/diagnostics/example' && n.name === 'lang/diagnostics/example' &&
n.fullDescription.text === 'Diagnostic name'
);
if (diagnosticNotification.length !== 1) {
core.setFailed(
'Expected exactly one notification for this diagnostic in the ' +
`'runs[].tool.driver.notifications[]' SARIF property, but found ` +
`${diagnosticNotification.length}. All notifications: ` +
`${JSON.stringify(notifications)}.`
);
}
core.info('Finished diagnostic export test');
run: |
cd ../action
npx tsx ./pr-checks/check-diagnostics.ts
env:
CODEQL_ACTION_EXPORT_DIAGNOSTICS: true
CODEQL_ACTION_TEST_MODE: true

138
.github/workflows/__diagnostics-without-db.yml generated vendored Normal file
View File

@@ -0,0 +1,138 @@
# Warning: This file is generated automatically, and should not be modified.
# Instead, please modify the template in the pr-checks directory and run:
# pr-checks/sync.sh
# to regenerate this file.
name: PR Check - Diagnostic export without database
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GO111MODULE: auto
on:
push:
branches:
- main
- releases/v*
pull_request: {}
merge_group:
types:
- checks_requested
schedule:
- cron: '0 5 * * *'
workflow_dispatch:
inputs: {}
workflow_call:
inputs: {}
defaults:
run:
shell: bash
concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' || false }}
group: diagnostics-without-db-${{github.ref}}
jobs:
diagnostics-without-db:
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
version: linked
name: Diagnostic export without database
if: github.triggering_actor != 'dependabot[bot]'
permissions:
contents: read
security-events: read
timeout-minutes: 45
runs-on: ${{ matrix.os }}
steps:
# We deliberately need to make this step fail before the database is initialised, but after
# some diagnostic has been produced. The simplest way to do that right now is to force the
# use of a nightly release and then fail because we are attempting to analyse Swift on
# a non-macOS runner.
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
with:
version: ${{ matrix.version }}
use-all-platform-bundle: 'false'
setup-kotlin: 'true'
- name: Initialise failing analysis
id: init
uses: ./../action/init
continue-on-error: true
env:
CODEQL_ACTION_FORCE_NIGHTLY: true
with:
languages: swift
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Analyze
uses: ./analyze
# In a real workflow, this step wouldn't run. Since we used `continue-on-error`
# above, we manually disable it with an `if` condition.
if: false
with:
category: '/test-codeql-version:${{ matrix.version }}'
- name: Check diagnostics appear on disk
env:
SARIF_PATH: '${{ runner.temp }}/codeql-action-temp-diagnostics/'
run: |
cd ../action
npx tsx ./pr-checks/diagnostics-without-db.ts
env:
CODEQL_ACTION_SKIP_GENERATE_FAILED_SARIF: false
CODEQL_ACTION_SKIP_SARIF_UPLOAD: true
CODEQL_ACTION_TESTING_ENVIRONMENT: codeql-action-pr-checks
CODEQL_ACTION_TEST_MODE: true
diagnostics-without-db-validation:
if: github.triggering_actor != 'dependabot[bot]'
needs:
- diagnostics-without-db
permissions:
contents: read
security-events: read
timeout-minutes: 5
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Download workflow artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: test-mode-sarif-*
path: sarif
- name: List artifact contents
run: ls -laR ./sarif
- name: Check diagnostics appear in SARIF
env:
SARIF_PATH: "./sarif/codeql-failed-run.sarif"
DIAGNOSTIC_ID: "codeql-action/forced-nightly-cli"
DIAGNOSTIC_NAME: "A nightly release of CodeQL was used"
EXPECTED_LOCATIONS: "0"
EXPECTED_MESSAGE: |
GitHub configured this analysis to use a nightly release of CodeQL to allow you to preview changes from an upcoming release.
run: npx tsx ./pr-checks/check-diagnostics.ts
env:
CODEQL_ACTION_TEST_MODE: true

View File

@@ -97,7 +97,7 @@ jobs:
with:
output: '${{ runner.temp }}/results'
- name: Upload SARIF
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: with-baseline-information-${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: '${{ runner.temp }}/results/javascript.sarif'

View File

@@ -62,7 +62,7 @@ jobs:
with:
output: '${{ runner.temp }}/results'
- name: Upload SARIF
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: ${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: '${{ runner.temp }}/results/javascript.sarif'

View File

@@ -192,6 +192,5 @@ jobs:
exit 1
fi
env:
CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES: true
CODEQL_ACTION_RESOLVE_SUPPORTED_LANGUAGES_USING_CLI: true
CODEQL_ACTION_TEST_MODE: true

View File

@@ -80,7 +80,7 @@ jobs:
go-version: ${{ inputs.go-version || '>=1.21.0' }}
cache: false
- name: Install Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm

View File

@@ -80,7 +80,7 @@ jobs:
go-version: ${{ inputs.go-version || '>=1.21.0' }}
cache: false
- name: Install Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm

View File

@@ -80,7 +80,7 @@ jobs:
go-version: ${{ inputs.go-version || '>=1.21.0' }}
cache: false
- name: Install Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm

View File

@@ -80,7 +80,7 @@ jobs:
go-version: ${{ inputs.go-version || '>=1.21.0' }}
cache: false
- name: Install Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm

122
.github/workflows/__unsupported-macos.yml generated vendored Normal file
View File

@@ -0,0 +1,122 @@
# Warning: This file is generated automatically, and should not be modified.
# Instead, please modify the template in the pr-checks directory and run:
# pr-checks/sync.sh
# to regenerate this file.
name: PR Check - Unsupported macOS
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GO111MODULE: auto
on:
push:
branches:
- main
- releases/v*
pull_request: {}
merge_group:
types:
- checks_requested
schedule:
- cron: '0 5 * * *'
workflow_dispatch:
inputs: {}
workflow_call:
inputs: {}
defaults:
run:
shell: bash
concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' || false }}
group: unsupported-macos-${{github.ref}}
jobs:
unsupported-macos:
strategy:
fail-fast: false
matrix:
include:
- os: xcode-27
version: linked
- os: xcode-27
version: nightly-latest
name: Unsupported macOS
if: github.triggering_actor != 'dependabot[bot]'
permissions:
contents: read
security-events: read
timeout-minutes: 45
runs-on: ${{ matrix.os }}
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
with:
version: ${{ matrix.version }}
use-all-platform-bundle: 'false'
setup-kotlin: 'true'
- uses: ./../action/init
id: init
with:
languages: swift
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Analyze
uses: ./../action/analyze
with:
category: '/test-codeql-version:${{ matrix.version }}'
env:
CODEQL_ACTION_SKIP_GENERATE_FAILED_SARIF: false
CODEQL_ACTION_SKIP_SARIF_UPLOAD: true
CODEQL_ACTION_TESTING_ENVIRONMENT: codeql-action-pr-checks
CODEQL_ACTION_TEST_MODE: true
unsupported-macos-sarif-checks:
if: github.triggering_actor != 'dependabot[bot]'
needs:
- unsupported-macos
permissions:
contents: read
security-events: read
timeout-minutes: 5
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Download workflow artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: test-mode-sarif-*
path: sarif
- name: List artifact contents
run: ls -laR ./sarif
- name: Check diagnostics appear in SARIF
env:
SARIF_PATH: "./sarif/codeql-failed-run.sarif"
DIAGNOSTIC_ID: "codeql-action/unsupported-traced-analysis-macos"
DIAGNOSTIC_NAME: "Traced analysis is not supported on this version of macOS"
EXPECTED_LOCATIONS: "0"
EXPECTED_MESSAGE: |
Traced analysis is not supported on macOS 27 or above.
run: npx tsx ./pr-checks/check-diagnostics.ts
env:
CODEQL_ACTION_TEST_MODE: true

View File

@@ -57,7 +57,7 @@ jobs:
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: 'npm'

View File

@@ -89,7 +89,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Download all artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
- name: Check expected artifacts exist
run: |
LANGUAGES="cpp csharp go java javascript python"

View File

@@ -82,7 +82,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Download all artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
- name: Check expected artifacts exist
run: |
VERSIONS="stable-v2.20.7 default linked nightly-latest"

View File

@@ -47,7 +47,7 @@ jobs:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0 # ensure we have all tags and can push commits
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: 'npm'

View File

@@ -42,7 +42,7 @@ jobs:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
@@ -110,7 +110,7 @@ jobs:
fi
- name: Set up Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: "npm"
@@ -175,7 +175,7 @@ jobs:
- name: Upload repo size comment
if: steps.fetch-base.outcome == 'success'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: repo-size-comment
path: ${{ runner.temp }}/repo-size/
@@ -223,7 +223,7 @@ jobs:
steps:
- name: Download repo size comment
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: repo-size-comment
path: repo-size-comment

View File

@@ -33,7 +33,7 @@ jobs:
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm

View File

@@ -30,7 +30,7 @@ jobs:
ref: ${{ env.HEAD_REF }}
- name: Set up Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: 'npm'

View File

@@ -41,7 +41,7 @@ jobs:
git config --global user.name "github-actions[bot]"
- name: Set up Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: 'npm'

View File

@@ -26,7 +26,7 @@ jobs:
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: 'npm'

View File

@@ -2,6 +2,10 @@
See the [releases page](https://github.com/github/codeql-action/releases) for the relevant changes to the CodeQL CLI and language packs.
## [UNRELEASED]
No user facing changes.
## 4.39.0 - 15 Oct 2026
- _Breaking change_: Bump the minimum required CodeQL bundle version to 2.20.7. [#4205](https://github.com/github/codeql-action/pull/4205)

1668
lib/entry-points.js generated

File diff suppressed because it is too large Load Diff

4
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "codeql",
"version": "4.39.0",
"version": "4.39.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "codeql",
"version": "4.39.0",
"version": "4.39.1",
"license": "MIT",
"workspaces": [
"pr-checks"

View File

@@ -1,6 +1,6 @@
{
"name": "codeql",
"version": "4.39.0",
"version": "4.39.1",
"private": true,
"description": "CodeQL action",
"scripts": {

View File

@@ -0,0 +1,92 @@
import fs from "fs";
import * as core from "@actions/core";
import * as sarif from "../src/sarif";
const DIAGNOSTIC_ID =
process.env["DIAGNOSTIC_ID"] ?? "lang/diagnostics/example";
const DIAGNOSTIC_NAME = process.env["DIAGNOSTIC_NAME"] ?? "Diagnostic name";
const EXPECTED_MESSAGE = process.env["EXPECTED_MESSAGE"] ?? "Plaintext message";
const EXPECTED_LOCATIONS = Number.parseInt(
process.env["EXPECTED_LOCATIONS"] ?? "1",
);
function checkStatusPageNotification(n: sarif.Notification) {
const expectedMessage = EXPECTED_MESSAGE.trim();
if (!n.message.text?.includes(expectedMessage)) {
if (n.message.text !== undefined) {
core.setFailed(
`Expected the status page diagnostic to contain the message '${expectedMessage}', but found '${n.message.text.replaceAll("\n", "\\n")}'.`,
);
} else {
core.setFailed(
`Expected the status page diagnostic to contain the message '${expectedMessage}', but found nothing.`,
);
}
}
const locationCount = n.locations?.length ?? 0;
if (locationCount !== EXPECTED_LOCATIONS) {
core.setFailed(
`Expected the status page diagnostic to have exactly ${EXPECTED_LOCATIONS} location(s), but found ${locationCount}.`,
);
}
}
const sarifPath = process.env["SARIF_PATH"];
if (sarifPath === undefined) {
throw new Error("The path to a SARIF file must be set in 'SARIF_PATH'.");
}
const log = JSON.parse(fs.readFileSync(sarifPath, "utf8")) as sarif.Log;
const run = log.runs[0];
if (run.invocations === undefined) {
throw new Error("run.invocations is undefined");
}
const toolExecutionNotifications =
run.invocations[0].toolExecutionNotifications;
if (toolExecutionNotifications === undefined) {
throw new Error("toolExecutionNotifications is undefined");
}
const statusPageNotifications = toolExecutionNotifications.filter(
(n) =>
n.descriptor?.id === DIAGNOSTIC_ID && n.properties?.visibility?.statusPage,
);
if (statusPageNotifications.length !== 1) {
core.setFailed(
"Expected exactly one status page reporting descriptor for this diagnostic in the " +
`'runs[].invocations[].toolExecutionNotifications[]' SARIF property, but found ` +
`${statusPageNotifications.length}. All notification reporting descriptors: ` +
`${JSON.stringify(toolExecutionNotifications)}.`,
);
}
checkStatusPageNotification(statusPageNotifications[0]);
const notifications = run.tool.driver.notifications;
if (notifications === undefined) {
throw new Error("notifications is undefined");
}
const diagnosticNotification = notifications.filter(
(n) =>
n.id === DIAGNOSTIC_ID &&
n.name === DIAGNOSTIC_ID &&
n.fullDescription?.text === DIAGNOSTIC_NAME,
);
if (diagnosticNotification.length !== 1) {
core.setFailed(
"Expected exactly one notification for this diagnostic in the " +
`'runs[].tool.driver.notifications[]' SARIF property, but found ` +
`${diagnosticNotification.length}. All notifications: ` +
`${JSON.stringify(notifications)}.`,
);
}
core.info("Finished diagnostic export test");

View File

@@ -46,7 +46,7 @@ steps:
post-processed-sarif-path: "${{ runner.temp }}/post-processed"
- name: Upload SARIF files
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: |
analysis-kinds-${{ matrix.os }}-${{ matrix.version }}-${{ matrix.analysis-kinds }}
@@ -54,7 +54,7 @@ steps:
retention-days: 7
- name: Upload post-processed SARIF
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: |
post-processed-${{ matrix.os }}-${{ matrix.version }}-${{ matrix.analysis-kinds }}

View File

@@ -14,7 +14,7 @@ steps:
output: "${{ runner.temp }}/results"
upload-database: false
- name: Upload SARIF
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: config-export-${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: "${{ runner.temp }}/results/javascript.sarif"

View File

@@ -5,6 +5,7 @@ versions:
- nightly-latest
env:
CODEQL_ACTION_EXPORT_DIAGNOSTICS: true
installNode: true
steps:
- uses: ./../action/init
id: init
@@ -27,58 +28,15 @@ steps:
output: "${{ runner.temp }}/results"
upload-database: false
- name: Upload SARIF
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: diagnostics-export-${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: "${{ runner.temp }}/results/javascript.sarif"
retention-days: 7
- name: Check diagnostics appear in SARIF
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
env:
SARIF_PATH: "${{ runner.temp }}/results/javascript.sarif"
with:
script: |
const fs = require('fs');
function checkStatusPageNotification(n) {
const expectedMessage = 'Plaintext message';
if (n.message.text !== expectedMessage) {
core.setFailed(`Expected the status page diagnostic to have the message '${expectedMessage}', but found '${n.message.text}'.`);
}
if (n.locations.length !== 1) {
core.setFailed(`Expected the status page diagnostic to have exactly 1 location, but found ${n.locations.length}.`);
}
}
const sarif = JSON.parse(fs.readFileSync(process.env['SARIF_PATH'], 'utf8'));
const run = sarif.runs[0];
const toolExecutionNotifications = run.invocations[0].toolExecutionNotifications;
const statusPageNotifications = toolExecutionNotifications.filter(n =>
n.descriptor.id === 'lang/diagnostics/example' && n.properties?.visibility?.statusPage
);
if (statusPageNotifications.length !== 1) {
core.setFailed(
'Expected exactly one status page reporting descriptor for this diagnostic in the ' +
`'runs[].invocations[].toolExecutionNotifications[]' SARIF property, but found ` +
`${statusPageNotifications.length}. All notification reporting descriptors: ` +
`${JSON.stringify(toolExecutionNotifications)}.`
);
}
checkStatusPageNotification(statusPageNotifications[0]);
const notifications = run.tool.driver.notifications;
const diagnosticNotification = notifications.filter(n =>
n.id === 'lang/diagnostics/example' && n.name === 'lang/diagnostics/example' &&
n.fullDescription.text === 'Diagnostic name'
);
if (diagnosticNotification.length !== 1) {
core.setFailed(
'Expected exactly one notification for this diagnostic in the ' +
`'runs[].tool.driver.notifications[]' SARIF property, but found ` +
`${diagnosticNotification.length}. All notifications: ` +
`${JSON.stringify(notifications)}.`
);
}
core.info('Finished diagnostic export test');
run: |
cd ../action
npx tsx ./pr-checks/check-diagnostics.ts

View File

@@ -0,0 +1,67 @@
name: "Diagnostic export without database"
description: "Tests that temporary diagnostics are correctly included in the failed SARIF."
versions:
# Overridden by the FF below.
- linked
installNode: true
env:
# Allow the failed SARIF file to be generated.
CODEQL_ACTION_SKIP_GENERATE_FAILED_SARIF: false
# But don't actually upload it.
CODEQL_ACTION_SKIP_SARIF_UPLOAD: true
# Mark telemetry for this workflow so it can be treated separately.
CODEQL_ACTION_TESTING_ENVIRONMENT: codeql-action-pr-checks
steps:
# We deliberately need to make this step fail before the database is initialised, but after
# some diagnostic has been produced. The simplest way to do that right now is to force the
# use of a nightly release and then fail because we are attempting to analyse Swift on
# a non-macOS runner.
- name: Initialise failing analysis
id: init
uses: ./../action/init
continue-on-error: true
env:
CODEQL_ACTION_FORCE_NIGHTLY: true
with:
languages: swift
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Analyze
uses: ./analyze
# In a real workflow, this step wouldn't run. Since we used `continue-on-error`
# above, we manually disable it with an `if` condition.
if: false
with:
category: "/test-codeql-version:${{ matrix.version }}"
- name: Check diagnostics appear on disk
env:
SARIF_PATH: "${{ runner.temp }}/codeql-action-temp-diagnostics/"
run: |
cd ../action
npx tsx ./pr-checks/diagnostics-without-db.ts
validationJobs:
diagnostics-without-db-validation:
checkout: true
installNode: true
steps:
- name: Download workflow artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: test-mode-sarif-*
path: sarif
- name: List artifact contents
run: ls -laR ./sarif
- name: Check diagnostics appear in SARIF
env:
SARIF_PATH: "./sarif/codeql-failed-run.sarif"
DIAGNOSTIC_ID: "codeql-action/forced-nightly-cli"
DIAGNOSTIC_NAME: "A nightly release of CodeQL was used"
EXPECTED_LOCATIONS: "0"
EXPECTED_MESSAGE: |
GitHub configured this analysis to use a nightly release of CodeQL to allow you to preview changes from an upcoming release.
run: npx tsx ./pr-checks/check-diagnostics.ts

View File

@@ -27,7 +27,7 @@ steps:
with:
output: "${{ runner.temp }}/results"
- name: Upload SARIF
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: with-baseline-information-${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: "${{ runner.temp }}/results/javascript.sarif"

View File

@@ -12,7 +12,7 @@ steps:
with:
output: "${{ runner.temp }}/results"
- name: Upload SARIF
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: ${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: "${{ runner.temp }}/results/javascript.sarif"

View File

@@ -14,7 +14,6 @@ operatingSystems:
- stable-v2.21.4
- stable-v2.22.4
env:
CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES: true
CODEQL_ACTION_RESOLVE_SUPPORTED_LANGUAGES_USING_CLI: true
installGo: true
installDotNet: true

View File

@@ -0,0 +1,54 @@
name: "Unsupported macOS"
description: "Tests that unsupported macOS versions are detected and result in appropriate diagnostics."
versions:
- linked
- nightly-latest
installNode: true
operatingSystems:
- os: macos
runner-image: xcode-27
env:
# Allow the failed SARIF file to be generated.
CODEQL_ACTION_SKIP_GENERATE_FAILED_SARIF: false
# But don't actually upload it.
CODEQL_ACTION_SKIP_SARIF_UPLOAD: true
# Mark telemetry for this workflow so it can be treated separately.
CODEQL_ACTION_TESTING_ENVIRONMENT: codeql-action-pr-checks
steps:
- uses: ./../action/init
id: init
with:
languages: swift
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Analyze
uses: ./../action/analyze
with:
category: "/test-codeql-version:${{ matrix.version }}"
validationJobs:
unsupported-macos-sarif-checks:
checkout: true
installNode: true
steps:
- name: Download workflow artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: test-mode-sarif-*
path: sarif
- name: List artifact contents
run: ls -laR ./sarif
- name: Check diagnostics appear in SARIF
env:
SARIF_PATH: "./sarif/codeql-failed-run.sarif"
DIAGNOSTIC_ID: "codeql-action/unsupported-traced-analysis-macos"
DIAGNOSTIC_NAME: "Traced analysis is not supported on this version of macOS"
EXPECTED_LOCATIONS: "0"
EXPECTED_MESSAGE: |
Traced analysis is not supported on macOS 27 or above.
run: npx tsx ./pr-checks/check-diagnostics.ts

View File

@@ -0,0 +1,48 @@
import * as fs from "fs";
import * as path from "path";
import * as core from "@actions/core";
import type * as diagnostics from "../src/diagnostics";
const DIAGNOSTIC_ID = "codeql-action/forced-nightly-cli";
const sarifDirectory = process.env["SARIF_PATH"];
if (sarifDirectory === undefined) {
throw new Error("SARIF_PATH is undefined.");
}
const paths = fs
.readdirSync(sarifDirectory, {
recursive: true,
encoding: "utf-8",
})
.filter((p) => p.endsWith(".json"));
core.info(`Found ${paths.length} paths in '${sarifDirectory}'.`);
core.info(`Searching for a diagnostic with id '${DIAGNOSTIC_ID}'...`);
let found = false;
for (const diagnosticPath of paths) {
if (path.extname(diagnosticPath) !== ".json") {
core.info(`Skipping ${diagnosticPath}.`);
continue;
}
core.info(`Found ${diagnosticPath}.`);
const contents = JSON.parse(
fs.readFileSync(path.resolve(sarifDirectory, diagnosticPath), "utf-8"),
) as diagnostics.DiagnosticMessage;
if (contents.source.id === DIAGNOSTIC_ID) {
core.info(
`Found '${DIAGNOSTIC_ID}' in '${diagnosticPath}'. Skipping remaining files.`,
);
found = true;
break;
}
}
if (!found) {
throw new Error(`Didn't find '${DIAGNOSTIC_ID}' diagnostic.`);
}

View File

@@ -90,7 +90,7 @@ interface Specification extends JobSpecification {
services?: any;
/** Additional jobs to run after the main PR check job. */
validationJobs?: Record<string, JobSpecification>;
validationJobs?: Record<string, ValidationJobSpecification>;
/** If set, this check is part of a named collection that gets its own caller workflow. */
collection?: string;
@@ -122,6 +122,10 @@ interface JobSpecification {
installYq?: boolean;
}
type ValidationJobSpecification = JobSpecification & {
checkout?: boolean;
};
/** Describes language/framework-specific steps and inputs. */
interface LanguageSetup {
specProperty: keyof JobSpecification;
@@ -215,8 +219,8 @@ const languageSetups: LanguageSetups = {
name: "Install Node.js",
uses: pinnedUses(
"actions/setup-node",
"820762786026740c76f36085b0efc47a31fe5020",
"v7.0.0",
"949feb2413d6458794dcd2491c4babbbce0c15c1",
"v7.1.0",
),
with: {
"node-version": defaultLanguageVersions.javascript,
@@ -462,6 +466,20 @@ function generateJobMatrix(
return matrix;
}
/**
* Gets the `actions/checkout` step.
*/
function getCheckoutStep(): Step {
return {
name: "Check out repository",
uses: pinnedUses(
"actions/checkout",
"3d3c42e5aac5ba805825da76410c181273ba90b1",
"v7.0.1",
),
};
}
/**
* Retrieves setup steps and additional input definitions based on specific languages or frameworks
* that are requested by the `checkSpecification`.
@@ -531,14 +549,7 @@ function generateJob(
// Construct the workflow steps needed for this check.
const steps: Step[] = [
{
name: "Check out repository",
uses: pinnedUses(
"actions/checkout",
"3d3c42e5aac5ba805825da76410c181273ba90b1",
"v7.0.1",
),
},
getCheckoutStep(),
...setupInfo.steps,
{
name: "Prepare test",
@@ -609,13 +620,16 @@ function generateJob(
/** Generates a validation job. */
function generateValidationJob(
specDocument: yaml.Document,
jobSpecification: JobSpecification,
jobSpecification: ValidationJobSpecification,
checkName: string,
name: string,
) {
// Determine which languages or frameworks have to be installed.
const { inputs, steps } = getSetupSteps(jobSpecification);
// Construct a checkout step if requested.
const checkoutStep = jobSpecification.checkout ? [getCheckoutStep()] : [];
// Extract the sequence of steps from the YAML document to persist as much formatting as possible.
const specSteps = specDocument.getIn([
"validationJobs",
@@ -624,7 +638,7 @@ function generateValidationJob(
]) as yaml.YAMLSeq;
// Add the generated steps in front of the ones from the specification.
specSteps.items.unshift(...steps);
specSteps.items.unshift(...checkoutStep, ...steps);
const validationJob: Record<string, any> = {
name: jobSpecification.name,
@@ -635,7 +649,7 @@ function generateValidationJob(
"security-events": "read",
},
"timeout-minutes": 5,
"runs-on": "ubuntu-slim",
"runs-on": "ubuntu-latest",
steps: specSteps,
};

View File

@@ -1,3 +1,5 @@
import * as os from "os";
import * as core from "@actions/core";
import { ActionsEnv, getActionsEnv } from "./actions-util";
@@ -13,6 +15,8 @@ import {
} from "./status-report";
import { getEnv, getErrorMessage, wrapError } from "./util";
export type { Logger } from "./logging";
/** Base state that is available to an Action on startup. */
export interface BaseState {
/** The name of the Action. */
@@ -23,6 +27,8 @@ export interface BaseState {
platform: NodeJS.Platform;
/** The architecture of the host. */
arch: NodeJS.Architecture;
/** The version of the operating system. */
osRelease: string;
}
/** Describes different state features that an Action may have. */
@@ -173,6 +179,7 @@ export async function runInActions(action: Action) {
startedAt,
platform: process.platform,
arch: process.arch,
osRelease: os.release(),
logger,
env,
actions: actionsEnv,

View File

@@ -72,7 +72,7 @@ export const getOptionalInput = function (name: string): string | undefined {
* directory that has been set in `CODEQL_ACTION_TEMP` by e.g. a previous step, or the
* value of `RUNNER_TEMP` otherwise.
*/
export function getTemporaryDirectory(env: Env = getEnv()): string {
export function getTemporaryDirectory(env: ReadOnlyEnv = getEnv()): string {
return (
env.getOptional(EnvVar.TEMP) ?? env.getRequired(ActionsEnvVars.RUNNER_TEMP)
);

View File

@@ -361,6 +361,7 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
uploadResults = await postProcessAndUploadSarif(
logger,
features,
config,
uploadKind,
checkoutPath,
outputDir,

View File

@@ -8,7 +8,14 @@ import * as actionsUtil from "./actions-util";
import * as api from "./api-client";
import { DO_NOT_RETRY_STATUSES } from "./api-client";
import { ActionsEnvVars, RegistryProxyVars } from "./environment";
import { callee, getTestEnv, setupTests } from "./testing-utils";
import { GITHUB_DOTCOM_API_URL, GITHUB_DOTCOM_URL } from "./github";
import {
callee,
getTestEnv,
RecordingLogger,
SAMPLE_DOTCOM_API_DETAILS,
setupTests,
} from "./testing-utils";
import * as util from "./util";
setupTests(test);
@@ -284,3 +291,87 @@ test("makeProxyRequestOptions - returns fetch with custom proxy", async (t) => {
githubUtils.defaults.request,
);
});
test.serial(
"getApiClientWithExternalAuth constructs clients with or without tokens",
(t) => {
t.notThrows(() => {
api.getApiClientWithExternalAuth(SAMPLE_DOTCOM_API_DETAILS);
api.getApiClientWithExternalAuth({
...SAMPLE_DOTCOM_API_DETAILS,
auth: undefined,
});
});
},
);
test("getAuthorizationHeaderFor - returns undefined if there's no token", (t) => {
const logger = new RecordingLogger();
t.is(
api.getAuthorizationHeaderFor(
logger,
{ ...SAMPLE_DOTCOM_API_DETAILS, auth: undefined },
`${GITHUB_DOTCOM_URL}/github`,
),
undefined,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
{ ...SAMPLE_DOTCOM_API_DETAILS, auth: "" },
`${GITHUB_DOTCOM_URL}/github`,
),
undefined,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
{ ...SAMPLE_DOTCOM_API_DETAILS, auth: " " },
`${GITHUB_DOTCOM_URL}/github`,
),
undefined,
);
});
test("getAuthorizationHeaderFor - returns header value with token if url is allowed", (t) => {
const logger = new RecordingLogger();
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`${GITHUB_DOTCOM_URL}/github`,
),
`token ${SAMPLE_DOTCOM_API_DETAILS.auth}`,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`${GITHUB_DOTCOM_API_URL}/github`,
),
`token ${SAMPLE_DOTCOM_API_DETAILS.auth}`,
);
});
test("getAuthorizationHeaderFor - returns undefined if url is not allowed", (t) => {
const logger = new RecordingLogger();
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`https://localhost/github`,
),
undefined,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`https://api.localhost/github`,
),
undefined,
);
});

View File

@@ -1,5 +1,6 @@
import * as core from "@actions/core";
import * as githubUtils from "@actions/github/lib/utils";
import type { OctokitOptions } from "@octokit/core";
import * as retry from "@octokit/plugin-retry";
import { RequestRequestOptions } from "@octokit/types";
import {
@@ -12,26 +13,32 @@ import {
import type { ActionState } from "./action-common";
import { getActionVersion, getRequiredInput } from "./actions-util";
import {
ActionsEnvVars,
EnvVar,
ReadOnlyEnv,
RegistryProxyVars,
getEnv,
exportEnvVar,
} from "./environment";
import {
getCurrentInstance,
GITHUB_DOTCOM_URL,
parseGitHubUrl,
type GitHubApiCombinedDetails,
type GitHubApiDetails,
} from "./github";
import { Logger } from "./logging";
import { getRepositoryNwo, RepositoryNwo } from "./repository";
import {
asHTTPError,
ConfigurationError,
getRequiredEnvParam,
GITHUB_DOTCOM_URL,
GitHubVariant,
GitHubVersion,
parseGitHubUrl,
parseMatrixInput,
} from "./util";
export type { GitHubApiDetails, GitHubApiCombinedDetails } from "./github";
const GITHUB_ENTERPRISE_VERSION_HEADER = "x-github-enterprise-version";
/**
@@ -43,21 +50,6 @@ const GITHUB_ENTERPRISE_VERSION_HEADER = "x-github-enterprise-version";
*/
export const DO_NOT_RETRY_STATUSES = [400, 410, 422, 451];
export type GitHubApiCombinedDetails = GitHubApiDetails &
GitHubApiExternalRepoDetails;
export interface GitHubApiDetails {
auth: string;
url: string;
apiURL: string | undefined;
}
export interface GitHubApiExternalRepoDetails {
externalRepoAuth?: string;
url: string;
apiURL: string | undefined;
}
/**
* Gets the configuration for the private registry authentication proxy,
* if it is available in the environment.
@@ -139,37 +131,60 @@ function createApiClientWithDetails(
{ allowExternal = false, proxy = undefined }: CreateApiClientOptions = {},
): ApiClient {
const auth =
(allowExternal && apiDetails.externalRepoAuth) || apiDetails.auth;
allowExternal && apiDetails.externalRepoAuth !== undefined
? apiDetails.externalRepoAuth
: apiDetails.auth;
const retryingOctokit = githubUtils.GitHub.plugin(retry.retry);
const requestOptions = makeProxyRequestOptions(proxy);
return new retryingOctokit(
githubUtils.getOctokitOptions(auth, {
baseUrl: apiDetails.apiURL,
userAgent: `CodeQL-Action/${getActionVersion()}`,
log: {
debug: core.debug,
info: core.info,
warn: core.warning,
error: core.error,
},
request: requestOptions,
retry: {
doNotRetry: DO_NOT_RETRY_STATUSES,
},
}),
);
const opts: OctokitOptions = {
auth:
auth !== undefined && auth.trim() !== "" ? `token ${auth}` : undefined,
baseUrl: apiDetails.apiURL,
userAgent: `CodeQL-Action/${getActionVersion()}`,
log: {
debug: core.debug,
info: core.info,
warn: core.warning,
error: core.error,
},
request: requestOptions,
retry: {
doNotRetry: DO_NOT_RETRY_STATUSES,
},
};
return new retryingOctokit(opts);
}
/**
* Gets a `GitHubApiDetails` object describing the GitHub instance we are currently running on.
*
* @param env The environment variables to read.
*/
export function getApiDetails(env: ReadOnlyEnv = getEnv()): GitHubApiDetails {
return {
auth: getRequiredInput("token"),
url: env.getRequired(ActionsEnvVars.GITHUB_SERVER_URL),
apiURL: env.getRequired(ActionsEnvVars.GITHUB_API_URL),
...getCurrentInstance(env),
};
}
export function getApiClient(env: ReadOnlyEnv = getEnv()) {
return createApiClientWithDetails(getApiDetails(env));
/**
* Constructs an {@link ApiClient} using {@link apiDetails}.
*
* If {@link apiDetails} is not specified, then the GitHub instance URLs and auth token
* are taken from {@link env}.
*
* If {@link env} is not specified, then the process environment is used.
*
* @param env The environment to obtain the GitHub instance URLs from.
* @param apiDetails The {@link GitHubApiDetails} to construct the {@link ApiClient} for.
*/
export function getApiClient(
env: ReadOnlyEnv = getEnv(),
apiDetails?: GitHubApiDetails,
) {
return createApiClientWithDetails(apiDetails ?? getApiDetails(env));
}
export function getApiClientWithExternalAuth(
@@ -194,6 +209,12 @@ export function getAuthorizationHeaderFor(
apiDetails: GitHubApiDetails,
url: string,
): string | undefined {
// We don't have to check the `url` if we don't have a token to begin with.
const token = apiDetails.auth?.trim();
if (token === undefined || token === "") {
return undefined;
}
// We only want to provide an authorization header if we are downloading
// from the same GitHub instance the Action is running on.
// This avoids leaking Enterprise tokens to dotcom.
@@ -202,7 +223,7 @@ export function getAuthorizationHeaderFor(
(apiDetails.apiURL && url.startsWith(`${apiDetails.apiURL}/`))
) {
logger.debug(`Providing an authorization token.`);
return `token ${apiDetails.auth}`;
return `token ${token}`;
}
logger.debug(`Not using an authorization token.`);

View File

@@ -2,7 +2,7 @@ import * as fs from "fs";
import path from "path";
import { getTemporaryDirectory } from "../actions-util";
import { Env } from "../environment";
import { ReadOnlyEnv } from "../environment";
import * as json from "../json";
import { Logger } from "../logging";
@@ -51,7 +51,7 @@ export function resetCachedCodeQlVersion(): void {
* Returns the path to the temporary file that backs the
* on-disk cache of CLI responses between workflow steps.
*/
export function getCommandCacheFilePath(env: Env): string {
export function getCommandCacheFilePath(env: ReadOnlyEnv): string {
return path.join(getTemporaryDirectory(env), COMMAND_CACHE_FILENAME);
}

View File

@@ -126,7 +126,7 @@ function mockReleaseApi({
});
}
function mockApiDetails(apiDetails: GitHubApiDetails) {
function mockApiDetails(apiDetails: Required<GitHubApiDetails>) {
// This is a workaround to mock `api.getApiDetails()` since it doesn't seem to be possible to
// mock this directly. The difficulty is that `getApiDetails()` is called locally in
// `api-client.ts`, but `sinon.stub(api, "getApiDetails")` only affects calls to

View File

@@ -18,7 +18,7 @@ import { CliError, wrapCliConfigurationError } from "./cli-errors";
import type { ActionConfig } from "./config/action-config";
import { appendExtraQueryExclusions, type Config } from "./config-utils";
import { DocUrl } from "./doc-url";
import { EnvVar, getEnv, exportEnvVar } from "./environment";
import { EnvVar, getEnv, exportEnvVar, Env, ReadOnlyEnv } from "./environment";
import {
CodeQLDefaultVersionInfo,
Feature,
@@ -199,6 +199,7 @@ export interface CodeQL {
sarifFile: string,
automationDetailsId: string | undefined,
config: ActionConfig<"tempDir">,
diagnosticDir: string | undefined,
): Promise<void>;
/** Get the location of an extractor for the specified language. */
resolveExtractor(language: Language): Promise<string>;
@@ -494,8 +495,9 @@ export function createStubCodeQL(partialCodeql: Partial<CodeQL>): CodeQL {
export async function getCodeQLForTesting(
cmd = "codeql-for-testing",
logger: Logger = getRunnerLogger(true),
env: Env = getEnv(),
): Promise<CodeQL> {
return getCodeQLForCmd(logger, cmd, false);
return getCodeQLForCmd(logger, cmd, false, env);
}
/**
@@ -510,13 +512,14 @@ async function getCodeQLForCmd(
logger: Logger,
cmd: string,
checkVersion: boolean,
env: Env = getEnv(),
): Promise<CodeQL> {
const codeql: CodeQL = {
getPath() {
return cmd;
},
async getVersion() {
const cacheFilePath = outputCache.getCommandCacheFilePath(getEnv());
const cacheFilePath = outputCache.getCommandCacheFilePath(env);
let result = outputCache.getCachedCodeQlVersion(
logger,
cacheFilePath,
@@ -642,7 +645,7 @@ async function getCodeQLForCmd(
}
},
async runAutobuild(config: Config, language: Language) {
applyAutobuildAzurePipelinesTimeoutFix();
applyAutobuildAzurePipelinesTimeoutFix(env);
const autobuildCmd = path.join(
await this.resolveExtractor(language),
@@ -652,8 +655,11 @@ async function getCodeQLForCmd(
// Bump the verbosity of the autobuild command if we're in debug mode
if (config.debugMode) {
process.env[EnvVar.CLI_VERBOSITY] =
process.env[EnvVar.CLI_VERBOSITY] || EXTRACTION_DEBUG_MODE_VERBOSITY;
env.set(
EnvVar.CLI_VERBOSITY,
env.getOptional(EnvVar.CLI_VERBOSITY) ??
EXTRACTION_DEBUG_MODE_VERBOSITY,
);
}
// On macOS, System Integrity Protection (SIP) typically interferes with
@@ -685,7 +691,7 @@ async function getCodeQLForCmd(
},
async extractUsingBuildMode(config: Config, language: Language) {
if (config.buildMode === BuildMode.Autobuild) {
applyAutobuildAzurePipelinesTimeoutFix();
applyAutobuildAzurePipelinesTimeoutFix(env);
}
try {
await runCli(cmd, [
@@ -817,7 +823,7 @@ async function getCodeQLForCmd(
"--sarif-group-rules-by-pack",
"--sarif-include-query-help=always",
"--sublanguage-file-coverage",
...(await getJobRunUuidSarifOptions()),
...(await getJobRunUuidSarifOptions(env)),
...getExtraOptionsFromEnv(["database", "interpret-results"]),
];
if (sarifRunPropertyFlag !== undefined) {
@@ -915,15 +921,32 @@ async function getCodeQLForCmd(
sarifFile: string,
automationDetailsId: string | undefined,
config: ActionConfig<"tempDir">,
diagnosticDir: string | undefined,
): Promise<void> {
// We may be exporting diagnostics for a failed SARIF, in which case the configuration
// file may not have been written to disk and there may not be a database. The CLI
// will produce a SARIF for diagnostics from `diagnosticDir` without it. Therefore,
// if `diagnosticDir` is set, we check whether `configPath` exists and only provide
// it to the CLI if it does. Although not needed, the CLI will produce an error if it
// is provided with an argument to `--sarif-codescanning-config` that does not exist.
let configPath: string | undefined =
getGeneratedCodeScanningConfigPath(config);
if (diagnosticDir !== undefined && !fs.existsSync(configPath)) {
configPath = undefined;
}
const args = [
"diagnostics",
"export",
"--format=sarif-latest",
`--output=${sarifFile}`,
`--sarif-codescanning-config=${getGeneratedCodeScanningConfigPath(
config,
)}`,
...(configPath !== undefined
? [`--sarif-codescanning-config=${configPath}`]
: []),
...(diagnosticDir !== undefined
? [`--diagnostic-dir=${diagnosticDir}`]
: []),
...getExtraOptionsFromEnv(["diagnostics", "export"]),
];
if (automationDetailsId !== undefined) {
@@ -1037,7 +1060,7 @@ async function getCodeQLForCmd(
);
} else if (
checkVersion &&
process.env[EnvVar.SUPPRESS_DEPRECATED_SOON_WARNING] !== "true" &&
env.getOptional(EnvVar.SUPPRESS_DEPRECATED_SOON_WARNING) !== "true" &&
!(await util.codeQlVersionAtLeast(codeql, CODEQL_NEXT_MINIMUM_VERSION))
) {
const result = await codeql.getVersion();
@@ -1259,17 +1282,20 @@ function getExtractionVerbosityArguments(
* Without the fix, long build processes will timeout when pulling down Java packages
* https://developercommunity.visualstudio.com/content/problem/292284/maven-hosted-agent-connection-timeout.html
*/
function applyAutobuildAzurePipelinesTimeoutFix() {
const javaToolOptions = process.env["JAVA_TOOL_OPTIONS"] || "";
process.env["JAVA_TOOL_OPTIONS"] = [
...javaToolOptions.split(/\s+/),
"-Dhttp.keepAlive=false",
"-Dmaven.wagon.http.pool=false",
].join(" ");
function applyAutobuildAzurePipelinesTimeoutFix(env: Env) {
const javaToolOptions = env.getOptional("JAVA_TOOL_OPTIONS") ?? "";
env.set(
"JAVA_TOOL_OPTIONS",
[
...javaToolOptions.split(/\s+/),
"-Dhttp.keepAlive=false",
"-Dmaven.wagon.http.pool=false",
].join(" "),
);
}
async function getJobRunUuidSarifOptions() {
const jobRunUuid = process.env[EnvVar.JOB_RUN_UUID];
async function getJobRunUuidSarifOptions(env: ReadOnlyEnv) {
const jobRunUuid = env.getOptional(EnvVar.JOB_RUN_UUID);
return jobRunUuid ? [`--sarif-run-property=jobRunUuid=${jobRunUuid}`] : [];
}

View File

@@ -9,8 +9,9 @@ import {
LoggedMessage,
makeMacro,
RecordingLogger,
setupActionsVars,
} from "../testing-utils";
import { ConfigurationError, prettyPrintPack } from "../util";
import { ConfigurationError, prettyPrintPack, withTmpDir } from "../util";
import * as dbConfig from "./db-config";
@@ -562,69 +563,84 @@ test("mergeDefaultSetupAndUserConfigs - keeps other properties from user-supplie
t.deepEqual(result, configFile);
});
test("mergeDefaultSetupAndUserConfigs - ignores, but warns about, unknown keys from Default Setup", async (t) => {
const logger = new RecordingLogger();
const configFile: dbConfig.UserConfig = {
"query-filters": [{ exclude: { a: "b" } }],
"paths-ignore": ["path"],
};
test.serial(
"mergeDefaultSetupAndUserConfigs - ignores, but warns about, unknown keys from Default Setup",
async (t) => {
const logger = new RecordingLogger();
const configFile: dbConfig.UserConfig = {
"query-filters": [{ exclude: { a: "b" } }],
"paths-ignore": ["path"],
};
const result = dbConfig.mergeDefaultSetupAndUserConfigs(
logger,
{
"default-setup": {
borg: [],
org: {
unknown: "foo",
"model-packs": [],
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const result = dbConfig.mergeDefaultSetupAndUserConfigs(
logger,
{
"default-setup": {
borg: [],
org: {
unknown: "foo",
"model-packs": [],
},
} as unknown as dbConfig.DefaultSetupConfig,
"paths-ignore": ["other-path"],
},
} as unknown as dbConfig.DefaultSetupConfig,
"paths-ignore": ["other-path"],
},
configFile,
);
configFile,
);
t.deepEqual(result, {
...configFile,
"default-setup": { org: { "model-packs": [] } },
});
t.deepEqual(result, {
...configFile,
"default-setup": { org: { "model-packs": [] } },
});
const expectedUnrecognisedKeys = [
".default-setup.org.unknown",
".default-setup.borg",
".paths-ignore",
].join(", ");
checkExpectedLogMessages(t, logger.messages, [
`Unrecognised keys in Default Setup configuration: ${expectedUnrecognisedKeys}`,
]);
});
const expectedUnrecognisedKeys = [
".default-setup.org.unknown",
".default-setup.borg",
".paths-ignore",
].join(", ");
checkExpectedLogMessages(t, logger.messages, [
`Unrecognised keys in Default Setup configuration: ${expectedUnrecognisedKeys}`,
]);
});
},
);
test("mergeDefaultSetupAndUserConfigs - warns about invalid keys from Default Setup", async (t) => {
const logger = new RecordingLogger();
const configFile: dbConfig.UserConfig = {};
test.serial(
"mergeDefaultSetupAndUserConfigs - warns about invalid keys from Default Setup",
async (t) => {
const logger = new RecordingLogger();
const configFile: dbConfig.UserConfig = {};
const result = dbConfig.mergeDefaultSetupAndUserConfigs(
logger,
{
"default-setup": {
org: {
"model-packs": [123],
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const result = dbConfig.mergeDefaultSetupAndUserConfigs(
logger,
{
"default-setup": {
org: {
"model-packs": [123],
},
} as unknown as dbConfig.DefaultSetupConfig,
},
} as unknown as dbConfig.DefaultSetupConfig,
},
configFile,
);
configFile,
);
t.deepEqual(result, {
...configFile,
"default-setup": { org: { "model-packs": [123] } },
});
t.deepEqual(result, {
...configFile,
"default-setup": { org: { "model-packs": [123] } },
});
const expectedInvalidKeys = [".default-setup.org.model-packs[0]"].join(", ");
checkExpectedLogMessages(t, logger.messages, [
`Invalid keys in Default Setup configuration: ${expectedInvalidKeys}`,
]);
});
const expectedInvalidKeys = [".default-setup.org.model-packs[0]"].join(
", ",
);
checkExpectedLogMessages(t, logger.messages, [
`Invalid keys in Default Setup configuration: ${expectedInvalidKeys}`,
]);
});
},
);
/** Parses `contents` as a configuration without validating it. */
function parseUnvalidatedConfig(contents: string): dbConfig.UserConfig {

View File

@@ -1,4 +1,5 @@
import * as fs from "fs";
import fs from "fs";
import path from "path";
import * as github from "@actions/github";
import test from "ava";
@@ -10,7 +11,10 @@ import { GitHubApiDetails } from "./api-client";
import * as apiClient from "./api-client";
import { createStubCodeQL } from "./codeql";
import { Config } from "./config-utils";
import { cleanupAndUploadDatabases } from "./database-upload";
import {
cleanupAndUploadDatabases,
uploadBundledDatabase,
} from "./database-upload";
import { Feature } from "./feature-flags";
import * as gitUtils from "./git-utils";
import { BuiltInLanguage } from "./languages";
@@ -23,6 +27,7 @@ import {
getTestEnv,
initAllState,
RecordingLogger,
SAMPLE_DOTCOM_API_DETAILS,
setupActionsVars,
setupTests,
} from "./testing-utils";
@@ -550,3 +555,86 @@ test.serial(
});
},
);
test.serial(
"uploadBundledDatabase provides expected request arguments",
async (t) =>
withTmpDir(async (tmpDir) => {
// Write some file that `uploadBundledDatabase` can use for its `fs.createReadStream(bundledDb)` call
// so we don't have to stub that or create a fake `ReadStream`.
const dbPath = path.join(tmpDir, "db");
const content = JSON.stringify({});
fs.writeFileSync(dbPath, content);
// A simple stub for `fetch` that satisfies Octokit when it inspects the response.
const fetchStub = sinon.stub().resolves({
status: 200,
headers: new Headers(),
text: async () => "",
});
// Construct a real `ApiClient`.
const client = apiClient.getApiClient(
getTestEnv(),
SAMPLE_DOTCOM_API_DETAILS,
);
// Then wrap the real `request` method in a stub that calls it with our stubbed `fetch`.
// This ensures that we have a real `ApiClient` with a mostly real `request`, but which
// does not perform any actual requests.
const request = client.request;
sinon.stub(client, "request").callsFake(async (route, options) =>
request(route, {
...options,
request: { ...options?.request, fetch: fetchStub },
}),
);
// Stub `getApiClient` to return our modified `ApiClient`.
const getApiClient = sinon
.stub(apiClient, "getApiClient")
.returns(client);
// Calling `uploadBundledDatabase` shouldn't throw.
const nwo: RepositoryNwo = { owner: "test", repo: "foo" };
const language: BuiltInLanguage = BuiltInLanguage.java;
await t.notThrowsAsync(async () => {
await uploadBundledDatabase(
nwo,
language,
"123",
dbPath,
content.length,
SAMPLE_DOTCOM_API_DETAILS,
);
});
// Check that our stubs were called and that they received the expected arguments.
t.is(getApiClient.callCount, 1);
t.is(fetchStub.callCount, 1);
const args = fetchStub.args[0];
t.is(args.length, 2);
const url = args[0];
t.is(
url,
`https://uploads.github.com/repos/${nwo.owner}/${nwo.repo}/code-scanning/codeql/databases/${language}?name=${language}-database&commit_oid=123`,
);
const options = args[1];
t.like(options, {
// Going via `new Headers` and `Object.fromEntries` ensures we get the same cases
// for the header keys as we are given.
headers: Object.fromEntries(
new Headers({
authorization: `token ${SAMPLE_DOTCOM_API_DETAILS.auth}`,
"Content-Type": "application/zip",
"Content-Length": content.length.toString(),
}).entries(),
),
method: "POST",
} satisfies RequestInit);
}),
);

View File

@@ -12,11 +12,12 @@ import { type CodeQL } from "./codeql";
import { Config } from "./config-utils";
import { Feature } from "./feature-flags";
import * as gitUtils from "./git-utils";
import { parseGitHubUrl } from "./github";
import { Logger, withGroupAsync } from "./logging";
import { OverlayDatabaseMode } from "./overlay/overlay-database-mode";
import { RepositoryNwo } from "./repository";
import * as util from "./util";
import { asHTTPError, bundleDb, CleanupLevel, parseGitHubUrl } from "./util";
import { asHTTPError, bundleDb, CleanupLevel } from "./util";
/** Information about a database upload. */
export interface DatabaseUploadResult {
@@ -258,7 +259,7 @@ async function recordClearCleanupSizes(
*
* @returns the duration of the upload in milliseconds
*/
async function uploadBundledDatabase(
export async function uploadBundledDatabase(
repositoryNwo: RepositoryNwo,
language: string,
commitOid: string,
@@ -266,7 +267,10 @@ async function uploadBundledDatabase(
bundledDbSize: number,
apiDetails: GitHubApiDetails,
): Promise<number> {
const client = getApiClient();
// Ensure that the `apiDetails` that `getApiClient` uses match the ones we were given here,
// because we use `apiDetails.auth` below which must be used with the same instance that
// `apiDetails` refers to.
const client = getApiClient(util.getEnv(), apiDetails);
const uploadsUrl = new URL(parseGitHubUrl(apiDetails.url));
uploadsUrl.hostname = `uploads.${uploadsUrl.hostname}`;
@@ -292,7 +296,6 @@ async function uploadBundledDatabase(
commit_oid: commitOid,
data: bundledDbReadStream,
headers: {
authorization: `token ${apiDetails.auth}`,
"Content-Type": "application/zip",
"Content-Length": bundledDbSize,
},

111
src/diagnostics.test.ts Normal file
View File

@@ -0,0 +1,111 @@
import * as fs from "fs";
import path from "path";
import test, { type ExecutionContext } from "ava";
import * as diagnostics from "./diagnostics";
import { BuiltInLanguage } from "./languages";
import {
checkExpectedLogMessages,
createTestConfig,
initAllState,
RecordingLogger,
setupTests,
} from "./testing-utils";
import { withTmpDir } from "./util";
setupTests(test);
const id = "codeql-action/test-diagnostic";
const name = "Test title";
test("makeDiagnostic - adds expected properties", (t) => {
const diagnostic = diagnostics.makeDiagnostic(id, name, {});
t.truthy(diagnostic.timestamp);
t.is(diagnostic.source.id, id);
t.is(diagnostic.source.name, name);
});
/**
* Diagnostic files that are written to disk don't have deterministic names.
* This function recursively finds all `.json` files at {@link rootPath},
* asserts that there is exactly one, and that it matches {@link expected}.
*
* @param rootPath The path to start searching from.
*/
function assertSingleDiagnosticIn(
t: ExecutionContext<unknown>,
rootPath: string,
expected: diagnostics.DiagnosticMessage,
) {
// Find all `.json` files in `rootPath` and check that there is exactly one.
const filePaths = fs
.readdirSync(rootPath, {
recursive: true,
encoding: "utf-8",
})
.filter((p) => p.endsWith(".json"));
t.is(filePaths.length, 1);
// Check that the written diagnostic matches the one we added.
const contents = fs.readFileSync(
path.resolve(rootPath, filePaths[0]),
"utf-8",
);
const readDiagnostic = JSON.parse(contents) as diagnostics.DiagnosticMessage;
t.deepEqual(readDiagnostic, expected);
}
test("addDiagnostic writes temporary diagnostics and flushDiagnostics moves them to the database", async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const tempDiagDir = path.resolve(tmpDir, "temp");
const testDbDir = path.resolve(tmpDir, "test", "database");
const diagnostic = diagnostics.makeDiagnostic(id, name, {});
// Write the diagnostic. The path provided for `dbLocation` doesn't exist,
// so the diagnostic will be written to a temporary location.
t.notThrows(() => {
diagnostics.addDiagnostic(
createTestConfig({ tempDir: tempDiagDir, dbLocation: testDbDir }),
BuiltInLanguage.actions,
diagnostic,
logger,
);
});
// Check that `tempDiagDir` was created.
t.true(fs.existsSync(tempDiagDir));
// Check that the diagnostic exists under `tempDiagDir`.
assertSingleDiagnosticIn(t, tempDiagDir, diagnostic);
// Flush the diagnostics. This will cause the diagnostic we wrote previously to
// be moved to a location under `dbLocation`.
t.notThrows(() => {
diagnostics.flushDiagnostics(
initAllState({ logger }),
createTestConfig({
tempDir: tempDiagDir,
dbLocation: testDbDir,
}),
);
});
// Check that `testDbDir` was created.
t.true(fs.existsSync(testDbDir));
// Check that the diagnostic exists under `testDbDir`.
assertSingleDiagnosticIn(t, testDbDir, diagnostic);
// Check that we logged that the database directory doesn't exist yet
// and that we moved one diagnostic.
checkExpectedLogMessages(t, logger.messages, [
`but the database at ${path.join(testDbDir, "actions")} does not exist yet`,
"Moving 1 diagnostic(s) to their respective databases.",
]);
}));

View File

@@ -1,10 +1,16 @@
import { existsSync, mkdirSync, writeFileSync } from "fs";
import * as fs from "fs";
import path from "path";
import type { Config } from "./config-utils";
import type { ActionState } from "./action-common";
import { getTemporaryDirectory } from "./actions-util";
import type { ActionConfig } from "./config/action-config";
import { Language } from "./languages";
import { getActionsLogger } from "./logging";
import { getCodeQLDatabasePath } from "./util";
import { getActionsLogger, type Logger } from "./logging";
import {
getCodeQLDatabasePath,
getErrorMessage,
isErrnoException,
} from "./util";
/**
* Known tags for diagnostics. There is currently only "internal-error",
@@ -84,22 +90,16 @@ export type DiagnosticMessage = DiagnosticMessageOptions & {
source: DiagnosticSource;
};
/** Represents a diagnostic message that has not yet been written to the database. */
interface UnwrittenDiagnostic {
/** The diagnostic message that has not yet been written. */
diagnostic: DiagnosticMessage;
/** The language the diagnostic is for. */
language: Language;
/** Represents a diagnostic message that has not yet been saved to the database. */
interface TemporaryDiagnostic {
/** The path to which the diagnostic has temporarily been written to. */
path: string;
/** The language the diagnostic is for, if any. */
language?: Language;
}
/** A list of diagnostics which have not yet been written to disk. */
let unwrittenDiagnostics: UnwrittenDiagnostic[] = [];
/**
* A list of diagnostics which have not yet been written to disk,
* and where the language does not matter.
*/
let unwrittenDefaultLanguageDiagnostics: DiagnosticMessage[] = [];
/** A list of diagnostics which have not yet been saved to the database. */
let temporaryDiagnostics: TemporaryDiagnostic[] = [];
/**
* Counter used to generate a unique suffix for each diagnostic filename, so that
@@ -128,6 +128,29 @@ export function makeDiagnostic(
};
}
/**
* Gets a path relative to {@link tmpDir} where diagnostics for {@link language}
* can temporarily be stored at before the database is initialised. If {@link language}
* is `undefined`, then the common base path for all languages relative to {@link tmpDir}
* is returned.
*/
export function getTempDiagnosticPath(tmpDir: string, language?: Language) {
const root = path.join(tmpDir, "codeql-action-temp-diagnostics");
return getDiagnosticsPath(root, language);
}
/**
* Gets a path relative to {@link basePath} where diagnostics for {@link language}
* should be stored at. If {@link language} is `undefined`, then {@link basePath}
* is returned instead.
*/
export function getDiagnosticsPath(basePath: string, language?: Language) {
if (language !== undefined) {
return path.resolve(basePath, language, "diagnostic", "codeql-action");
}
return basePath;
}
/**
* Adds the given diagnostic to the database. If the database does not yet exist,
* the diagnostic will be written to it once it has been created.
@@ -137,32 +160,34 @@ export function makeDiagnostic(
* @param diagnostic The diagnostic message to add to the database.
*/
export function addDiagnostic(
config: Config,
config: ActionConfig<"dbLocation" | "tempDir">,
language: Language,
diagnostic: DiagnosticMessage,
logger: Logger = getActionsLogger(),
) {
const logger = getActionsLogger();
const databasePath = language
? getCodeQLDatabasePath(config, language)
: config.dbLocation;
// Check that the database exists before writing to it. If the database does not yet exist,
// store the diagnostic in memory and write it later.
if (existsSync(databasePath)) {
writeDiagnostic(config, language, diagnostic);
if (fs.existsSync(databasePath)) {
writeDiagnostic({ logger }, config, language, diagnostic);
} else {
logger.debug(
`Writing a diagnostic for ${language}, but the database at ${databasePath} does not exist yet.`,
);
unwrittenDiagnostics.push({ diagnostic, language });
// Write the diagnostic to a temporary location.
writeTemporaryDiagnosticFile(logger, config.tempDir, language, diagnostic);
}
}
/** Adds a diagnostic that is not specific to any language. */
export function addNoLanguageDiagnostic(
config: Config | undefined,
config: ActionConfig<"languages" | "dbLocation" | "tempDir"> | undefined,
diagnostic: DiagnosticMessage,
logger: Logger = getActionsLogger(),
) {
if (config !== undefined) {
addDiagnostic(
@@ -173,10 +198,82 @@ export function addNoLanguageDiagnostic(
diagnostic,
);
} else {
unwrittenDefaultLanguageDiagnostics.push(diagnostic);
logger.debug(
`Writing a diagnostic not associated with any language, but we have no configuration to tell us where the database is yet.`,
);
// Write the diagnostic to a temporary location.
writeTemporaryDiagnosticFile(
logger,
getTemporaryDirectory(),
undefined,
diagnostic,
);
}
}
/**
* Writes {@link diagnostic} for an optional {@link language} to a subdirectory of {@link tempDir}
* and tracks it as a diagnostic that was written to a temporary location.
*/
function writeTemporaryDiagnosticFile(
logger: Logger,
tempDir: string,
language: Language | undefined,
diagnostic: DiagnosticMessage,
) {
const tempDiagnosticsPath = getTempDiagnosticPath(tempDir, language);
try {
// Write the diagnostic to a temporary location.
const diagnosticPath = writeDiagnosticFile(tempDiagnosticsPath, diagnostic);
// Track the temporary file.
temporaryDiagnostics.push({ path: diagnosticPath, language });
} catch (err) {
logger.warning(
`Unable to write temporary diagnostic to '${tempDiagnosticsPath}': ${getErrorMessage(err)}\n\n${JSON.stringify(diagnostic)}`,
);
}
}
/**
* Writes {@link diagnostic} to a file in {@link diagnosticsPath}.
*/
function writeDiagnosticFile(
diagnosticsPath: string,
diagnostic: DiagnosticMessage,
): string {
// Create the directory if it doesn't exist yet.
fs.mkdirSync(diagnosticsPath, { recursive: true });
// Include a monotonically increasing suffix to avoid filename collisions
// between diagnostics produced within the same millisecond.
const uniqueSuffix = (diagnosticCounter++).toString();
// We should only need to remove colons, but to be defensive, only allow a restricted set of
// characters.
const sanitizedTimestamp = diagnostic.timestamp.replace(
/[^a-zA-Z0-9.-]/g,
"",
);
const jsonPath = path.resolve(
diagnosticsPath,
`codeql-action-${sanitizedTimestamp}-${uniqueSuffix}.json`,
);
fs.writeFileSync(jsonPath, JSON.stringify(diagnostic));
return jsonPath;
}
/** Gets the path where diagnostics for {@link language} should be stored in the database. */
export function getDatabaseDiagnosticsPath(
config: ActionConfig<"dbLocation">,
language: Language | undefined,
) {
return getDiagnosticsPath(config.dbLocation, language);
}
/**
* Writes the given diagnostic to the database.
*
@@ -185,78 +282,85 @@ export function addNoLanguageDiagnostic(
* @param diagnostic The diagnostic message to add to the database.
*/
function writeDiagnostic(
config: Config,
action: ActionState<["Logger"]>,
config: ActionConfig<"dbLocation">,
language: Language | undefined,
diagnostic: DiagnosticMessage,
) {
const logger = getActionsLogger();
const databasePath = language
? getCodeQLDatabasePath(config, language)
: config.dbLocation;
const diagnosticsPath = path.resolve(
databasePath,
"diagnostic",
"codeql-action",
);
const diagnosticsPath = getDatabaseDiagnosticsPath(config, language);
try {
// Create the directory if it doesn't exist yet.
mkdirSync(diagnosticsPath, { recursive: true });
// Include a monotonically increasing suffix to avoid filename collisions
// between diagnostics produced within the same millisecond.
const uniqueSuffix = (diagnosticCounter++).toString();
// We should only need to remove colons, but to be defensive, only allow a restricted set of
// characters.
const sanitizedTimestamp = diagnostic.timestamp.replace(
/[^a-zA-Z0-9.-]/g,
"",
);
const jsonPath = path.resolve(
diagnosticsPath,
`codeql-action-${sanitizedTimestamp}-${uniqueSuffix}.json`,
);
writeFileSync(jsonPath, JSON.stringify(diagnostic));
writeDiagnosticFile(diagnosticsPath, diagnostic);
} catch (err) {
logger.warning(`Unable to write diagnostic message to database: ${err}`);
logger.debug(JSON.stringify(diagnostic));
action.logger.warning(
`Unable to write diagnostic message to database: ${err}`,
);
action.logger.debug(JSON.stringify(diagnostic));
}
}
/** Report if there are unwritten diagnostics and write them to the log. */
export function logUnwrittenDiagnostics() {
const logger = getActionsLogger();
const num = unwrittenDiagnostics.length;
/** Report if there are temporary diagnostics and write them to the log. */
export function logTemporaryDiagnostics(action: ActionState<["Logger"]>) {
const num = temporaryDiagnostics.length;
if (num > 0) {
logger.warning(
action.logger.warning(
`${num} diagnostic(s) could not be written to the database and will not appear on the Tool Status Page.`,
);
for (const unwritten of unwrittenDiagnostics) {
logger.debug(JSON.stringify(unwritten.diagnostic));
for (const temporary of temporaryDiagnostics) {
action.logger.debug(fs.readFileSync(temporary.path, "utf-8"));
}
}
}
/** Writes all unwritten diagnostics to disk. */
export function flushDiagnostics(config: Config) {
const logger = getActionsLogger();
/** Relocates all temporary diagnostics to the respective databases. */
export function flushDiagnostics(
action: ActionState<["Logger"]>,
config: ActionConfig<"dbLocation" | "languages">,
) {
const diagnosticsCount = temporaryDiagnostics.length;
action.logger.debug(
`Moving ${diagnosticsCount} diagnostic(s) to their respective databases.`,
);
const diagnosticsCount =
unwrittenDiagnostics.length + unwrittenDefaultLanguageDiagnostics.length;
logger.debug(`Writing ${diagnosticsCount} diagnostic(s) to database.`);
const failedFlushDiagnostics: TemporaryDiagnostic[] = [];
for (const temporary of temporaryDiagnostics) {
// If `temporary.language` is `undefined`, then we didn't have a `config` at the time that
// `addNoLanguageDiagnostic` was called. In that case, we arbitrarily choose the first
// configured language here like we would have done in `addNoLanguageDiagnostic`.
const directory = getDatabaseDiagnosticsPath(
config,
temporary.language ?? config.languages[0],
);
const filename = path.basename(temporary.path);
const destination = path.join(directory, filename);
for (const unwritten of unwrittenDiagnostics) {
writeDiagnostic(config, unwritten.language, unwritten.diagnostic);
}
for (const unwritten of unwrittenDefaultLanguageDiagnostics) {
addNoLanguageDiagnostic(config, unwritten);
try {
fs.mkdirSync(directory, { recursive: true });
try {
fs.renameSync(temporary.path, destination);
} catch (renameErr: unknown) {
// Rethrow the error if it isn't an EXDEV one.
if (!isErrnoException(renameErr) || renameErr.code !== "EXDEV") {
throw renameErr;
}
// The error was an EXDEV one: try to copy the file and delete the temporary one.
fs.copyFileSync(temporary.path, destination);
fs.unlinkSync(temporary.path);
}
} catch (err) {
action.logger.warning(
`Failed to relocate '${temporary.path}' to '${destination}': ${getErrorMessage(err)}`,
);
failedFlushDiagnostics.push(temporary);
}
}
// Reset the unwritten diagnostics arrays.
unwrittenDiagnostics = [];
unwrittenDefaultLanguageDiagnostics = [];
// Reset the temporary diagnostics arrays.
temporaryDiagnostics = failedFlushDiagnostics;
}
/**

View File

@@ -165,6 +165,13 @@ export enum EnvVar {
*/
SARIF_DUMP_DIR = "CODEQL_ACTION_SARIF_DUMP_DIR",
/**
* Whether to skip generating a failed SARIF file. Intended for testing purposes.
* This setting is more specific than `CODEQL_ACTION_TEST_MODE` and
* `CODEQL_ACTION_SKIP_SARIF_UPLOAD`, which imply this option.
*/
SKIP_GENERATE_FAILED_SARIF = "CODEQL_ACTION_SKIP_GENERATE_FAILED_SARIF",
/**
* Whether to skip uploading SARIF results to GitHub. Intended for testing purposes.
* This setting is more specific than `CODEQL_ACTION_TEST_MODE`, which implies this option.

View File

@@ -74,11 +74,6 @@ export enum Feature {
AllowMergeConfigFiles = "allow_merge_config_files",
/** Controls whether we allow multiple values for the `analysis-kinds` input. */
AllowMultipleAnalysisKinds = "allow_multiple_analysis_kinds",
/**
* Controls whether we delete CodeQL bundles that we are not going to use from the toolcache
* before downloading a different bundle, in order to reclaim disk space.
*/
CleanupToolcacheBundles = "cleanup_toolcache_bundles",
CleanupTrapCaches = "cleanup_trap_caches",
/** Whether to allow the `config-file` input to be specified via a repository property. */
ConfigFileRepositoryProperty = "config_file_repository_property",
@@ -170,6 +165,8 @@ export enum Feature {
*/
PerLanguageBundles = "per_language_bundles_v2",
QaTelemetryEnabled = "qa_telemetry_enabled",
/** Whether we should fail early if we detect that traced analysis is unsupported. */
SkipUnsupportedTracedAnalysis = "skip_unsupported_traced_analysis",
/** Routes (some) API requests through the registry proxy. */
ProxyApiRequests = "proxy_api_requests",
/** Note that this currently only disables baseline file coverage information. */
@@ -221,11 +218,6 @@ export const featureConfig = {
envVar: "CODEQL_ACTION_ALLOW_MULTIPLE_ANALYSIS_KINDS",
minimumVersion: undefined,
},
[Feature.CleanupToolcacheBundles]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES",
minimumVersion: undefined,
},
[Feature.CleanupTrapCaches]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CLEANUP_TRAP_CACHES",
@@ -466,6 +458,11 @@ export const featureConfig = {
envVar: "CODEQL_ACTION_START_PROXY_USE_FEATURES_RELEASE",
minimumVersion: undefined,
},
[Feature.SkipUnsupportedTracedAnalysis]: {
defaultValue: false,
envVar: "CODEQL_ACTION_SKIP_UNSUPPORTED_TRACED_ANALYSIS",
minimumVersion: undefined,
},
[Feature.ToolsRepositoryProperty]: {
defaultValue: false,
envVar: "CODEQL_ACTION_TOOLS_REPOSITORY_PROPERTY",

111
src/github.test.ts Normal file
View File

@@ -0,0 +1,111 @@
import test from "ava";
import { ActionsEnvVars } from "./environment";
import * as github from "./github";
import { getTestEnv, setupTests } from "./testing-utils";
setupTests(test);
test("parseGitHubUrl", (t) => {
t.deepEqual(github.parseGitHubUrl("github.com"), "https://github.com");
t.deepEqual(
github.parseGitHubUrl("https://github.com"),
"https://github.com",
);
t.deepEqual(
github.parseGitHubUrl("https://api.github.com"),
"https://github.com",
);
t.deepEqual(
github.parseGitHubUrl("https://github.com/foo/bar"),
"https://github.com",
);
t.deepEqual(
github.parseGitHubUrl("github.example.com"),
"https://github.example.com/",
);
t.deepEqual(
github.parseGitHubUrl("https://github.example.com"),
"https://github.example.com/",
);
t.deepEqual(
github.parseGitHubUrl("https://api.github.example.com"),
"https://github.example.com/",
);
t.deepEqual(
github.parseGitHubUrl("https://github.example.com/api/v3"),
"https://github.example.com/",
);
t.deepEqual(
github.parseGitHubUrl("https://github.example.com:1234"),
"https://github.example.com:1234/",
);
t.deepEqual(
github.parseGitHubUrl("https://api.github.example.com:1234"),
"https://github.example.com:1234/",
);
t.deepEqual(
github.parseGitHubUrl("https://github.example.com:1234/api/v3"),
"https://github.example.com:1234/",
);
t.deepEqual(
github.parseGitHubUrl("https://github.example.com/base/path"),
"https://github.example.com/base/path/",
);
t.deepEqual(
github.parseGitHubUrl("https://github.example.com/base/path/api/v3"),
"https://github.example.com/base/path/",
);
t.throws(() => github.parseGitHubUrl(""), {
message: '"" is not a valid URL',
});
t.throws(() => github.parseGitHubUrl("ssh://github.com"), {
message: '"ssh://github.com" is not a http or https URL',
});
t.throws(() => github.parseGitHubUrl("http:///::::433"), {
message: '"http:///::::433" is not a valid URL',
});
});
test("getCurrentInstance throws if required env vars aren't set", (t) => {
t.throws(() => github.getCurrentInstance(getTestEnv()));
t.throws(() => {
const env = getTestEnv({
[ActionsEnvVars.GITHUB_SERVER_URL]: "https://localhost",
});
github.getCurrentInstance(env);
});
t.throws(() => {
const env = getTestEnv({
[ActionsEnvVars.GITHUB_API_URL]: "https://api.localhost",
});
github.getCurrentInstance(env);
});
});
test("getCurrentInstance constructs instance from environment", (t) => {
const url = "https://localhost";
const apiUrl = "https://api.localhost";
const env = getTestEnv({
[ActionsEnvVars.GITHUB_SERVER_URL]: url,
[ActionsEnvVars.GITHUB_API_URL]: apiUrl,
});
const instance = github.getCurrentInstance(env);
t.is(instance.url, url);
t.is(instance.apiURL, apiUrl);
});
test("getCurrentInstance constructs dotcom instance with dotcom env vars", (t) => {
const env = getTestEnv({
[ActionsEnvVars.GITHUB_SERVER_URL]: github.GITHUB_DOTCOM_URL,
[ActionsEnvVars.GITHUB_API_URL]: github.GITHUB_DOTCOM_API_URL,
});
const instance = github.getCurrentInstance(env);
const dotcomInstance = github.getDotcomInstance();
t.deepEqual(dotcomInstance, instance);
});

110
src/github.ts Normal file
View File

@@ -0,0 +1,110 @@
import { ActionsEnvVars, type ReadOnlyEnv } from "./environment";
import { ConfigurationError } from "./util";
/** A string representing a GitHub instance address. */
export type InstanceURL = string;
/**
* The URL for github.com.
*/
export const GITHUB_DOTCOM_URL: InstanceURL = "https://github.com";
/** A string representing a GitHub API instance address. */
export type InstanceApiURL = string;
/**
* The URL for api.github.com.
*/
export const GITHUB_DOTCOM_API_URL: InstanceApiURL = "https://api.github.com";
/**
* Represents information about a GitHub instance, how to authenticate to it,
* and how to authenticate to other repos on it.
*/
export type GitHubApiCombinedDetails = GitHubApiDetails &
GitHubApiExternalRepoDetails;
/** Represents information about a GitHub instance. */
export type GitHubInstance = {
/** The base URL of the GitHub instance, such as `https://github.com`. */
url: string;
/**
* The API base URL of the GitHub instance, such as `https://api.github.com`.
* Typically this is the `api.` subdomain of the `url` domain.
*/
apiURL: string | undefined;
};
/** Represents a {@link GitHubInstance} with a token for accessing the API. */
export type GitHubApiDetails = GitHubInstance & {
/** The token to use for accessing the API of the GitHub instance. */
auth?: string;
};
/**
* Represents information for accessing a repository on the same GitHub instance as the Action
* is currently running on.
*/
export type GitHubApiExternalRepoDetails = {
/** An optional token for accessing repo(s) on the same GitHub instance. */
externalRepoAuth?: string;
};
/**
* Parses user input of a github.com or GHES URL to a canonical form.
* Removes any API prefix or suffix if one is present.
*/
export function parseGitHubUrl(inputUrl: string): string {
const originalUrl = inputUrl;
if (inputUrl.indexOf("://") === -1) {
inputUrl = `https://${inputUrl}`;
}
if (!inputUrl.startsWith("http://") && !inputUrl.startsWith("https://")) {
throw new ConfigurationError(`"${originalUrl}" is not a http or https URL`);
}
let url: URL;
try {
url = new URL(inputUrl);
} catch {
throw new ConfigurationError(`"${originalUrl}" is not a valid URL`);
}
// If we detect this is trying to be to github.com
// then return with a fixed canonical URL.
if (url.hostname === "github.com" || url.hostname === "api.github.com") {
return GITHUB_DOTCOM_URL;
}
// Remove the API prefix if it's present
if (url.pathname.indexOf("/api/v3") !== -1) {
url.pathname = url.pathname.substring(0, url.pathname.indexOf("/api/v3"));
}
// Also consider subdomain isolation on GHES
if (url.hostname.startsWith("api.")) {
url.hostname = url.hostname.substring(4);
}
// Normalise path to having a trailing slash for consistency
if (!url.pathname.endsWith("/")) {
url.pathname = `${url.pathname}/`;
}
return url.toString();
}
/** Retrieves a {@link GitHubInstance} describing the instance we are running on. */
export function getCurrentInstance(env: ReadOnlyEnv): GitHubInstance {
return {
url: env.getRequired(ActionsEnvVars.GITHUB_SERVER_URL),
apiURL: env.getRequired(ActionsEnvVars.GITHUB_API_URL),
};
}
/** Retrieves a {@link GitHubInstance} describing the GitHub.com instance. */
export function getDotcomInstance(): GitHubInstance {
return {
url: GITHUB_DOTCOM_URL,
apiURL: GITHUB_DOTCOM_API_URL,
};
}

View File

@@ -1,3 +1,6 @@
import * as fs from "fs";
import * as path from "path";
import * as core from "@actions/core";
import test, { ExecutionContext } from "ava";
import * as sinon from "sinon";
@@ -8,6 +11,7 @@ import * as apiClient from "./api-client";
import * as codeql from "./codeql";
import * as configUtils from "./config-utils";
import * as debugArtifacts from "./debug-artifacts";
import { getTempDiagnosticPath } from "./diagnostics";
import { EnvVar } from "./environment";
import { Feature } from "./feature-flags";
import * as initActionPostHelper from "./init-action-post-helper";
@@ -35,6 +39,47 @@ const NUM_BYTES_PER_GIB = 1024 * 1024 * 1024;
setupTests(test);
test.serial("generateFailedSarif picks up unwritten diagnostics", async (t) =>
util.withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const tempDiagDir = getTempDiagnosticPath(tmpDir);
fs.mkdirSync(tempDiagDir);
const codeqlObject = await codeql.getCodeQLForTesting();
const diagnosticsExport = sinon
.stub(codeqlObject, "diagnosticsExport")
.resolves();
sinon.stub(codeql, "getCodeQL").resolves(codeqlObject);
const sarifFile = path.join(tmpDir, "failed.sarif");
const result = await initActionPostHelper.generateFailedSarif(
logger,
createFeatures([]),
createTestConfig({
tempDir: tmpDir,
// Explicitly undefined to ensure we get the "no database" path.
dbLocation: undefined,
}),
"category",
"",
sarifFile,
);
t.truthy(result);
t.is(diagnosticsExport.callCount, 1);
t.true(
diagnosticsExport.calledOnceWith(
sarifFile,
"category",
sinon.match.any,
tempDiagDir,
),
);
}),
);
test.serial("init-post action with debug mode off", async (t) => {
return await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
@@ -793,6 +838,7 @@ async function testFailedSarifUpload(
}
t.true(
uploadFiles.calledOnceWith(
config,
sinon.match.string,
sinon.match.string,
category,

View File

@@ -24,6 +24,7 @@ import {
type UploadDebugArtifactsConfig,
} from "./debug-artifacts";
import * as dependencyCaching from "./dependency-caching";
import { getTempDiagnosticPath } from "./diagnostics";
import { EnvVar, ReadOnlyEnv } from "./environment";
import { Feature, FeatureEnablement } from "./feature-flags";
import { Logger } from "./logging";
@@ -44,6 +45,7 @@ import {
getRequiredEnvParam,
parseMatrixInput,
Result,
shouldSkipGenerateFailedSarif,
shouldSkipSarifUpload,
Success,
wrapError,
@@ -115,7 +117,7 @@ async function prepareFailedSarif(
const jobName = getRequiredEnvParam("GITHUB_JOB");
const matrix = parseMatrixInput(actionsUtil.getRequiredInput("matrix"));
if (shouldSkipSarifUpload()) {
if (shouldSkipGenerateFailedSarif()) {
return new Failure({
upload_failed_run_skipped_because: "SARIF upload is disabled",
});
@@ -176,7 +178,16 @@ async function prepareFailedSarif(
}
}
async function generateFailedSarif(
/**
* Generates a SARIF file for a failed run, i.e. an analysis where no SARIF was produced as a result of the analysis.
*
* Not intended to be called directly, other than by `prepareFailedSarif`.
*
* @param category The analysis category to include in the SARIF.
* @param checkoutPath The path where the repository was checked out at.
* @param sarifFile The SARIF file to produce.
*/
export async function generateFailedSarif(
logger: Logger,
features: FeatureEnablement,
config: ActionConfig<"codeQLCmd" | "tempDir">,
@@ -197,7 +208,19 @@ async function generateFailedSarif(
databasePath === undefined ||
!(await features.getValue(Feature.ExportDiagnosticsEnabled, codeql))
) {
await codeql.diagnosticsExport(sarifFile, category, config);
// If we don't have a database path, then we may have diagnostics that were written to a
// temporary location. Get the path that they would have been written to and check whether
// it exists. If so, pass it on to the CLI. The CLI will not complain if we give it a
// path that doesn't exist, so we are just being extra defensive here.
let diagnosticDir: string | undefined = getTempDiagnosticPath(
config.tempDir,
);
if (!fs.existsSync(diagnosticDir)) {
diagnosticDir = undefined;
}
await codeql.diagnosticsExport(sarifFile, category, config, diagnosticDir);
} else {
// We call 'database export-diagnostics' to find any per-database diagnostics.
await codeql.databaseExportDiagnostics(databasePath, sarifFile, category);
@@ -226,6 +249,7 @@ async function maybeUploadFailedSarif(
logger.info(`Uploading failed SARIF file ${failedSarif.sarifFile}`);
const uploadResult = await uploadLib.uploadFiles(
config,
failedSarif.sarifFile,
failedSarif.checkoutPath,
failedSarif.category,
@@ -239,6 +263,14 @@ async function maybeUploadFailedSarif(
logger,
{ isUnsuccessfulExecution: true },
);
// If SARIF upload is disabled for testing, `uploadFiles` will not have uploaded the failed
// SARIF file and instead will have written it to disk. Upload it as a workflow artifact so that
// we can inspect it in validation jobs.
if (uploadResult.sarifID && shouldSkipSarifUpload()) {
await uploadSarifArtifact(logger, "test-mode-sarif", failedSarif);
}
return uploadResult
? { ...uploadResult.statusReport, sarifID: uploadResult.sarifID }
: {};
@@ -257,23 +289,34 @@ async function maybeUploadFailedSarifArtifact(
}
const failedSarif = failedSarifResult.value;
return uploadSarifArtifact(logger, "sarif-artifact", failedSarif);
}
logger.info(
`Uploading failed SARIF file ${failedSarif.sarifFile} as artifact`,
);
/** Uploads a failed SARIF file as workflow artifact. */
async function uploadSarifArtifact(
logger: Logger,
prefix: string,
failedSarif: FailedSarifInfo,
): Promise<UploadFailedSarifResult> {
logger.startGroup(`Uploading failed SARIF file as artifact`);
logger.info(`Uploading '${failedSarif.sarifFile}' as artifact`);
const gitHubVersion = await getGitHubVersion();
const client = await getArtifactUploaderClient(logger, gitHubVersion.type);
try {
const gitHubVersion = await getGitHubVersion();
const client = await getArtifactUploaderClient(logger, gitHubVersion.type);
const suffix = getArtifactSuffix(actionsUtil.getOptionalInput("matrix"));
const name = sanitizeArtifactName(`sarif-artifact-${suffix}`);
await client.uploadArtifact(
name,
[path.normalize(failedSarif.sarifFile)],
path.normalize(".."),
);
const suffix = getArtifactSuffix(actionsUtil.getOptionalInput("matrix"));
const name = sanitizeArtifactName(`${prefix}-${suffix}`);
await client.uploadArtifact(
name,
[path.normalize(failedSarif.sarifFile)],
path.normalize(path.dirname(failedSarif.sarifFile)),
);
return { sarifID: name };
return { sarifID: name };
} finally {
logger.endGroup();
}
}
/**

View File

@@ -76,6 +76,9 @@ async function run(action: ActionState<["Base", "Logger", "Env"]>) {
// Try to load the "failed" configuration file instead which may have been written by the `init` step
// with a minimal configuration for this post-action.
if (config === undefined) {
logger.debug(
"Will try to load the configuration file from an aborted 'init' step, since there is no regular one.",
);
config = getAbortedInitConfig(action, tmpDir);
}

View File

@@ -40,7 +40,7 @@ import {
addDiagnostic,
addNoLanguageDiagnostic,
flushDiagnostics,
logUnwrittenDiagnostics,
logTemporaryDiagnostics,
makeDiagnostic,
makeTelemetryDiagnostic,
} from "./diagnostics";
@@ -50,6 +50,7 @@ import {
loadRepositoryProperties,
RepositoryPropertyName,
} from "./feature-flags/properties";
import * as github from "./github";
import {
checkInstallPython311,
checkPacksForOverlayCompatibility,
@@ -61,6 +62,7 @@ import {
runDatabaseInitCluster,
} from "./init";
import { JavaEnvVars, BuiltInLanguage } from "./languages";
import { checkRunnerSupportsTracedAnalysis } from "./languages/traced";
import { Logger, withGroupAsync } from "./logging";
import {
downloadOverlayBaseDatabaseFromCache,
@@ -233,8 +235,7 @@ async function run(
apiDetails = {
auth: getRequiredInput("token"),
externalRepoAuth: getOptionalInput("external-repository-token"),
url: actionState.env.getRequired(ActionsEnvVars.GITHUB_SERVER_URL),
apiURL: actionState.env.getRequired(ActionsEnvVars.GITHUB_API_URL),
...github.getCurrentInstance(actionState.env),
};
const gitHubVersion = await getGitHubVersion();
@@ -418,14 +419,11 @@ async function run(
logger,
});
if (
config.languages.includes(BuiltInLanguage.swift) &&
process.platform !== "darwin"
) {
throw new ConfigurationError(
`Swift analysis is only supported on macOS runner images. Please migrate to a macOS runner.`,
);
}
await checkRunnerSupportsTracedAnalysis(
actionStateWithFeatures,
config,
codeql,
);
if (repositoryPropertiesResult.isFailure()) {
addNoLanguageDiagnostic(
@@ -484,6 +482,7 @@ async function run(
if (statusReportBase !== undefined) {
await sendStatusReport(statusReportBase);
}
logTemporaryDiagnostics(actionState);
return;
}
@@ -766,9 +765,9 @@ async function run(
);
}
// Write diagnostics to the database that we previously stored in memory because the database
// did not exist until now.
flushDiagnostics(config);
// Move diagnostics to the database that we previously kept in a temporary location
// because the database did not exist until now
flushDiagnostics(actionState, config);
// We save the config here instead of at the end of `initConfig` because we
// may have updated the config returned from `initConfig`, e.g. to revert to
@@ -797,7 +796,7 @@ async function run(
);
return;
} finally {
logUnwrittenDiagnostics();
logTemporaryDiagnostics(actionState);
}
await sendCompletedStatusReport(
startedAt,

View File

@@ -0,0 +1,89 @@
import test from "ava";
import * as sinon from "sinon";
import { getCodeQLForTesting } from "../codeql";
import { ActionsEnvVars } from "../environment";
import {
createTestConfig,
getTestEnv,
makeVersionInfo,
RecordingLogger,
setupTests,
} from "../testing-utils";
import { ToolsFeature } from "../tools-features";
import { withTmpDir } from "../util";
import { isSwiftCompatible } from "./swift";
import { BuiltInLanguage } from ".";
setupTests(test);
test("isSwiftCompatible doesn't throw for non-Swift languages", async (t) => {
for (const language of Object.values(BuiltInLanguage)) {
if (language === BuiltInLanguage.swift) {
continue;
}
const codeql = await getCodeQLForTesting();
await t.notThrowsAsync(
isSwiftCompatible(
createTestConfig({ languages: [language] }),
codeql,
// A value of `undefined` should make `isSwiftCompatible` throw for Swift by default
// since it means we are on a platform other than macOS
undefined,
),
);
}
});
test("isSwiftCompatible doesn't throw for Swift if CLI supports swiftSupportsAllPlatforms", async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const codeql = await getCodeQLForTesting("codeql-for-testing", logger, env);
const supportsFeature = sinon
.stub(codeql, "supportsFeature")
.withArgs(ToolsFeature.SwiftSupportsAllPlatforms)
.resolves(true);
await t.notThrowsAsync(
isSwiftCompatible(
createTestConfig({ languages: [BuiltInLanguage.swift] }),
codeql,
// A value of `undefined` should make `isSwiftCompatible` throw for Swift by default
// since it means we are on a platform other than macOS
undefined,
),
);
t.is(supportsFeature.callCount, 1);
t.deepEqual(supportsFeature.args[0], [
ToolsFeature.SwiftSupportsAllPlatforms,
]);
}));
test("isSwiftCompatible doesn't throw for Swift on darwin", async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const codeql = await getCodeQLForTesting("codeql-for-testing", logger, env);
sinon.stub(codeql, "getVersion").resolves(makeVersionInfo("2.27.0"));
await t.notThrowsAsync(
isSwiftCompatible(
createTestConfig({
tempDir: tmpDir,
languages: [BuiltInLanguage.swift],
}),
codeql,
// A value other than `undefined` means we are on macOS.
"some-macos-version",
),
);
}));

42
src/languages/swift.ts Normal file
View File

@@ -0,0 +1,42 @@
import type { SemVer } from "semver";
import { CodeQL } from "../codeql";
import type { ActionConfig } from "../config/action-config";
import { ToolsFeature } from "../tools-features";
import { ConfigurationError } from "../util";
import { BuiltInLanguage } from ".";
/**
* Determines whether we can run a Swift analysis on the current runner.
*
* @param config The Action configuration.
* @param codeql The {@link CodeQL} CLI instance.
* @param macosVersion The version of macOS, if any.
*
* @throws {ConfigurationError} If Swift analysis is not possible on the current runner.
*/
export async function isSwiftCompatible(
config: ActionConfig<"languages">,
codeql: CodeQL,
macosVersion: SemVer | string | undefined,
) {
// The checks are not relevant if we are not trying to analyse Swift.
if (!config.languages.includes(BuiltInLanguage.swift)) {
return;
}
// Skip the checks if the `swiftSupportsAllPlatforms` feature is supported by the CLI.
// This is a forward-looking measure that allows a future CLI update to disable these
// platform checks in the Action when they shouldn't be enforced anymore.
if (await codeql.supportsFeature(ToolsFeature.SwiftSupportsAllPlatforms)) {
return;
}
// If `macosVersion` is undefined, then we are not on macOS and Swift analysis is unsupported.
if (macosVersion === undefined) {
throw new ConfigurationError(
`Swift analysis is only supported on macOS runner images. Please migrate to a macOS runner.`,
);
}
}

View File

@@ -0,0 +1,472 @@
import fs from "fs";
import test from "ava";
import * as sinon from "sinon";
import { getCodeQLForTesting, type CodeQL } from "../codeql";
import * as diagnostics from "../diagnostics";
import { ActionsEnvVars, type Env } from "../environment";
import { Feature } from "../feature-flags";
import type { Logger } from "../logging";
import { XCODE_SELECT_LINK_PATH } from "../platform";
import {
checkExpectedLogMessages,
checkUnexpectedLogMessages,
createFeatures,
createTestConfig,
getTestEnv,
initAllState,
makeVersionInfo,
RecordingLogger,
setupTests,
} from "../testing-utils";
import { BuildMode, withTmpDir } from "../util";
import { checkRunnerSupportsTracedAnalysis } from "./traced";
import { BuiltInLanguage } from ".";
setupTests(test);
const TEST_DEFAULT_TRACED_LANGUAGES = {
[BuiltInLanguage.swift]: true,
[BuiltInLanguage.go]: true,
};
async function getCodeQLWithTracedStub(
cmd: string,
logger: Logger,
env: Env,
tracedLanguages: Partial<
Record<BuiltInLanguage, boolean>
> = TEST_DEFAULT_TRACED_LANGUAGES,
): Promise<CodeQL> {
const codeql = await getCodeQLForTesting(cmd, logger, env);
sinon.stub(codeql, "isTracedLanguage").callsFake(async (language) => {
return tracedLanguages[language] === true;
});
return codeql;
}
const nonDarwinPlatforms: NodeJS.Platform[] = ["linux", "win32"];
for (const nonDarwinPlatform of nonDarwinPlatforms) {
test(`checkRunnerSupportsTracedAnalysis throws for Swift on ${nonDarwinPlatform}`, async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const codeql = await getCodeQLWithTracedStub(
"codeql-for-testing",
logger,
env,
);
sinon.stub(codeql, "getVersion").resolves(makeVersionInfo("2.27.0"));
await t.throwsAsync(
checkRunnerSupportsTracedAnalysis(
initAllState({ platform: nonDarwinPlatform, env, logger }),
createTestConfig({ languages: [BuiltInLanguage.swift] }),
codeql,
),
);
}));
}
test(`checkRunnerSupportsTracedAnalysis doesn't throw for BMN`, async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const codeql = await getCodeQLWithTracedStub(
"codeql-for-testing",
logger,
env,
);
sinon.stub(codeql, "getVersion").resolves(makeVersionInfo("2.27.0"));
await t.notThrowsAsync(
checkRunnerSupportsTracedAnalysis(
initAllState({
platform: "darwin",
osRelease: "27.0.0",
env,
logger,
}),
createTestConfig({ buildMode: BuildMode.None }),
codeql,
),
);
}));
// Some languages not in `TEST_DEFAULT_TRACED_LANGUAGES`.
const nonTracedLanguages: BuiltInLanguage[] = [
BuiltInLanguage.javascript,
BuiltInLanguage.actions,
];
for (const nonTracedLanguage of nonTracedLanguages) {
test(`checkRunnerSupportsTracedAnalysis doesn't throw for ${nonTracedLanguage} on unsupported macOS`, async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const codeql = await getCodeQLWithTracedStub(
"codeql-for-testing",
logger,
env,
);
sinon.stub(codeql, "getVersion").resolves(makeVersionInfo("2.27.0"));
await t.notThrowsAsync(
checkRunnerSupportsTracedAnalysis(
initAllState({
platform: "darwin",
osRelease: "27.0.0",
env,
logger,
}),
createTestConfig({ languages: [nonTracedLanguage] }),
codeql,
),
);
}));
}
test("checkRunnerSupportsTracedAnalysis warns if version string is not a semver", async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const codeql = await getCodeQLWithTracedStub(
"codeql-for-testing",
logger,
env,
);
sinon.stub(codeql, "getVersion").resolves(makeVersionInfo("2.27.0"));
await checkRunnerSupportsTracedAnalysis(
initAllState({
logger,
platform: "darwin",
osRelease: "unexpected",
env,
}),
createTestConfig({ languages: [BuiltInLanguage.go] }),
codeql,
);
checkExpectedLogMessages(t, logger.messages, [
"Unable to determine version of macOS, got: unexpected",
]);
}));
// `addDiagnostic` changes global state and we must stub it, so this test must be serial.
test.serial(
"checkRunnerSupportsTracedAnalysis logs and adds diagnostic if macOS version is unsupported",
async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const codeql = await getCodeQLWithTracedStub(
"codeql-for-testing",
logger,
env,
);
sinon.stub(codeql, "getVersion").resolves(makeVersionInfo("2.27.0"));
const addDiagnostic = sinon.stub(diagnostics, "addDiagnostic");
const config = createTestConfig({ languages: [BuiltInLanguage.swift] });
await checkRunnerSupportsTracedAnalysis(
initAllState({
logger,
platform: "darwin",
osRelease: "27.0.0",
env,
}),
config,
codeql,
);
checkExpectedLogMessages(t, logger.messages, [
"Traced analysis is not supported on macOS 27",
]);
t.is(addDiagnostic.callCount, 1);
t.like(addDiagnostic.args[0], [
config,
BuiltInLanguage.swift,
{
attributes: {
languages: [BuiltInLanguage.swift],
macOSVersion: "27.0.0",
},
severity: "warning",
source: {
id: "codeql-action/unsupported-traced-analysis-macos",
name: "Traced analysis is not supported on this version of macOS",
},
visibility: {
cliSummaryTable: true,
statusPage: true,
telemetry: true,
},
} satisfies Partial<diagnostics.DiagnosticMessage>,
]);
}),
);
// `addDiagnostic` changes global state and we must stub it, so this test must be serial.
test.serial(
"checkRunnerSupportsTracedAnalysis throws if macOS version is unsupported and FF is enabled",
async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const features = createFeatures([Feature.SkipUnsupportedTracedAnalysis]);
const codeql = await getCodeQLWithTracedStub(
"codeql-for-testing",
logger,
env,
);
sinon.stub(codeql, "getVersion").resolves(makeVersionInfo("2.27.0"));
const addDiagnostic = sinon.stub(diagnostics, "addDiagnostic");
const config = createTestConfig({ languages: [BuiltInLanguage.swift] });
await t.throwsAsync(
checkRunnerSupportsTracedAnalysis(
initAllState({
logger,
platform: "darwin",
osRelease: "27.0.0",
env,
features,
}),
config,
codeql,
),
);
t.is(addDiagnostic.callCount, 1);
t.like(addDiagnostic.args[0], [
config,
BuiltInLanguage.swift,
{
attributes: {
languages: [BuiltInLanguage.swift],
macOSVersion: "27.0.0",
},
severity: "error",
source: {
id: "codeql-action/unsupported-traced-analysis-macos",
name: "Traced analysis is not supported on this version of macOS",
},
visibility: {
cliSummaryTable: true,
statusPage: true,
telemetry: true,
},
} satisfies Partial<diagnostics.DiagnosticMessage>,
]);
}),
);
// `addDiagnostic` changes global state and we must stub it, so this test must be serial.
test.serial(
"checkRunnerSupportsTracedAnalysis doesn't add a diagnostic if Xcode version is supported",
async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const statSync = sinon
.stub(fs, "statSync")
.returns({ isSymbolicLink: () => true } as fs.Stats);
const readlinkSync = sinon
.stub(fs, "readlinkSync")
.returns("/Applications/Xcode_26.0.app/Contents/Developer");
const codeql = await getCodeQLWithTracedStub(
"codeql-for-testing",
logger,
env,
);
sinon.stub(codeql, "getVersion").resolves(makeVersionInfo("2.27.0"));
const addDiagnostic = sinon.stub(diagnostics, "addDiagnostic");
const config = createTestConfig({ languages: [BuiltInLanguage.swift] });
await checkRunnerSupportsTracedAnalysis(
initAllState({
logger,
platform: "darwin",
osRelease: "26.0.0",
env,
}),
config,
codeql,
);
t.is(statSync.callCount, 1);
t.deepEqual(statSync.args[0], [XCODE_SELECT_LINK_PATH]);
t.is(readlinkSync.callCount, 1);
t.deepEqual(readlinkSync.args[0], [XCODE_SELECT_LINK_PATH]);
checkUnexpectedLogMessages(t, logger.messages, [
"Traced analysis is not supported on Xcode 27",
]);
t.is(addDiagnostic.callCount, 0);
}),
);
// `addDiagnostic` changes global state and we must stub it, so this test must be serial.
test.serial(
"checkRunnerSupportsTracedAnalysis logs and adds diagnostic if Xcode version is unsupported",
async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const statSync = sinon
.stub(fs, "statSync")
.returns({ isSymbolicLink: () => true } as fs.Stats);
const readlinkSync = sinon
.stub(fs, "readlinkSync")
.returns("/Applications/Xcode_27.0.app/Contents/Developer");
const codeql = await getCodeQLWithTracedStub(
"codeql-for-testing",
logger,
env,
);
sinon.stub(codeql, "getVersion").resolves(makeVersionInfo("2.27.0"));
const addDiagnostic = sinon.stub(diagnostics, "addDiagnostic");
const config = createTestConfig({ languages: [BuiltInLanguage.swift] });
await checkRunnerSupportsTracedAnalysis(
initAllState({
logger,
platform: "darwin",
osRelease: "26.0.0",
env,
}),
config,
codeql,
);
t.is(statSync.callCount, 1);
t.deepEqual(statSync.args[0], [XCODE_SELECT_LINK_PATH]);
t.is(readlinkSync.callCount, 1);
t.deepEqual(readlinkSync.args[0], [XCODE_SELECT_LINK_PATH]);
checkExpectedLogMessages(t, logger.messages, [
"Traced analysis is not supported on Xcode 27",
]);
t.is(addDiagnostic.callCount, 1);
t.like(addDiagnostic.args[0], [
config,
BuiltInLanguage.swift,
{
attributes: {
languages: [BuiltInLanguage.swift],
xcodeVersion: "27.0.0",
},
severity: "warning",
source: {
id: "codeql-action/unsupported-traced-analysis-xcode",
name: "Traced analysis is not supported on this version of Xcode",
},
visibility: {
cliSummaryTable: true,
statusPage: true,
telemetry: true,
},
} satisfies Partial<diagnostics.DiagnosticMessage>,
]);
}),
);
// `addDiagnostic` changes global state and we must stub it, so this test must be serial.
test.serial(
"checkRunnerSupportsTracedAnalysis throws if Xcode version is unsupported and FF is enabled",
async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const statSync = sinon
.stub(fs, "statSync")
.returns({ isSymbolicLink: () => true } as fs.Stats);
const readlinkSync = sinon
.stub(fs, "readlinkSync")
.returns("/Applications/Xcode_27.0.app/Contents/Developer");
const features = createFeatures([Feature.SkipUnsupportedTracedAnalysis]);
const codeql = await getCodeQLWithTracedStub(
"codeql-for-testing",
logger,
env,
);
sinon.stub(codeql, "getVersion").resolves(makeVersionInfo("2.27.0"));
const addDiagnostic = sinon.stub(diagnostics, "addDiagnostic");
const config = createTestConfig({ languages: [BuiltInLanguage.swift] });
await t.throwsAsync(
checkRunnerSupportsTracedAnalysis(
initAllState({
logger,
platform: "darwin",
osRelease: "26.0.0",
env,
features,
}),
config,
codeql,
),
);
t.is(statSync.callCount, 1);
t.deepEqual(statSync.args[0], [XCODE_SELECT_LINK_PATH]);
t.is(readlinkSync.callCount, 1);
t.deepEqual(readlinkSync.args[0], [XCODE_SELECT_LINK_PATH]);
t.is(addDiagnostic.callCount, 1);
t.like(addDiagnostic.args[0], [
config,
BuiltInLanguage.swift,
{
attributes: {
languages: [BuiltInLanguage.swift],
xcodeVersion: "27.0.0",
},
severity: "error",
source: {
id: "codeql-action/unsupported-traced-analysis-xcode",
name: "Traced analysis is not supported on this version of Xcode",
},
visibility: {
cliSummaryTable: true,
statusPage: true,
telemetry: true,
},
} satisfies Partial<diagnostics.DiagnosticMessage>,
]);
}),
);

161
src/languages/traced.ts Normal file
View File

@@ -0,0 +1,161 @@
import * as semver from "semver";
import type { ActionState, Logger } from "../action-common";
import type { CodeQL } from "../codeql";
import type { ActionConfig } from "../config/action-config";
import { addDiagnostic, makeDiagnostic } from "../diagnostics";
import { Feature } from "../feature-flags";
import { macOSVersion, xcodeVersion } from "../platform";
import { ConfigurationError } from "../util";
import { isSwiftCompatible } from "./swift";
/** macOS 27 and above do not support traced extraction. */
export const UNSUPPORTED_MACOS_FOR_TRACED = 27;
/** Xcode 27 and above do not support traced extraction. */
export const UNSUPPORTED_XCODE_FOR_TRACED = 27;
/**
* Creates a diagnostic indicating that `version` of `product` is unsupported for traced analysis.
* Depending on `skipUnsupportedTracedAnalysis`, this function then either throws a {@link ConfigurationError}
* or logs the problem as a warning.
*
* @param logger The logger to use.
* @param config The CodeQL Action configuration.
* @param skipUnsupportedTracedAnalysis Whether this is a fatal error.
* @param product The product that the version is unsupported of.
* @param version The unsupported version.
*/
function handleUnsupportedVersion(
logger: Logger,
config: ActionConfig<"languages" | "dbLocation" | "tempDir">,
skipUnsupportedTracedAnalysis: boolean,
product: "macOS" | "Xcode",
version: semver.SemVer,
) {
const baseMessage = [
`Traced analysis is not supported on ${product} ${UNSUPPORTED_MACOS_FOR_TRACED} or above.`,
`Configure your analysis to run on macOS ${UNSUPPORTED_MACOS_FOR_TRACED - 1} or below`,
`and XCode ${UNSUPPORTED_XCODE_FOR_TRACED - 1} or below.`,
].join(" ");
const attributeName = product === "macOS" ? "macOSVersion" : "xcodeVersion";
// Create a diagnostic that will show up on the TSP.
for (const language of config.languages) {
addDiagnostic(
config,
language,
makeDiagnostic(
`codeql-action/unsupported-traced-analysis-${product.toLowerCase()}`,
`Traced analysis is not supported on this version of ${product}`,
{
attributes: {
languages: config.languages,
[attributeName]: version.toString(),
},
markdownMessage: baseMessage,
severity: skipUnsupportedTracedAnalysis ? "error" : "warning",
visibility: {
cliSummaryTable: true,
statusPage: true,
telemetry: true,
},
},
),
);
}
// Throw an error to abort the analysis if the FF is enabled or log the message.
if (skipUnsupportedTracedAnalysis) {
// ConfigurationErrors are converted to the "aborted" status by the exception handler
// in `init-action.ts` that guards the call to `checkRunnerSupportsTracedAnalysis`.
throw new ConfigurationError(baseMessage);
} else {
// This will also show up as a workflow annotation.
logger.warning(baseMessage);
}
}
/**
* Decides whether {@link config} has any traced languages by looking for tracer
* configurations in the {@link codeql} CLI installation.
*/
export async function hasTracedLanguages(
config: ActionConfig<"languages">,
codeql: CodeQL,
): Promise<boolean> {
for (const language of config.languages) {
if (await codeql.isTracedLanguage(language)) {
return true;
}
}
return false;
}
/**
* Determines whether we can run a traced analysis on the current runner.
*
* @throws {ConfigurationError} If traced analysis is not possible on the current runner.
*/
export async function checkRunnerSupportsTracedAnalysis(
action: ActionState<["Base", "Logger", "FeatureFlags"]>,
config: ActionConfig<"languages" | "dbLocation" | "tempDir">,
codeql: CodeQL,
) {
// Try to get the macOS version.
const version = macOSVersion(action);
// Perform some initial, Swift-specific checks.
await isSwiftCompatible(config, codeql, version);
// `build-mode: none` doesn't use the tracer, so it is always supported.
if (config.buildMode !== undefined && config.buildMode === "none") {
return;
}
// If none of the languages are traced, then we can skip the compatibility checks below as well.
if (!(await hasTracedLanguages(config, codeql))) {
return;
}
const skipUnsupportedTracedAnalysis = await action.features.getValue(
Feature.SkipUnsupportedTracedAnalysis,
);
if (typeof version === "string") {
// If we got a string, we are on macOS but couldn't parse the version string.
action.logger.warning(
`Unable to determine version of macOS, got: ${version}`,
);
} else if (
version !== undefined &&
version.major >= UNSUPPORTED_MACOS_FOR_TRACED
) {
handleUnsupportedVersion(
action.logger,
config,
skipUnsupportedTracedAnalysis,
"macOS",
version,
);
}
// Determining whether the Xcode version is supported only makes sense on macOS, so we only do it
// after determining that we are running on macOS.
const xcodeVer = xcodeVersion(action.logger);
if (
xcodeVer !== undefined &&
xcodeVer.major >= UNSUPPORTED_XCODE_FOR_TRACED
) {
handleUnsupportedVersion(
action.logger,
config,
skipUnsupportedTracedAnalysis,
"Xcode",
xcodeVer,
);
}
}

View File

@@ -1,6 +1,23 @@
import test from "ava";
import fs from "fs";
import { BundlePlatform, getBundlePlatform } from "./platform";
import test from "ava";
import * as sinon from "sinon";
import {
BundlePlatform,
getBundlePlatform,
macOSVersion,
XCODE_SELECT_LINK_PATH,
xcodeVersion,
} from "./platform";
import {
checkExpectedLogMessages,
initAllState,
RecordingLogger,
setupTests,
} from "./testing-utils";
setupTests(test);
for (const [platform, arch, expected] of [
["linux", "x64", BundlePlatform.Linux64],
@@ -16,3 +33,160 @@ for (const [platform, arch, expected] of [
t.is(getBundlePlatform(platform, arch), expected);
});
}
const platforms: NodeJS.Platform[] = ["linux", "win32", "freebsd"];
for (const platform of platforms) {
test(`macOSVersion returns undefined on ${platform}`, (t) => {
t.is(macOSVersion(initAllState({ platform })), undefined);
});
}
test("macOSVersion returns raw string if semver parsing fails", (t) => {
const invalidSemVer = "sealOS-2026";
t.is(
macOSVersion(
initAllState({ platform: "darwin", osRelease: invalidSemVer }),
),
invalidSemVer,
);
});
test("macOSVersion returns semver if parsing succeeds", (t) => {
const validSemVer = "27.0.1";
const version = macOSVersion(
initAllState({ platform: "darwin", osRelease: validSemVer }),
);
// Check that `version` is not undefined and narrow the type; throws if undefined.
if (t.truthy(version)) {
// Check that it's also not just a string.
t.not(typeof version, "string");
// Should be an object with the expected properties.
t.is(typeof version, "object");
t.is(version["major"], 27);
t.is(version["minor"], 0);
t.is(version["patch"], 1);
}
});
test.serial("xcodeVersion returns undefined if symlink doesn't exist", (t) => {
const logger = new RecordingLogger();
const statSync = sinon.stub(fs, "statSync").throws(new Error("ENOENT"));
t.is(xcodeVersion(logger), undefined);
t.is(statSync.callCount, 1);
t.deepEqual(statSync.args[0], [XCODE_SELECT_LINK_PATH]);
checkExpectedLogMessages(t, logger.messages, [
"Unable to determine Xcode version: ENOENT",
]);
});
test.serial("xcodeVersion returns undefined if file is not a symlink", (t) => {
const logger = new RecordingLogger();
const statSync = sinon
.stub(fs, "statSync")
.returns({ isSymbolicLink: () => false } as fs.Stats);
t.is(xcodeVersion(logger), undefined);
t.is(statSync.callCount, 1);
t.deepEqual(statSync.args[0], [XCODE_SELECT_LINK_PATH]);
checkExpectedLogMessages(t, logger.messages, [
"exists, but is not a symbolic link",
]);
});
test.serial(
"xcodeVersion returns undefined if resolving the symlink returns nothing",
(t) => {
const logger = new RecordingLogger();
const statSync = sinon
.stub(fs, "statSync")
.returns({ isSymbolicLink: () => true } as fs.Stats);
const readlinkSync = sinon.stub(fs, "readlinkSync").returns("");
t.is(xcodeVersion(logger), undefined);
t.is(statSync.callCount, 1);
t.deepEqual(statSync.args[0], [XCODE_SELECT_LINK_PATH]);
t.is(readlinkSync.callCount, 1);
t.deepEqual(readlinkSync.args[0], [XCODE_SELECT_LINK_PATH]);
checkExpectedLogMessages(t, logger.messages, [
"unexpectedly returned nothing",
]);
},
);
test.serial(
"xcodeVersion returns undefined if resolved path doesn't include pattern",
(t) => {
const logger = new RecordingLogger();
const statSync = sinon
.stub(fs, "statSync")
.returns({ isSymbolicLink: () => true } as fs.Stats);
const readlinkSync = sinon
.stub(fs, "readlinkSync")
.returns("/Applications/Xcode.app/Contents/Developer");
t.is(xcodeVersion(logger), undefined);
t.is(statSync.callCount, 1);
t.deepEqual(statSync.args[0], [XCODE_SELECT_LINK_PATH]);
t.is(readlinkSync.callCount, 1);
t.deepEqual(readlinkSync.args[0], [XCODE_SELECT_LINK_PATH]);
checkExpectedLogMessages(t, logger.messages, [
"does not contain expected pattern",
]);
},
);
test.serial("xcodeVersion returns undefined if match can't be parsed", (t) => {
const logger = new RecordingLogger();
const statSync = sinon
.stub(fs, "statSync")
.returns({ isSymbolicLink: () => true } as fs.Stats);
const readlinkSync = sinon
.stub(fs, "readlinkSync")
.returns("/Applications/Xcode_00.0.app/Contents/Developer");
t.is(xcodeVersion(logger), undefined);
t.is(statSync.callCount, 1);
t.deepEqual(statSync.args[0], [XCODE_SELECT_LINK_PATH]);
t.is(readlinkSync.callCount, 1);
t.deepEqual(readlinkSync.args[0], [XCODE_SELECT_LINK_PATH]);
checkExpectedLogMessages(t, logger.messages, [
"Couldn't parse '00.0' as a semantic version.",
]);
});
test.serial("xcodeVersion returns version from resolved path", (t) => {
const logger = new RecordingLogger();
const statSync = sinon
.stub(fs, "statSync")
.returns({ isSymbolicLink: () => true } as fs.Stats);
const readlinkSync = sinon
.stub(fs, "readlinkSync")
.returns("/Applications/Xcode_16.4.app/Contents/Developer");
const result = xcodeVersion(logger);
if (t.truthy(result)) {
t.is(result.major, 16);
t.is(result.minor, 4);
}
t.is(statSync.callCount, 1);
t.deepEqual(statSync.args[0], [XCODE_SELECT_LINK_PATH]);
t.is(readlinkSync.callCount, 1);
t.deepEqual(readlinkSync.args[0], [XCODE_SELECT_LINK_PATH]);
});

View File

@@ -1,3 +1,10 @@
import fs from "fs";
import * as semver from "semver";
import type { ActionState, Logger } from "./action-common";
import { getErrorMessage } from "./util";
/** Platform identifiers used in CodeQL bundle asset names. */
export enum BundlePlatform {
Linux64 = "linux64",
@@ -24,3 +31,92 @@ export function getBundlePlatform(
return undefined;
}
}
/**
* Tries to determine the version of macOS.
*
* @returns
* The version as either a semantic version object, the raw version string
* if it is not a semantic version, or `undefined` if we are not on macOS.
*/
export function macOSVersion(
action: ActionState<["Base"]>,
): semver.SemVer | string | undefined {
// Skip if we are not running on macOS.
if (action.platform !== "darwin") {
return undefined;
}
// Try to parse the OS version string.
const version = semver.parse(action.osRelease);
if (version === null) {
return action.osRelease;
}
return version;
}
/** The static path we check for a symbolic link to the (dynamic) Xcode location. */
export const XCODE_SELECT_LINK_PATH = "/private/var/db/xcode_select_link";
/** The pattern we expect to find in the Xcode path. */
export const XCODE_APP_FILENAME_PATTERN = new RegExp(
/(?<filename>Xcode_(?<majorMinor>\d+.\d+).app)/,
);
/**
* Tries to determine the version of Xcode that is installed.
*
* @param logger The logger to use.
* @returns The Xcode version or `undefined` if it couldn't be determined.
*/
export function xcodeVersion(logger: Logger): semver.SemVer | undefined {
try {
// Stat the expected symbolic link to check that it exists and is a symbolic link.
// The `readlinkSync` call below returns an empty string in either case and so
// this check allows us to distinguish between the two cases.
const stats = fs.statSync(XCODE_SELECT_LINK_PATH);
if (!stats.isSymbolicLink()) {
logger.warning(
`${XCODE_SELECT_LINK_PATH} exists, but is not a symbolic link.`,
);
return undefined;
}
// Read what the symbolic link points to.
const xcodePath = fs.readlinkSync(XCODE_SELECT_LINK_PATH);
if (xcodePath === "") {
logger.warning(
`Resolving ${XCODE_SELECT_LINK_PATH} unexpectedly returned nothing.`,
);
return undefined;
}
// Try to extract the version from the path.
const matchResult = xcodePath.match(XCODE_APP_FILENAME_PATTERN);
if (matchResult?.groups === undefined) {
logger.warning(
`Xcode path '${xcodePath}' does not contain expected pattern.`,
);
return undefined;
}
const majorMinor = matchResult.groups["majorMinor"];
const version = semver.coerce(majorMinor);
if (version === null) {
logger.warning(`Couldn't parse '${majorMinor}' as a semantic version.`);
return undefined;
}
return version;
} catch (err) {
logger.warning(
`Unable to determine Xcode version: ${getErrorMessage(err)}`,
);
return undefined;
}
}

View File

@@ -17,9 +17,10 @@ import { getGitHubVersion } from "./api-client";
import { CodeQL } from "./codeql";
import { ComputedInput, getToolsInput } from "./config/inputs";
import { getRawLanguagesNoAutodetect } from "./config-utils";
import { ActionsEnvVars, EnvVar } from "./environment";
import { EnvVar } from "./environment";
import { initFeatures } from "./feature-flags";
import { loadRepositoryProperties } from "./feature-flags/properties";
import * as github from "./github";
import { initCodeQL } from "./init";
import { Logger } from "./logging";
import { getRepositoryNwo } from "./repository";
@@ -112,8 +113,7 @@ async function run(
const apiDetails = {
auth: getRequiredInput("token"),
externalRepoAuth: getOptionalInput("external-repository-token"),
url: actionState.env.getRequired(ActionsEnvVars.GITHUB_SERVER_URL),
apiURL: actionState.env.getRequired(ActionsEnvVars.GITHUB_API_URL),
...github.getCurrentInstance(actionState.env),
};
const gitHubVersion = await getGitHubVersion();

View File

@@ -596,7 +596,7 @@ test.serial(
undefined, // rawLanguages
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
createFeatures([Feature.CleanupToolcacheBundles]),
createFeatures([]),
getRunnerLogger(true),
);
@@ -1375,10 +1375,7 @@ for (const bundle of ["per-language", "combined", "fallback"] as const) {
};
});
const addDiagnostic = sinon.stub(diagnostics, "addNoLanguageDiagnostic");
const features = createFeatures([
Feature.PerLanguageBundles,
Feature.CleanupToolcacheBundles,
]);
const features = createFeatures([Feature.PerLanguageBundles]);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
@@ -1629,16 +1626,20 @@ test.serial(
},
]);
const result = await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersion]),
getRunnerLogger(true),
);
t.deepEqual(result, [
{ cliVersion: "2.20.1", tagName: "codeql-bundle-v2.20.1" },
{ cliVersion: "2.20.0", tagName: "codeql-bundle-v2.20.0" },
]);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const result =
await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersion]),
getRunnerLogger(true),
);
t.deepEqual(result, [
{ cliVersion: "2.20.1", tagName: "codeql-bundle-v2.20.1" },
{ cliVersion: "2.20.0", tagName: "codeql-bundle-v2.20.0" },
]);
});
},
);
@@ -1755,21 +1756,25 @@ test.serial(
},
]);
const result = await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersionDryRun]),
getRunnerLogger(true),
);
t.deepEqual(
result,
[],
"Dry-run should return an empty list so the caller falls back.",
);
t.assert(
listStub.calledOnce,
"Dry-run should still list Actions caches to populate the diagnostic.",
);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const result =
await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersionDryRun]),
getRunnerLogger(true),
);
t.deepEqual(
result,
[],
"Dry-run should return an empty list so the caller falls back.",
);
t.assert(
listStub.calledOnce,
"Dry-run should still list Actions caches to populate the diagnostic.",
);
});
},
);
@@ -1783,18 +1788,22 @@ test.serial(
},
]);
const result = await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
createFeatures([
Feature.OverlayAnalysisMatchCodeqlVersion,
Feature.OverlayAnalysisMatchCodeqlVersionDryRun,
]),
getRunnerLogger(true),
);
t.deepEqual(result, [
{ cliVersion: "2.20.1", tagName: "codeql-bundle-v2.20.1" },
]);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const result =
await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
createFeatures([
Feature.OverlayAnalysisMatchCodeqlVersion,
Feature.OverlayAnalysisMatchCodeqlVersionDryRun,
]),
getRunnerLogger(true),
);
t.deepEqual(result, [
{ cliVersion: "2.20.1", tagName: "codeql-bundle-v2.20.1" },
]);
});
},
);
@@ -1826,11 +1835,15 @@ function createToolcacheEntry(
*/
async function runDownloadCodeQL(
toolcacheRoot: string,
features: Feature[],
bundleVersion: string | undefined,
env = getTestEnv({
[ActionsEnvVars.RUNNER_TOOL_CACHE]: toolcacheRoot,
}),
{
env = getTestEnv({
[ActionsEnvVars.RUNNER_ENVIRONMENT]: "github-hosted",
[ActionsEnvVars.RUNNER_TOOL_CACHE]: toolcacheRoot,
}),
}: {
env?: Env;
} = {},
): Promise<{
codeqlFolder: string;
cleanupDiagnostic: toolsDownload.ToolcacheCleanupResult | undefined;
@@ -1841,7 +1854,6 @@ async function runDownloadCodeQL(
const { codeqlFolder } = await setupCodeql.downloadCodeQLBundle(
initAllState({
env,
features: createFeatures(features),
}),
{
bundle: {
@@ -1879,11 +1891,9 @@ async function runDownloadCodeQL(
async function testToolcacheCleanup(
t: ExecutionContext<unknown>,
{
features,
runnerEnvironment,
setUp,
}: {
features: Feature[];
runnerEnvironment: string | undefined;
setUp?: (env: Env) => void;
},
@@ -1895,12 +1905,8 @@ async function testToolcacheCleanup(
) {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
if (runnerEnvironment === undefined) {
delete process.env[ActionsEnvVars.RUNNER_ENVIRONMENT];
} else {
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = runnerEnvironment;
}
const env = getTestEnv({
[ActionsEnvVars.RUNNER_ENVIRONMENT]: runnerEnvironment,
[ActionsEnvVars.RUNNER_TOOL_CACHE]: tmpDir,
});
setUp?.(env);
@@ -1920,9 +1926,8 @@ async function testToolcacheCleanup(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
features,
CLEANUP_BUNDLE_VERSION,
env,
{ env },
);
t.true(
@@ -1934,28 +1939,12 @@ async function testToolcacheCleanup(
});
}
test.serial(
"downloadCodeQLBundle does not clean up the toolcache when the feature flag is disabled",
async (t) => {
await testToolcacheCleanup(
t,
{ features: [], runnerEnvironment: "github-hosted" },
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.true(fs.existsSync(staleDirectory));
t.true(fs.existsSync(destinationDirectory));
t.is(cleanupDiagnostic, undefined);
},
);
},
);
test.serial(
"downloadCodeQLBundle does not clean up the toolcache when the runner is not GitHub-hosted",
async (t) => {
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "self-hosted",
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
@@ -1975,7 +1964,6 @@ test.serial(
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: undefined,
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
@@ -1988,12 +1976,11 @@ test.serial(
);
test.serial(
"downloadCodeQLBundle deletes other CodeQL bundles from the toolcache when enabled on a GitHub-hosted runner",
"downloadCodeQLBundle deletes other CodeQL bundles from the toolcache on a GitHub-hosted runner",
async (t) => {
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "github-hosted",
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
@@ -2027,7 +2014,6 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2057,7 +2043,6 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2189,7 +2174,6 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
toolcacheRoot,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2212,7 +2196,6 @@ test.serial(
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "github-hosted",
setUp: (env) => {
env.set(EnvVar.HAS_SET_UP_CODEQL, "true");
@@ -2242,9 +2225,13 @@ test.serial(
);
const { codeqlFolder, cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
getTestEnv({ [EnvVar.HAS_SET_UP_CODEQL]: "true" }),
{
env: getTestEnv({
[ActionsEnvVars.RUNNER_ENVIRONMENT]: "github-hosted",
[EnvVar.HAS_SET_UP_CODEQL]: "true",
}),
},
);
t.true(fs.existsSync(staleDirectory));
@@ -2312,7 +2299,7 @@ test.serial(
undefined, // rawLanguages
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
createFeatures([Feature.CleanupToolcacheBundles]),
createFeatures([]),
getRunnerLogger(true),
),
{ is: error },
@@ -2347,7 +2334,6 @@ test.serial(
const { codeqlFolder, cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
undefined, // bundleVersion
);
@@ -2381,7 +2367,6 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2417,7 +2402,6 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2467,7 +2451,6 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
toolcacheRoot,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);

View File

@@ -839,7 +839,7 @@ function getToolcacheDestination(
}
/**
* Reclaims disk space by deleting the CodeQL tools from the toolcache, if enabled.
* Reclaims disk space by deleting the CodeQL tools from the toolcache on GitHub-hosted runners.
*
* On GitHub-hosted runners the toolcache shares a filesystem with the workspace, so tools left in
* the toolcache take up space that the analysis could use instead. This holds wherever we extract
@@ -847,9 +847,8 @@ function getToolcacheDestination(
*/
async function tryDeleteToolcacheBundles({
env,
features,
logger,
}: ActionState<["Logger", "ReadOnlyEnv", "FeatureFlags"]>): Promise<void> {
}: ActionState<["Logger", "ReadOnlyEnv"]>): Promise<void> {
// A step that has already set up CodeQL may hand out a path into the toolcache that a later step
// runs, so only the first step to set it up can know that nothing else relies on the toolcache.
if (env.getOptional(EnvVar.HAS_SET_UP_CODEQL) !== undefined) {
@@ -860,11 +859,7 @@ async function tryDeleteToolcacheBundles({
return;
}
if (
!isGitHubHostedRunner() ||
!isToolcacheOnWorkspaceFilesystem(logger) ||
!(await features.getValue(Feature.CleanupToolcacheBundles))
) {
if (!isGitHubHostedRunner(env) || !isToolcacheOnWorkspaceFilesystem(logger)) {
return;
}
@@ -983,7 +978,7 @@ export async function setupCodeQLBundle(
break;
case "download": {
const result = await downloadCodeQLBundle(
{ env: getEnv(), features, logger },
{ env: getEnv(), logger },
source,
apiDetails,
zstdAvailability.version,
@@ -1019,7 +1014,7 @@ export async function setupCodeQLBundle(
* @returns The extraction directory and download timings.
*/
export async function downloadCodeQLBundle(
action: ActionState<["Logger", "ReadOnlyEnv", "FeatureFlags"]>,
action: ActionState<["Logger", "ReadOnlyEnv"]>,
source: CodeQLDownloadSource,
apiDetails: api.GitHubApiDetails,
tarVersion: tar.TarVersion | undefined,

View File

@@ -2,12 +2,13 @@ import { ActionState } from "../action-common";
import { isRunningLocalAction } from "../actions-util";
import * as api from "../api-client";
import { ActionsEnvVars } from "../environment";
import { GITHUB_DOTCOM_URL } from "../github";
import {
isRepositoryNwoStr,
parseRepositoryNwo,
type RepositoryNwoStr,
} from "../repository";
import { GITHUB_DOTCOM_URL, type InstanceURL } from "../util";
import { type InstanceURL } from "../util";
/** The NWO of the standard CodeQL Action repository. */
export const CODEQL_DEFAULT_ACTION_REPOSITORY = "github/codeql-action";

View File

@@ -31,6 +31,7 @@ import {
exportEnvVar,
} from "./environment";
import { getRef } from "./git-utils";
import { GITHUB_DOTCOM_URL } from "./github";
import * as json from "./json";
import type { Logger } from "./logging";
import type { OverlayBaseDatabaseDownloadStats } from "./overlay/caching";
@@ -41,7 +42,6 @@ import {
ConfigurationError,
getRequiredEnvParam,
isInTestMode,
GITHUB_DOTCOM_URL,
DiskUsage,
assertNever,
BuildMode,

View File

@@ -1,4 +1,5 @@
import { TextDecoder } from "node:util";
import * as os from "os";
import path from "path";
import * as github from "@actions/github";
@@ -34,6 +35,7 @@ import { Logger } from "./logging";
import { OverlayDatabaseMode } from "./overlay/overlay-database-mode";
import { getBundlePlatform } from "./platform";
import { ActionName } from "./status-report";
import { ToolsFeature } from "./tools-features";
import {
DEFAULT_DEBUG_ARTIFACT_NAME,
DEFAULT_DEBUG_DATABASE_NAME,
@@ -272,6 +274,7 @@ export function initAllState(
startedAt: new Date(),
platform: process.platform,
arch: process.arch,
osRelease: os.release(),
logger: new RecordingLogger(),
env,
actions: getTestActionsEnv(env),
@@ -910,7 +913,7 @@ export function mockLanguagesInRepo(languages: string[]) {
*/
export const makeVersionInfo = (
version: string,
features?: { [name: string]: boolean },
features?: { [key in ToolsFeature]?: boolean },
overlayVersion?: number,
): VersionInfo => ({
version,

View File

@@ -6,6 +6,7 @@ export enum ToolsFeature {
BundleSupportsOverlay = "bundleSupportsOverlay",
IndirectTracingSupportsStaticBinaries = "indirectTracingSupportsStaticBinaries",
SuppressesMissingFileBaselineWarning = "suppressesMissingFileBaselineWarning",
SwiftSupportsAllPlatforms = "swiftSupportsAllPlatforms",
}
/**

View File

@@ -12,12 +12,13 @@ import * as analyses from "./analyses";
import * as api from "./api-client";
import { getGitHubVersion, wrapApiConfigurationError } from "./api-client";
import { CodeQL, getCodeQL } from "./codeql";
import { getConfig } from "./config-utils";
import type { ActionConfig } from "./config/action-config";
import { readDiffRangesJsonFile } from "./diff-informed-analysis-utils";
import { ActionsEnvVars, EnvVar, exportEnvVar } from "./environment";
import { FeatureEnablement } from "./feature-flags";
import * as fingerprints from "./fingerprints";
import * as gitUtils from "./git-utils";
import * as github from "./github";
import { initCodeQL } from "./init";
import { Logger } from "./logging";
import { getRepositoryNwo, RepositoryNwo } from "./repository";
@@ -33,7 +34,6 @@ import * as util from "./util";
import {
ConfigurationError,
getErrorMessage,
getRequiredEnvParam,
GitHubVariant,
GitHubVersion,
satisfiesGHESVersion,
@@ -94,16 +94,22 @@ async function shouldDisableCombineSarifFiles(
return true;
}
// Takes a list of paths to sarif files and combines them together using the
// CLI `github merge-results` command when all SARIF files are produced by
// CodeQL. Otherwise, it will fall back to combining the files in the action.
// Returns the contents of the combined sarif file.
/**
* Takes a list of paths to sarif files and combines them together using the
* CLI `github merge-results` command when all SARIF files are produced by
* CodeQL. Otherwise, it will fall back to combining the files in the action.
*
* @returns The contents of the combined sarif file.
*/
async function combineSarifFilesUsingCLI(
sarifFiles: string[],
gitHubVersion: GitHubVersion,
features: FeatureEnablement,
logger: Logger,
config: ActionConfig<"codeQLCmd" | "tempDir"> | undefined,
): Promise<Partial<sarif.Log>> {
const env = util.getEnv();
logger.info("Combining SARIF files using the CodeQL CLI");
const sarifObjects = sarifFiles.map(sarif.readSarifFile);
@@ -138,7 +144,6 @@ async function combineSarifFilesUsingCLI(
let codeQL: CodeQL;
let tempDir: string = actionsUtil.getTemporaryDirectory();
const config = await getConfig(tempDir, logger);
if (config !== undefined) {
codeQL = await getCodeQL(logger, config.codeQLCmd);
tempDir = config.tempDir;
@@ -152,8 +157,7 @@ async function combineSarifFilesUsingCLI(
externalRepoAuth: actionsUtil.getOptionalInput(
"external-repository-token",
),
url: getRequiredEnvParam("GITHUB_SERVER_URL"),
apiURL: getRequiredEnvParam("GITHUB_API_URL"),
...github.getCurrentInstance(env),
};
const codeQLDefaultVersionInfo =
@@ -227,6 +231,14 @@ function getAutomationID(
return api.computeAutomationID(analysis_key, environment);
}
/** Gets the path to a saved payload file for {@link analysisKind}. */
export function getSavedPayloadPath(analysisKind: analyses.AnalysisKind) {
return path.join(
actionsUtil.getTemporaryDirectory(),
`payload-${analysisKind}.json`,
);
}
/**
* Upload the given payload.
* If the request fails then this will retry a small number of times.
@@ -241,10 +253,7 @@ export async function uploadPayload(
logger.info("Uploading results");
if (util.shouldSkipSarifUpload()) {
const payloadSaveFile = path.join(
actionsUtil.getTemporaryDirectory(),
`payload-${analysis.kind}.json`,
);
const payloadSaveFile = getSavedPayloadPath(analysis.kind);
logger.info(
`SARIF upload disabled by an environment variable. Saving to ${payloadSaveFile}`,
);
@@ -598,6 +607,7 @@ export interface PostProcessingResults {
export async function postProcessSarifFiles(
logger: Logger,
features: FeatureEnablement,
config: ActionConfig<"codeQLCmd" | "tempDir"> | undefined,
checkoutPath: string,
sarifPaths: string[],
category: string | undefined,
@@ -622,6 +632,7 @@ export async function postProcessSarifFiles(
gitHubVersion,
features,
logger,
config,
);
} else {
const sarifPath = sarifPaths[0];
@@ -682,6 +693,7 @@ export async function writePostProcessedFiles(
* to.
*/
export async function uploadFiles(
config: ActionConfig<"codeQLCmd" | "tempDir"> | undefined,
inputSarifPath: string,
checkoutPath: string,
category: string | undefined,
@@ -696,6 +708,7 @@ export async function uploadFiles(
return uploadSpecifiedFiles(
sarifPaths,
config,
checkoutPath,
category,
features,
@@ -709,6 +722,7 @@ export async function uploadFiles(
*/
async function uploadSpecifiedFiles(
sarifPaths: string[],
config: ActionConfig<"codeQLCmd" | "tempDir"> | undefined,
checkoutPath: string,
category: string | undefined,
features: FeatureEnablement,
@@ -718,6 +732,7 @@ async function uploadSpecifiedFiles(
const processingResults: PostProcessingResults = await postProcessSarifFiles(
logger,
features,
config,
checkoutPath,
sarifPaths,
category,
@@ -856,6 +871,16 @@ export async function waitForProcessing(
): Promise<void> {
logger.startGroup("Waiting for processing to finish");
try {
// If SARIF uplaod is disabled and we reach this point, then the file was written to disk and
// there is nothing to wait for. We still allow ourselves to get to this point so that we can
// see in the log that we would have waited for processing to finish if the upload had happened.
if (util.shouldSkipSarifUpload()) {
logger.info(
"Nothing to wait for, because SARIF upload is disabled by an environment variable and the SARIF was saved to disk.",
);
return;
}
const client = api.getApiClient();
// Do an initial wait because processing will always take a minimum of 2-3 seconds

View File

@@ -5,6 +5,7 @@ import * as actionsUtil from "./actions-util";
import { getActionVersion, getTemporaryDirectory } from "./actions-util";
import * as analyses from "./analyses";
import { getGitHubVersion } from "./api-client";
import { getConfig } from "./config-utils";
import { initFeatures } from "./feature-flags";
import { Logger } from "./logging";
import { getRepositoryNwo } from "./repository";
@@ -66,11 +67,12 @@ async function run({ startedAt, logger }: ActionState<["Base", "Logger"]>) {
// Make inputs accessible in the `post` step.
actionsUtil.persistInputs();
const tempDir = getTemporaryDirectory();
const repositoryNwo = getRepositoryNwo();
const features = initFeatures(
gitHubVersion,
repositoryNwo,
getTemporaryDirectory(),
tempDir,
logger,
);
@@ -91,9 +93,15 @@ async function run({ startedAt, logger }: ActionState<["Base", "Logger"]>) {
const checkoutPath = actionsUtil.getRequiredInput("checkout_path");
const category = actionsUtil.getOptionalInput("category");
// Although we wouldn't typically expect there to be a configuration since we don't expect another
// CodeQL Action step to have come before `upload-sarif`, this has previously been the behaviour
// when `combineSarifFilesUsingCLI` called `getConfig` directly. Since the only effect of this would
// be that an existing CLI that was installed gets re-used, we maintain this behaviour.
const maybeConfig = await getConfig(tempDir, logger);
const uploadResults = await postProcessAndUploadSarif(
logger,
features,
maybeConfig,
"always",
checkoutPath,
sarifPath,

View File

@@ -26,6 +26,7 @@ function mockPostProcessSarifFiles() {
const analysisConfig = getAnalysisConfig(analysisKind);
postProcessSarifFiles
.withArgs(
sinon.match.object,
sinon.match.any,
sinon.match.any,
sinon.match.any,
@@ -75,9 +76,11 @@ const postProcessAndUploadSarifMacro = makeMacro({
fs.writeFileSync(sarifFile, "");
}
const config = undefined;
const actual = await postProcessAndUploadSarif(
logger,
features,
config,
"always",
"",
testPath,
@@ -94,6 +97,7 @@ const postProcessAndUploadSarifMacro = makeMacro({
postProcessSarifFiles.calledWith(
logger,
features,
config,
sinon.match.any,
analysisKindResult.expectedFiles?.map(toFullPath) ??
fullSarifPaths,
@@ -223,6 +227,7 @@ test.serial(
const actual = await postProcessAndUploadSarif(
logger,
features,
undefined,
"never",
"",
tempDir,
@@ -253,6 +258,7 @@ test.serial(
const actual = await postProcessAndUploadSarif(
logger,
features,
undefined,
"never",
"",
tempDir,

View File

@@ -1,5 +1,6 @@
import { UploadKind } from "./actions-util";
import * as analyses from "./analyses";
import type { ActionConfig } from "./config/action-config";
import { FeatureEnablement } from "./feature-flags";
import { Logger } from "./logging";
import * as upload_lib from "./upload-lib";
@@ -26,6 +27,7 @@ export type UploadSarifResults = Partial<
export async function postProcessAndUploadSarif(
logger: Logger,
features: FeatureEnablement,
config: ActionConfig<"codeQLCmd" | "tempDir"> | undefined,
uploadKind: UploadKind,
checkoutPath: string,
sarifPath: string,
@@ -45,6 +47,7 @@ export async function postProcessAndUploadSarif(
const postProcessingResults = await upload_lib.postProcessSarifFiles(
logger,
features,
config,
checkoutPath,
sarifFiles,
category,

View File

@@ -11,7 +11,7 @@ import * as sinon from "sinon";
import * as api from "./api-client";
import { EnvVar } from "./environment";
import { getRunnerLogger } from "./logging";
import { setupTests } from "./testing-utils";
import { getTestEnv, setupTests } from "./testing-utils";
import * as util from "./util";
setupTests(test);
@@ -164,66 +164,6 @@ test.serial("getExtraOptionsEnvParam() fails on invalid JSON", (t) => {
process.env.CODEQL_ACTION_EXTRA_OPTIONS = origExtraOptions;
});
test("parseGitHubUrl", (t) => {
t.deepEqual(util.parseGitHubUrl("github.com"), "https://github.com");
t.deepEqual(util.parseGitHubUrl("https://github.com"), "https://github.com");
t.deepEqual(
util.parseGitHubUrl("https://api.github.com"),
"https://github.com",
);
t.deepEqual(
util.parseGitHubUrl("https://github.com/foo/bar"),
"https://github.com",
);
t.deepEqual(
util.parseGitHubUrl("github.example.com"),
"https://github.example.com/",
);
t.deepEqual(
util.parseGitHubUrl("https://github.example.com"),
"https://github.example.com/",
);
t.deepEqual(
util.parseGitHubUrl("https://api.github.example.com"),
"https://github.example.com/",
);
t.deepEqual(
util.parseGitHubUrl("https://github.example.com/api/v3"),
"https://github.example.com/",
);
t.deepEqual(
util.parseGitHubUrl("https://github.example.com:1234"),
"https://github.example.com:1234/",
);
t.deepEqual(
util.parseGitHubUrl("https://api.github.example.com:1234"),
"https://github.example.com:1234/",
);
t.deepEqual(
util.parseGitHubUrl("https://github.example.com:1234/api/v3"),
"https://github.example.com:1234/",
);
t.deepEqual(
util.parseGitHubUrl("https://github.example.com/base/path"),
"https://github.example.com/base/path/",
);
t.deepEqual(
util.parseGitHubUrl("https://github.example.com/base/path/api/v3"),
"https://github.example.com/base/path/",
);
t.throws(() => util.parseGitHubUrl(""), {
message: '"" is not a valid URL',
});
t.throws(() => util.parseGitHubUrl("ssh://github.com"), {
message: '"ssh://github.com" is not a http or https URL',
});
t.throws(() => util.parseGitHubUrl("http:///::::433"), {
message: '"http:///::::433" is not a valid URL',
});
});
test("allowed API versions", async (t) => {
t.is(util.apiVersionInRange("1.33.0", "1.33", "2.0"), undefined);
t.is(util.apiVersionInRange("1.33.1", "1.33", "2.0"), undefined);
@@ -553,3 +493,70 @@ test("Failure.orElse returns the default value for a failure result", (t) => {
const result = new util.Failure(new Error("test error"));
t.is(result.orElse("default value"), "default value");
});
test("shouldSkipGenerateFailedSarif", (t) => {
const env = getTestEnv();
// Don't skip by default
t.is(util.shouldSkipGenerateFailedSarif(env), false);
// Skip in test mode
env.setAll({
[EnvVar.TEST_MODE]: "true",
[EnvVar.SKIP_SARIF_UPLOAD]: undefined,
[EnvVar.SKIP_GENERATE_FAILED_SARIF]: undefined,
});
t.is(util.shouldSkipGenerateFailedSarif(env), true);
// Skip if explicitly requested
env.setAll({
[EnvVar.TEST_MODE]: undefined,
[EnvVar.SKIP_SARIF_UPLOAD]: undefined,
[EnvVar.SKIP_GENERATE_FAILED_SARIF]: "true",
});
t.is(util.shouldSkipGenerateFailedSarif(env), true);
// Skip if not uploading SARIF
env.setAll({
[EnvVar.TEST_MODE]: undefined,
[EnvVar.SKIP_SARIF_UPLOAD]: "true",
[EnvVar.SKIP_GENERATE_FAILED_SARIF]: undefined,
});
t.is(util.shouldSkipGenerateFailedSarif(env), true);
// Don't skip if explicitly requested
env.setAll({
[EnvVar.TEST_MODE]: "true",
[EnvVar.SKIP_SARIF_UPLOAD]: "true",
[EnvVar.SKIP_GENERATE_FAILED_SARIF]: "false",
});
t.is(util.shouldSkipGenerateFailedSarif(env), false);
});
test("shouldSkipSarifUpload", (t) => {
const env = getTestEnv();
// Don't skip by default
t.is(util.shouldSkipSarifUpload(env), false);
// Skip in test mode
env.setAll({
[EnvVar.TEST_MODE]: "true",
[EnvVar.SKIP_SARIF_UPLOAD]: undefined,
});
t.is(util.shouldSkipSarifUpload(env), true);
// Skip if explicitly requested
env.setAll({
[EnvVar.TEST_MODE]: undefined,
[EnvVar.SKIP_SARIF_UPLOAD]: "true",
});
t.is(util.shouldSkipSarifUpload(env), true);
// Don't skip if explicitly requested
env.setAll({
[EnvVar.TEST_MODE]: "true",
[EnvVar.SKIP_SARIF_UPLOAD]: "false",
});
t.is(util.shouldSkipSarifUpload(env), false);
});

View File

@@ -18,10 +18,10 @@ import type { Config } from "./config-utils";
import {
EnvVar,
getRequiredEnvParam,
isInTestMode,
exportEnvVar,
getEnv,
type Env,
type ReadOnlyEnv,
} from "./environment";
import * as json from "./json";
import { Language } from "./languages";
@@ -50,11 +50,6 @@ const BROKEN_VERSIONS = ["0.0.0-20211207"];
/** A string representing a GitHub instance. */
export type InstanceURL = string;
/**
* The URL for github.com.
*/
export const GITHUB_DOTCOM_URL: InstanceURL = "https://github.com";
/**
* Default name of the debugging artifact.
*/
@@ -470,49 +465,6 @@ export function getGeneratedSuitePath(config: Config, language: Language) {
);
}
/**
* Parses user input of a github.com or GHES URL to a canonical form.
* Removes any API prefix or suffix if one is present.
*/
export function parseGitHubUrl(inputUrl: string): string {
const originalUrl = inputUrl;
if (inputUrl.indexOf("://") === -1) {
inputUrl = `https://${inputUrl}`;
}
if (!inputUrl.startsWith("http://") && !inputUrl.startsWith("https://")) {
throw new ConfigurationError(`"${originalUrl}" is not a http or https URL`);
}
let url: URL;
try {
url = new URL(inputUrl);
} catch {
throw new ConfigurationError(`"${originalUrl}" is not a valid URL`);
}
// If we detect this is trying to be to github.com
// then return with a fixed canonical URL.
if (url.hostname === "github.com" || url.hostname === "api.github.com") {
return GITHUB_DOTCOM_URL;
}
// Remove the API prefix if it's present
if (url.pathname.indexOf("/api/v3") !== -1) {
url.pathname = url.pathname.substring(0, url.pathname.indexOf("/api/v3"));
}
// Also consider subdomain isolation on GHES
if (url.hostname.startsWith("api.")) {
url.hostname = url.hostname.substring(4);
}
// Normalise path to having a trailing slash for consistency
if (!url.pathname.endsWith("/")) {
url.pathname = `${url.pathname}/`;
}
return url.toString();
}
const CODEQL_ACTION_WARNED_ABOUT_VERSION_ENV_VAR =
"CODEQL_ACTION_WARNED_ABOUT_VERSION";
@@ -732,11 +684,30 @@ export function isGoodVersion(versionSpec: string) {
return !BROKEN_VERSIONS.includes(versionSpec);
}
/**
* Returns whether we specifically want to skip generating a failed SARIF.
*/
export function shouldSkipGenerateFailedSarif(
env: ReadOnlyEnv = getEnv(),
): boolean {
const skipGenerateFailedSarif = env.getOptional(
EnvVar.SKIP_GENERATE_FAILED_SARIF,
);
return (
skipGenerateFailedSarif !== "false" &&
(shouldSkipSarifUpload(env) || skipGenerateFailedSarif === "true")
);
}
/**
* Returns whether we specifically want to skip uploading SARIF files.
*/
export function shouldSkipSarifUpload(): boolean {
return isInTestMode() || process.env[EnvVar.SKIP_SARIF_UPLOAD] === "true";
export function shouldSkipSarifUpload(env: ReadOnlyEnv = getEnv()): boolean {
const skipSarifUpload = env.getOptional(EnvVar.SKIP_SARIF_UPLOAD);
return (
skipSarifUpload !== "false" &&
(env.isSkippingUploadsInTests() || skipSarifUpload === "true")
);
}
/**
@@ -929,6 +900,21 @@ export function getErrorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
/**
* Approximates whether {@link error} is like a {@link NodeJS.ErrnoException}.
* We only typically care about the `code` property, so that's what we check for here.
*/
export function isErrnoException(
error: unknown,
): error is NodeJS.ErrnoException {
return (
typeof error === "object" &&
error !== null &&
"code" in error &&
typeof error.code === "string"
);
}
export function prettyPrintPack(pack: Pack) {
return `${pack.name}${pack.version ? `@${pack.version}` : ""}${
pack.path ? `:${pack.path}` : ""

View File

@@ -2,6 +2,16 @@
Change-notes are Markdown files used to document user-facing changes. When making a change that affects users, create a Markdown file here that describes the change. During the next release, the change-note files in `unreleased-change-notes/` will automatically be added to `CHANGELOG.md`.
### What do document
Some guidelines to help you decide whether a change needs a change note:
- New functionality that is gated behind a feature flag should be documented when it is added, highlighting that the feature will only be rolled out later. Ideally, this should also include an estimate for when the feature is expected to become available.
- When a feature flag is removed and the gated functionality becomes the default behaviour, this does not need to be documented.
- Changes that may require users to make changes to their workflows must be documented.
If in doubt, see `CHANGELOG.md` for what we have included previously.
### Change-note file format
Change-note files must follow a certain format so that they can be automatically validated and processed. Failure to follow the format will result in a failed PR check.