AI first-pass review on open pull requests

Every hour, the open pull requests in one GitHub repository are checked and each one not yet reviewed gets a first pass: its changed files are fetched, the AI is asked to note what is plainly visible in the diff, such as missing error handling, unclear names or new behaviour without a test, and those points are posted as a comment.

The reviewer starts from something rather than nothing. The comment says it is a first pass, and nothing is ever approved, merged or closed. Each pull request is reviewed once: after commenting, a label is added and anything carrying it is skipped, so a re-run never posts the same review twice; remove the label to have it looked at again. Drafts are left alone, and on a large pull request only the first batch of changed files is read, with the comment stating how many it covers.

You need a GitHub token that can read the repository, since the changed files are read from GitHub's API directly, and the label must already exist in the repository.

Run every hour.Run every hour.

Set repositoryOwner to blank.Set repositoryOwner to blank.

Set repositoryName to blank.Set repositoryName to blank.

Set repositoryReadToken to blank.Set repositoryReadToken to blank.

Set reviewedLabel to first-pass-reviewed.Set reviewedLabel to first-pass-reviewed.

Set maxPullRequestsPerRun to 5.Set maxPullRequestsPerRun to 5.

Set maxFilesPerReview to 20.Set maxFilesPerReview to 20.

Set maxDiffCharactersPerFile to 3000.Set maxDiffCharactersPerFile to 3000.

Set githubHeaders to a record of Authorization, Accept, and X-GitHub-Api-Version.Set githubHeaders to a record of Authorization, Accept, and X-GitHub-Api-Version.

Join a list of [https://api.github.com/repos/, repositoryOwner, /, repositoryName] with nothing and save it as repositoryApiUrl.Join a list of [https://api.github.com/repos/, repositoryOwner, /, repositoryName] with nothing and save it as repositoryApiUrl.

Join a list of [repositoryApiUrl, /pulls] with nothing and save it as openPullRequestsUrl.Join a list of [repositoryApiUrl, /pulls] with nothing and save it as openPullRequestsUrl.

Start an empty list and save it as openPullRequests.Start an empty list and save it as openPullRequests.

If a step below fails, save the error as pullRequestListErrorIf a step below fails, save the error as pullRequestListError

Try

Change openPullRequests to <expr>.body.Change openPullRequests to <expr>.body.

On error

Stop and report a problem: a list of [Could not read the open pull requests from GitHub: , pullRequestListError.message with openPullRequestsUrl replaced by the GitHub API address, Check the repository details and the token in the 'repositoryReadToken' step.] joined by nothing.Stop and report a problem: a list of [Could not read the open pull requests from GitHub: , pullRequestListError.message with openPullRequestsUrl replaced by the GitHub API address, Check the repository details and the token in the 'repositoryReadToken' step.] joined by nothing.

Set reviewedThisRun to 0.Set reviewedThisRun to 0.

For each pullRequest in openPullRequests:For each pullRequest in openPullRequests:

If reviewedThisRun is greater than or equal to maxPullRequestsPerRunIf reviewedThisRun is greater than or equal to maxPullRequestsPerRun

Yes

Show in the output: a list of [Reviewed , maxPullRequestsPerRun, pull requests, which is this run's limit. The rest keep their place in the queue and are picked up on the next run.] joined by nothing.Show in the output: a list of [Reviewed , maxPullRequestsPerRun, pull requests, which is this run's limit. The rest keep their place in the queue and are picked up on the next run.] joined by nothing.

Stop repeating.Stop repeating.

No

No steps

Set pullNumber to pullRequest.number.Set pullNumber to pullRequest.number.

If pullNumber is noIf pullNumber is no

Yes

Show in the output: Skipped a pull request that GitHub returned without a number, so there was nothing to comment on..Show in the output: Skipped a pull request that GitHub returned without a number, so there was nothing to comment on..

Skip to the next item.Skip to the next item.

No

No steps

If pullRequest.draft is yesIf pullRequest.draft is yes

Yes

Show in the output: a list of [Pull request #, pullNumber, is still a draft, so it was left alone.] joined by nothing.Show in the output: a list of [Pull request #, pullNumber, is still a draft, so it was left alone.] joined by nothing.

Skip to the next item.Skip to the next item.

No

No steps

Set alreadyReviewed to false.Set alreadyReviewed to false.

For each pullRequestLabel in pullRequest.labels:For each pullRequestLabel in pullRequest.labels:

If pullRequestLabel.name is reviewedLabelIf pullRequestLabel.name is reviewedLabel

Yes

Change alreadyReviewed to true.Change alreadyReviewed to true.

Stop repeating.Stop repeating.

No

No steps

If alreadyReviewed is yesIf alreadyReviewed is yes

Yes

Skip to the next item.Skip to the next item.

No

No steps

Join a list of [repositoryApiUrl, /pulls/, pullNumber, /files] with nothing and save it as changedFilesUrl.Join a list of [repositoryApiUrl, /pulls/, pullNumber, /files] with nothing and save it as changedFilesUrl.

Start an empty list and save it as changedFiles.Start an empty list and save it as changedFiles.

If a step below fails, save the error as changedFilesErrorIf a step below fails, save the error as changedFilesError

Try

Change changedFiles to <expr>.body.Change changedFiles to <expr>.body.

On error

Show in the output: a list of [Could not read the changed files for pull request #, pullNumber, : , changedFilesError.message with changedFilesUrl replaced by the GitHub API address] joined by nothing.Show in the output: a list of [Could not read the changed files for pull request #, pullNumber, : , changedFilesError.message with changedFilesUrl replaced by the GitHub API address] joined by nothing.

If changedFiles is noIf changedFiles is no

Yes

Show in the output: a list of [Pull request #, pullNumber, has no changed files this workflow can read, so it was left for a human.] joined by nothing.Show in the output: a list of [Pull request #, pullNumber, has no changed files this workflow can read, so it was left for a human.] joined by nothing.

Skip to the next item.Skip to the next item.

No

No steps

Count the items in changedFiles and save it as changedFileCount.Count the items in changedFiles and save it as changedFileCount.

Take the first maxFilesPerReview items of changedFiles and save it as filesToReview.Take the first maxFilesPerReview items of changedFiles and save it as filesToReview.

Count the items in filesToReview and save it as reviewedFileCount.Count the items in filesToReview and save it as reviewedFileCount.

Start an empty list and save it as diffBlocks.Start an empty list and save it as diffBlocks.

For each changedFile in filesToReview:For each changedFile in filesToReview:

Set changedFileName to blank.Set changedFileName to blank.

If a step below fails, save the error as fileNameErrorIf a step below fails, save the error as fileNameError

Try

Trim changedFile.filename and save it as changedFileName.Trim changedFile.filename and save it as changedFileName.

On error

Change changedFileName to blank.Change changedFileName to blank.

If changedFileName is emptyIf changedFileName is empty

Yes

Show in the output: a list of [Skipped a changed file in pull request #, pullNumber, because GitHub returned it without a filename.] joined by nothing.Show in the output: a list of [Skipped a changed file in pull request #, pullNumber, because GitHub returned it without a filename.] joined by nothing.

Skip to the next item.Skip to the next item.

No

No steps

Set changedFilePatch to blank.Set changedFilePatch to blank.

If a step below fails, save the error as filePatchErrorIf a step below fails, save the error as filePatchError

Try

Trim changedFile.patch and save it as changedFilePatch.Trim changedFile.patch and save it as changedFilePatch.

On error

Change changedFilePatch to blank.Change changedFilePatch to blank.

Set diffBlockBody to (GitHub sent no text diff for this file - it may be binary, a pure rename, or too large to display.).Set diffBlockBody to (GitHub sent no text diff for this file - it may be binary, a pure rename, or too large to display.).

If changedFilePatch isn't emptyIf changedFilePatch isn't empty

Yes

Shorten changedFilePatch to maxDiffCharactersPerFile characters and save it as diffBlockBody.Shorten changedFilePatch to maxDiffCharactersPerFile characters and save it as diffBlockBody.

No

No steps

Add a list of [File: , changedFileName, "Add a list of [File: , changedFileName, , diffBlockBody] joined by nothing to the end of diffBlocks and save it as diffBlocks.

If diffBlocks is noIf diffBlocks is no

Yes

Show in the output: a list of [None of the changed files in pull request #, pullNumber, could be read, so it was left for a human.] joined by nothing.Show in the output: a list of [None of the changed files in pull request #, pullNumber, could be read, so it was left for a human.] joined by nothing.

Skip to the next item.Skip to the next item.

No

No steps

Set coverageNote to blank.Set coverageNote to blank.

If changedFileCount is greater than reviewedFileCountIf changedFileCount is greater than reviewedFileCount

Yes

Join a list of [This pull request changes , changedFileCount, files. This first pass read only the first , reviewedFileCount, of them, in the order GitHub lists them - the rest have not been looked at.] with nothing and save it as coverageNote.Join a list of [This pull request changes , changedFileCount, files. This first pass read only the first , reviewedFileCount, of them, in the order GitHub lists them - the rest have not been looked at.] with nothing and save it as coverageNote.

No

Join a list of [This first pass read all , changedFileCount, changed files.] with nothing and save it as coverageNote.Join a list of [This first pass read all , changedFileCount, changed files.] with nothing and save it as coverageNote.

Join a list of ["You are giving a pull request a first-pass read for a human reviewer who has not opened it yet.Join a list of [You are giving a pull request a first-pass read for a human reviewer who has not opened it yet. , Repository: , repositoryOwner, /, repositoryName, Pull request #, pullNumber, : , pullRequest.title, Opened by: , pullRequest.user.login, , coverageNote, The changed files and their diffs follow. Lines starting with + or - are the change itself. , diffBlocks joined by a blank line, Write short markdown bullet points about what is plainly visible in the diff above: - error handling that looks missing, such as an unchecked failure or a swallowed exception - names that do not say what the thing is - new behaviour that arrives without a test - anything else obviously risky, such as a hardcoded secret or an unbounded loop Group the points by file and quote the identifier or the line you mean. Make at most eight points, most important first. If nothing stands out, reply with the single sentence: Nothing obvious stood out in this first pass. Do not approve the pull request, do not say LGTM, and do not suggest merging - a human decides that. Do not comment on anything that is not in the diff above.] with nothing and save it as reviewPrompt.

Set reviewText to blank.Set reviewText to blank.

If a step below fails, save the error as reviewErrorIf a step below fails, save the error as reviewError

Try

Trim the response text from the OpenAI “Ask OpenAI” step and save it as reviewText.Trim the response text from the OpenAI “Ask OpenAI” step and save it as reviewText.

On error

Show in the output: a list of [Could not get a first-pass review for pull request #, pullNumber, : , reviewError.message] joined by nothing.Show in the output: a list of [Could not get a first-pass review for pull request #, pullNumber, : , reviewError.message] joined by nothing.

If reviewText is emptyIf reviewText is empty

Yes

Show in the output: a list of [The model returned nothing for pull request #, pullNumber, , so no comment was posted and it stays unlabelled for the next run.] joined by nothing.Show in the output: a list of [The model returned nothing for pull request #, pullNumber, , so no comment was posted and it stays unlabelled for the next run.] joined by nothing.

Skip to the next item.Skip to the next item.

No

No steps

Join a list of ["### Automated first-pass reviewJoin a list of [### Automated first-pass review , coverageNote, , reviewText, This is a first pass, meant to give a human reviewer somewhere to start. It is not an approval, and this workflow never approves, merges or closes anything. Remove the ', reviewedLabel, ' label from this pull request to have it looked at again.] with nothing and save it as commentBody.

GitHubComment on issue pullNumber in repositoryOwner/repositoryName.Comment on issue pullNumber in repositoryOwner/repositoryName.commentBody

If a step below fails, save the error as labelErrorIf a step below fails, save the error as labelError

Try

GitHubAdd label reviewedLabel to issue pullNumber in repositoryOwner/repositoryName.Add label reviewedLabel to issue pullNumber in repositoryOwner/repositoryName.

On error

Show in the output: a list of [Commented on pull request #, pullNumber, but could not add the ', reviewedLabel, ' label, so the next run will comment on it again. Add the label by hand to stop that: , labelError.message] joined by nothing.Show in the output: a list of [Commented on pull request #, pullNumber, but could not add the ', reviewedLabel, ' label, so the next run will comment on it again. Add the label by hand to stop that: , labelError.message] joined by nothing.

Add 1 to reviewedThisRun.Add 1 to reviewedThisRun.

Show in the output: a list of [First-pass review finished. Commented on , reviewedThisRun, pull request(s).] joined by nothing.Show in the output: a list of [First-pass review finished. Commented on , reviewedThisRun, pull request(s).] joined by nothing.