Update Check - Now throws a NotFound

Bug Report

Current Behavior When using ProBot to do an update to an existing Check it has stopped working

import { Application, Context, Octokit} from 'probot';
import Webhooks from '@octokit/webhooks';
import fs from 'fs-extra';
import { ChecksUpdateParamsOutputAnnotations } from '@octokit/rest';
import { execSync } from 'child_process';

/**
 * Default config if no YML file found or its malformed
 */
let defaultConfig:IConfig = {
  Vale: {
    Enabled: true,
    Paths: {
      Configuration: "/_vale.ini",
      Styles: "/vale/DocsStyles/"
    },
    Success: {
      Header: "DEFAULT Congratulations, there are no errors",
      Message: "DEFAULT The Vale Docs linter did not find any issues",
      ShowImage: true,
      ImageUrl: "https://media.giphy.com/media/6nuiJjOOQBBn2/giphy.gif"
    },
    Error: {
      Header: "DEFAULT Failed automated validation",
      Message: "DEFAULT There are one or more automated warnings with your PR",
      ShowImage: false,
      ImageUrl: ""
    }
  }
};


export = (app: Application) => {

  app.log('Booting up Vale Linter ProBot');

  // Listen for these PR webhook payload types
  app.on(['pull_request.opened', 'pull_request.synchronize', 'pull_request_reopened'], async (context: Context<Webhooks.WebhookPayloadPullRequest>) => {

    // Get the config from the repo that this bot installed
    // & if does not exist or malformed fallback to deafult one
    // .github/vale-linter.yml
    let config:IConfig = await context.config('vale-linter.yml', defaultConfig);

    if(config.Vale.Enabled === false){
      app.log('Vale Linter Config Disabled. Exiting Early');
      return;
    }

    app.log('Reacting to a PR webhook payload');

    const repoOwner = context.payload.repository.owner.login;
    const repoName = context.payload.repository.name;
    const headSha = context.payload.pull_request.head.sha;
    const prNumber = context.payload.number;

    // Create a NEW Check (For new PR's or code updated/sync'd or has been re-opened)
    // We will listen/hear back with a new WebHook event once it's been created
    await context.github.checks.create({
      name: 'Markdown Doc Lint',
      owner: repoOwner,
      repo: repoName,
      head_sha: headSha
    });

    app.log('Posted to GitHub API to create a Check for PR', prNumber);

  });

  app.on('check_run.rerequested', async (context: Context<Webhooks.WebhookPayloadCheckRun>) => {

    // Get the config from the repo that this bot installed
    // & if does not exist or malformed fallback to deafult one
    // .github/vale-linter.yml
    let config:IConfig = await context.config('vale-linter.yml', defaultConfig);

    if(config.Vale.Enabled === false){
      app.log('Vale Linter Config Disabled. Exiting Early');
      return;
    }

    // When we re-request. Lets create a NEW check
    // Thus the Check Run Created WebHook will fire below..

    const repoOwner = context.payload.repository.owner.login;
    const repoName = context.payload.repository.name;
    const headSha = context.payload.check_run.head_sha;

    await context.github.checks.create({
      name: 'Markdown Doc Lint',
      owner: repoOwner,
      repo: repoName,
      head_sha: headSha
    });

  });

  // Listen for when check is created from PR opened or PR Sync'd events
  app.on(['check_run.created'], async (context: Context<Webhooks.WebhookPayloadCheckRun>) => {

    // Get the config from the repo that this bot installed
    // & if does not exist or malformed fallback to deafult one
    // .github/vale-linter.yml
    let config:IConfig = await context.config('vale-linter.yml', defaultConfig);

    app.log('CONFIG', config);

    if(config.Vale.Enabled === false){
      app.log('Vale Linter Config Disabled. Exiting Early');
      return;
    }

    app.log('Reacting to a Check Run Created payload');

    //TODO: Need to 100% verify the check is for us & not another tool/servic
    //Think we can verify in either
    //context.payload.check_run.app.id
    //context.payload.check_run.app.name

    const repoOwner = context.payload.repository.owner.login;
    const repoName = context.payload.repository.name;
    const checkRunId = context.payload.check_run.id;

    const startTime = new Date();

    app.log('repoOwner', repoOwner);
    app.log('repoName', repoName);
    app.log('CheckRun', checkRunId);

    app.log('GitHub API Update Check with start time');

    // Update the check with in-progress state & date-time stamp
    // It goes KABOOM 🔥 💀 here
    await context.github.checks.update({
      owner: repoOwner,
      repo: repoName,
      check_run_id: checkRunId,
      status: "in_progress",
      started_at: startTime.toISOString()
    });

    const headSha = context.payload.check_run.head_sha;

    // Create the vale-lint folder to do our processing in & subdir for the vale YML style rules
    fs.mkdirpSync(`./vale-lint_${headSha}/files`);
    fs.mkdirpSync(`./vale-lint_${headSha}/${config.Vale.Paths.Styles}`);

    // Download _vale.ini from master branch
    const valeFile = await context.github.repos.getContents({ owner: repoOwner, repo: repoName, path: config.Vale.Paths.Configuration });
    const valeRawData = valeFile.data.content;
    const valeFileContent = Buffer.from(valeRawData, "base64").toString();

    // Save _vale.ini file
    await fs.writeFile(`./vale-lint_${headSha}/${config.Vale.Paths.Configuration}`, valeFileContent);
    app.log(`Written ./vale-lint_${headSha}/${config.Vale.Paths.Configuration} file`);


    // Get all files from 'vale/DocStyles' in repo root from master/default branch
    const valeStyles = await context.github.repos.getContents({ owner: repoOwner, repo: repoName, path: config.Vale.Paths.Styles });
    const valeStylesItems:Array<any> = valeStyles.data;

    for(const item of valeStylesItems){
      const filePath = item.path; // 'vale/DocStyles/BadWords.yml'

      // Download each file from the master/default branch
      const blobContent = await context.github.git.getBlob({owner: repoOwner, repo: repoName, file_sha: item.sha });
      const rawData = blobContent.data.content;
      const fileContent = Buffer.from(rawData, "base64").toString();

      fs.writeFileSync(`./vale-lint_${headSha}/${filePath}`, fileContent);
      app.log(`Written ./vale-lint_${headSha}/${filePath} file from default branch`);
    }


    // Fetch the list of files in the PR from the head_sha
    // Gives us an array of files that changed & their sha's so we can get their file contents
    var tree = await context.github.git.getTree({ owner: repoOwner, repo: repoName, tree_sha: headSha });
    var treeResponse = tree.data;
    var treeData:Array<any> = treeResponse.tree; // Array of objects

    // Filter the list of items (don't want images or anything else but .md files)
    const markdownFiles = treeData.filter(path => {
      const {path: filename, type} = path
      return type === 'blob'  && filename.endsWith('.md')
    });

    // For each markdown file - go and download it & save it in 'vale-lint/files'
    for(const file of markdownFiles){
      // Download each file
      const prBlob = await context.github.git.getBlob({owner: repoOwner, repo: repoName, file_sha: file.sha });
      var rawData = prBlob.data.content;
      var fileContent = Buffer.from(rawData, "base64").toString();

      // Write file to disk
      fs.writeFileSync(`./vale-lint_${headSha}/files/${file.path}`, fileContent);
      app.log(`Written file from PR to ./vale-lint_${headSha}/files/${file.path}`);
    }

    // Call the VALE CLI tool to output results as JSON
    // We must pass --no-exit otherwise if we find an error, we dont get the result but an error saying it exited etc
    var valeLint = execSync("vale --no-exit --output=JSON .", { cwd: `./vale-lint_${headSha}/files`});
    var rawString = valeLint.toString();
    app.log('Raw JSON string', rawString);

    // Example JSON result
    // {
    //   "README.md": [
    //     {
    //       "Check": "DocsStyles.BadWords",
    //       "Description": "",
    //       "Line": 6,
    //       "Link": "",
    //       "Message": "'slave' should be changed",
    //       "Severity": "error",
    //       "Span": [
    //         22,
    //         26
    //       ],
    //       "Hide": false,
    //       "Match": "slave"
    //     }
    //   ]
    // }

    // Convert to a proper JSON object to query/iterate over etc
    const resultJson = JSON.parse(rawString) as IValeJSON;


    // Check the JSON is NOT an empty json object `{}`
    const hasLints = Object.entries(resultJson).length > 0 && resultJson.constructor === Object;
    app.log('Has Lints?', hasLints);

    // No Errors - so let's bail out early & POST a SUCCESS message/check result
    if(hasLints === false){
      // Finish time
      const finishTime = new Date();

      let successCheck:Octokit.ChecksUpdateParams = {
        owner: repoOwner,
        repo: repoName,
        check_run_id: checkRunId,
        status: "completed",
        conclusion: "success",
        completed_at: finishTime.toISOString()
      };

      let successOutput:Octokit.ChecksUpdateParamsOutput = {
        title: config.Vale.Success.Header,
        summary: config.Vale.Success.Message
      }

      // If we have images enabled add it into the output object that makes up the larger check object
      if(config.Vale.Success.ShowImage){

        let images: Array<Octokit.ChecksUpdateParamsOutputImages> = new Array<Octokit.ChecksUpdateParamsOutputImages>();
        images.push({
          alt: config.Vale.Success.Header,
          image_url: config.Vale.Success.ImageUrl
        });

        // Update the output object
        successOutput.images = images;
      }

      //Add the output to the main object to send back to GitHub
      successCheck.output = successOutput;

      // Update check with SUCCESS result
      await context.github.checks.update(successCheck);

      // Delete vale-lint folder
      fs.removeSync(`./vale-lint_${headSha}`);

      // Exit/finish early
      return;
    }

    // An array to store our annotations in
    const annotations:Array<ChecksUpdateParamsOutputAnnotations> = new Array<ChecksUpdateParamsOutputAnnotations>();

    // For each error create an annotation
    for (const objectKey of Object.getOwnPropertyNames(resultJson)){
      const fileName = objectKey;
      const lintItems = resultJson[objectKey];

      // The filename can contain an array of objects
      for(const item of lintItems){

        // Vale states to map to GitHub Check Annotation Level
        // error = failure
        // warning = warning
        // suggestion = notice
        var githubLevel : "notice" | "warning" | "failure"  = "notice";

        switch (item.Severity) {
          case "suggestion":
            githubLevel = "notice";
            break;

          case "warning":
            githubLevel = "warning";
            break;

          case "error":
            githubLevel = "failure";
            break;

          default:
            githubLevel = "notice";
            break;
        }

        annotations.push({
          title: item.Check,
          message: item.Message,
          annotation_level: githubLevel,
          path: fileName,
          start_line: item.Line,
          end_line: item.Line,
          start_column: item.Span[0],
          end_column: item.Span[1]
        });
      }
    }

    // NOTE: GitHub API only accepts 50 annotations
    // If we have 50+ items will need to do multiple updates to the check
    app.log('Annotations', annotations);

    // Delete vale-lint folder
    fs.removeSync(`./vale-lint_${headSha}`);

    // Finish time
    const finishTime = new Date();

    // Update check with final result
    let errorCheck:Octokit.ChecksUpdateParams = {
      owner: repoOwner,
      repo: repoName,
      check_run_id: checkRunId,
      status: "completed",
      conclusion: "failure",
      completed_at: finishTime.toISOString()
    };

    let errorOutput:Octokit.ChecksUpdateParamsOutput = {
      title: config.Vale.Error.Header,
      summary: config.Vale.Error.Message
    }

    // If we have images enabled add it into the output object that makes up the larger check object
    if(config.Vale.Error.ShowImage){

      let images: Array<Octokit.ChecksUpdateParamsOutputImages> = new Array<Octokit.ChecksUpdateParamsOutputImages>();
      images.push({
        alt: config.Vale.Error.Header,
        image_url: config.Vale.Error.ImageUrl
      });

      // Update the output object
      errorOutput.images = images;
    }

    //Add the output to the main object to send back to GitHub
    errorCheck.output = errorOutput;

    // Update check with SUCCESS result
    await context.github.checks.update(errorCheck);

  });

  // For more information on building apps:
  // https://probot.github.io/docs/

  // To get your app running against GitHub, see:
  // https://probot.github.io/docs/development/
}

Expected behavior/code For the Octokit to perform the REST API call & update the Check.

Environment

• Probot version(s): 9.2.10
• Node/npm version: Node v10.15.3 & npm 6.9.0
• OS: Windows 10

Additional context

15:31:15.389Z  INFO probot: GitHub API Update Check with start time
15:31:16.444Z ERROR event: Not Found (id=aed8ac00-83b7-11e9-9115-ea0af161e4b6)
  HttpError: Not Found
      at response.text.then.message (C:\Code-Personal\_Docker-Playground\vale-linter\probot-app\node_modules\@octokit\request\lib\request.js:56:27)
      at process._tickCallback (internal/process/next_tick.js:68:7)
  --
  event: {
    "event": "check_run.created",
    "id": "aed8ac00-83b7-11e9-9115-ea0af161e4b6",
    "installation": 940633,
    "repository": "warrenbuckley/Checks-Linting-Testing"
  }
15:31:16.447Z ERROR probot: Not Found
  HttpError: Not Found
      at response.text.then.message (C:\Code-Personal\_Docker-Playground\vale-linter\probot-app\node_modules\@octokit\request\lib\request.js:56:27)
      at process._tickCallback (internal/process/next_tick.js:68:7)
15:31:16.452Z  INFO http: POST / 500 - 1584.96 ms (id=e5ff2c20-b3c9-4ee9-986d-71408c705827)
15:31:16.455Z ERROR probot: Internal Server Error
  Error: Internal Server Error
      at Request.callback (C:\Code-Personal\_Docker-Playground\vale-linter\probot-app\node_modules\superagent\lib\node\index.js:706:15)
      at IncomingMessage.parser (C:\Code-Personal\_Docker-Playground\vale-linter\probot-app\node_modules\superagent\lib\node\index.js:916:18)
      at IncomingMessage.emit (events.js:194:15)
      at IncomingMessage.EventEmitter.emit (domain.js:441:20)
      at endReadableNT (_stream_readable.js:1125:12)
      at process._tickCallback (internal/process/next_tick.js:63:19)

[warrenbuckley]

Closing this issue as embarrassingly it was a mistake by me & was not due to creating the updated check as I first thought, but trying to get a file with the GitHub Octokit rest.js API which did not exist (typo in path from the config yml file)

However still getting the error occasionally regarding the JWT time too far in future. Will see if I can get more details on that & make sure its not a dumb thing from me first & if not I will create a new issue @gr2m

Embarrassed

[warrenbuckley]

OK I can not reproduce the error with the JWT expiration token at boot, however I still can reproduce and run into the original error as reported @gr2m

I have pushed my rough proof of concept code to this repo here https://github.com/warrenbuckley/vale-linter

I would appreciate if you can clone this and run this as a new app yourself to see if you can help me find the root cause of this problem.

The ProBot/GitHub App I am now testing with the newly scaffolded TypeScript is https://github.com/apps/vale-linter ID: 32646

Cheers, Warren 😄

<div> GitHub</div><div>warrenbuckley/vale-linter</div><div>A GitHub bot built with ProBot to lint PRs using the Vale tool - warrenbuckley/vale-linter</div>

<div> GitHub</div><div>Build software better, together</div><div>GitHub is where people build software. More than 36 million people use GitHub to discover, fork, and contribute to over 100 million projects.</div>