Developer experience repository audit from @mntnr!

👋 Hey there! This is the developer experience audit from @mntnr for this repository. I’ve added in my thoughts below, in the form of a checklist. Looking forward to seeing what you think; let’s see if we can resolve all of the open issues and make this repository shine ✨ 💖 ✨

Repository Review: probot/probot

🤖 A framework for building GitHub Apps to automate and improve your workflow

For notes on anything crossed out, look below. Note: I use [~] to mean that I have proposed a fix in a PR. I know it doesn’t render properly in Markdown, but it works pretty well otherwise for that purpose. If I think that something is fine, even if it isn’t valid according to this checklist, I’ve checked it off and included a note.

Reviewing the Repository Docs

• Is there a README?
  • Is it standardized?
    • It’s OK that it isn’t standardized, BUT I am confused by this repository, because I know that it holds the docs, but I don’t know specifically what probot/probot does. Do I install it? How do I use it in particular? I think an install and usage section would be incredibly helpful here. For instance, you have the section on using robot - but I don’t know if I need to include this library in my code or not. I am a bit confused. That means the docs could be tighter. Standardization might help with making sure that basic information is there for new users.
  • Is it spellchecked?
    • See commit:eaed8f3
• Is there a Code of Conduct?
  • Is it mentioned in the Contribute section of the README?
    • No, but it is a top-level file and in the CONTRIBUTING.md, so I think this is fine.
  • Does it reference an email address for violations?
  • Does it reference a second email address?
    • No, but I’m assuming that that is a GitHub-level email that doesn’t go directly to the Probot maintainers. This is important in case the maintainers themselves don’t adhere to the conduct and a user wants to complain. If this goes straight to @bkeepers, for instance, I would suggest adding another email address as a redundancy.
• Is there a LICENSE file?
  • Is this matched in the package.json?
  • Is the year correct? Updated to 2018. commit:3e6e2a2
• Is there a .github or docs folder?
  • It looks like you’re using the docs folder to render docs on probot.github.io, which is clutch. However, you don’t have syncing with the README for README.md, and I wonder if content could be deduplicated.
  • Is there an ISSUE_TEMPLATE.md?
  • Is there a PULL_REQUEST_TEMPLATE.md?
• Is there a CONTRIBUTING.md file?
  • There’s a lot of work which can be done here; let’s talk about it!
• Is there a CHANGELOG? No. There should be ways to automate this. I would look at standard-changelog and also keepachangelog.com for understanding why.
• Does this pass alex adequately? Run alex *.md.
• Does the repository name itself pass on http://wordsafety.com?

Process

• Can I install easily? No. I am unclear whether I ever install this repository, and there are no install instructions. It’s very possible something I am missing something, and I’d like to have a note in the README saying that you don’t need to install it.

Issues and Pull Requests

• Are there an acceptable amount of pull requests?
• Are there an acceptable amount of issues?
• Are an acceptable amount of issues less than six months old?
• Are there useful issue labels?
• Are the labels being used?
• Is there a good for beginners or up-for-grabs label?
• Is there a waiting on contributor label? I find this useful for drive-by issues in particular; when it is the contributor’s job to provide more information, this is the best signal to them and you that there’s nothing for the maintainers to do.

Bots

Note: Neither of these are necessary, but they can help with some things. Check out https://probot.github.io/ for some tools. (Note: I had this in my template for all of these checklists; keeping it here for posterity, even though it is somewhat circular in this case.)

• Are there bots enabled? Of course! 😄
• Is there a CLA?
  • Is there a bot to help with the CLA?
  • I don’t know if GitHub does this, but it doesn’t seem like you’d need it for ISC? Ask your lawyer.

Metadata

• Is there a description on GitHub?
  • Does the description match the README?
• Are the topics useful? Could have some more, maybe: automation, github, open-source, pm.
• Is there a website?
  • Does the website match the project?

Package Metadata

Note: These should apply to package.json (JavaScript), *.cabal (Haskell), metadata.yml (Perl), and gemspec files (Ruby).

• Does the description match the gh-description?
• Is there a bugs field?
• Is there a homepage field?
• Are there appropriate keywords?
  • [~] Do these match the keywords on GitHub?
• Run depcheck; do the deps make sense?
  • There are some unused dependencies. I would suggest looking at these in more depth.

TODO

This is a generally good repository as far as documentation goes, but I think there’s a lot of work to be done. Specifically, it’s unclear to me what your main README is doing as far as telling people to use probot/probot. I know that it is emphasizing probot as a tool, but I don’t know how to use this particular repo very well. Some clarification would be great. I can’t install this, because I’m confused why - that’s not ideal. Let’s work on making this messaging clearer!

As well, your Contributing doc limits interactions that are encouraged to checking out good first issues and submitting PRs; but there are loads more ways to get involved, like triaging, joining the slack, requesting features, reporting bugs, helping write docs, and so on. I would add sections for these things.

Generic

• I would add a maintainers section, to make it clear who is on the maintainers team. This helps set expectations and clarifies for the users who they can talk to.
• Add a link to your Slack! You want to engage with users there.
• Consider adding a secondary email to the Code of Conduct as a contact - someone may have an issue with you but not want to tell you directly. I know, this idea may be awkward. But you will give them an option in case they do have an option, and this may be good for the overall health of the project.
• Consider adding ISSUE_TEMPLATE.md and PULL_REQUEST_TEMPLATE.md files to your repository. It looks like you have your PRs well under control, but these may help you in the future. At the least, ask them to run the tests, first, and to read the Usage guides.
• Consider adding a CLA or asking for signed commits, if you’re worried about legality. This is generally only needed when you are building a business out of a code base. In your case, I think you’re OK.
• This audit does not cover license dependency. For that, I suggest using either licensee or an external tool like Fossa. Let me know if you want more help here.

Issues

• Add ‘discussion’ labels to longstanding issues. I think your ‘pinned’ label does this, though.
• Considering adding up-for-grabs as well as good first issue. See http://up-for-grabs.net. No reason you can’t have both!
• I label pull requests where I am waiting on the Contributor to respond waiting on contributor. This helps alleviate pressure on you to close them.

Contribute back?

This checklist is open source! If you have suggestions or think it could be better, contribute back on this GitHub repository: mntnr/audit-templates.

[gr2m]

Considering adding up-for-grabs

I’ve heard people saying that up-for-grabs might sound odd, especially to some non-native English speaking people. I prefer available. The label is configurable if we want to add the project to http://up-for-grabs.net

<div>Up For Grabs</div><div>Want to contribute to open source, but not sure where to start?</div>

[gr2m]

Is there a CHANGELOG?

Changes are in release notes: https://github.com/probot/probot/releases. Which I prefer to a file committed to the repository because we can edit it after the fact.

<div> GitHub</div><div>probot/probot</div><div>probot - 🤖 A framework for building GitHub Apps to automate and improve your workflow</div>