Implementing a more robust issue/PR tagging system

Proposal

As I’ve been working on improving the contribution documentation, I realize that we can benefit greatly from implementing a tagging system on our issues and our PRs. I imagine that on on each issue or PR we would have the following tags.

1. A tag representing whether the issue related to a front-end (Javascript/CSS), backend (Python), or documentation problem.
2. A tag representing the difficulty level of submitting a PR on that issue. We can determine difficulty using some of these metrics.
   • The amount of Jupyter related knowledge needed to submit a PR on this
   • The amount of Javascript/Python/CSS knowledge needed to submit a PR on this
   • Other metrics probably
3. A tag representing the effort level to submit a PR for particular issue. I imagine the following divisions being issued on this.
   • easy — issues that can be knocked down in a couple of hours of work by a semi-experienced developer, very similar to the sprint-friendly tag that we use now
   • medium — issues that require a bit more time than the easy level issues and are for beginners or semi-frequent contributors who want a challenge
   • hard — issues that are probably best addressed by someone who has a lot of experience with the code base

In line with this, I think we should avoid using the sprint-friendly tag to represent issues that are easy to tackle. If I am a new contributor, I don’t really understand what sprint-friendly means in the context of difficulty but a tag like difficulty: easy is much easier to understand.

There is room for more tags to be included but I think the three below cover the basics.

There are some pros and cons to implementing this tagging system as outlined below.

Cons

• More time has to be dedicated to organizing and tagging issues from the owners of each repo
• It would be difficult to maintain consistency in the way that we tag difficulty an effort level as it depends on the person tagging

Pros

• It shows new contributors (and existing ones) that we care about making the issues approachable for them
• It will help us figure out how we are allocating our manpower (for example, is Safia taking too many of the easy issues when she can tackle some more difficult ones? Should Steve focus on front-end issues while Bob focuses on back-end?) so that we can work through things more efficiently
• Will help first time contributors (and existing ones) navigate the issues and hopefully increase the amount of community contributions

And I’m sure there are other pros and cons but I can’t think of them at the moment. I’m welcome to hearing them though!

EDIT: Added a pro

cc/ @minrk @Carreau @ellisonbg @fperez or anyone else who should be included in the discussion

[Carreau]

The naming of our tags to do that is maybe not good enough. For a few years we’ve been using “Sprint-Friendly” (https://github.com/search?q=org%3Ajupyter+label%3Asprint-friendly). We use to have also “quickfix” that we merged with sprint-friendly.

We’ll try to do a better job at tagging.

[betatim]

Safia just pointed me here. I would welcome a “easy” or “contributor needed” tag. Pretty much -0 on all other tags.

If you want people to start contributing you need to put out some starting points, and then mentor them. The main purpose of the easy issues is to get people hooked on how fun it is to contribute. Mostly I think it is helping people to get over being nervous about screwing up in public.

Some fraction of newcomers will stay for good. It means that the core-devs end up writing less code and doing more reviewing/mentoring/community building. It can work extremely well but costs some effort.

I got my start in open-source with scikit-learn and because it was a success I think they do a pretty good job (confirmation bias anyone? 😃). Andy (and the other devs) spends a large amount of time creating easy issues and then mentoring people when they pick them up. Pretty sure he could fix them more quickly than he can create the issues. If you are there for questions it is also not a problem if the “easy” issue turns out not to be easy. If the PR gets abandoned you can easily pick it up and finish it (in scikit-learn some other new comer might come along and pick it up)

I don’t think that the easy issues are just for people learning to code, but also for people like me who are fairly confident but don’t know the “style” (both in a technical and personality sense) of the project. Learning your way around a new code base takes commitment and effort. Sweeten the deal by giving people a sense of achievement (a merged PR, even if it is an easy one). I’d love to contribute more to jupyterhub, but right now I only send tiny patches when something is broken for me.

I’ll go out on a limb and claim that most (easy) issues don’t need urgent (a few days) repair. So it is worth leaving them out there as a “trap”. Once you get a reputation for having issues for newcomers you end up with the state of scikit-learn were it is virtually impossible to find an easy issue that doesn’t have a PR on the go already.