Refactor the resolver to take a list of parsed requirements

What’s the problem this feature will solve? Currently, the (legacy) resolver takes a list of InstallRequirement objects, which are in an odd state compared to the later parts of pip’s code. They contain a bunch of config information (option flags, etc) but don’t necessarily refer to an actual distribution file (they can be something like pip >= 20.0). The resolver puts these requirements into a RequirementSet and then mutates them as it works through the resolution, preparing them, etc.

The problem here is that it means that the InstallRequirement object has an uncertain status - it is mostly (in particular, in later parts of the processing) expected to contain a link to a distribution, but early on it can be a “requirement specifier style” object that has no actual distribution behind it (and so, for example, can’t be prepared).

Describe the solution you’d like Make the resolver take ParsedRequirement objects, which are what the argument parsing code produces and which don’t have any role in “post resolution” code. This will more completely decouple the classes used by “pre-resolver” code and “post-resolver” code.

The new resolver could then take ParsedRequirement objects as the basis of the resolvelib Requirement objects and InstallRequirement objects as the basis of the resolvelib Candidate objects, and not have to deal with the dual nature of InstallRequirement.

Alternative Solutions The current prototype of the new resolver tries to work with InstallRequirement objects for both phases, but it is hitting errors which appear to be a result of being unable to correctly manage the state changes of InstallRequirement objects.

Additional context Longer term, it might be useful to completely remove the state where an InstallRequirement doesn’t have a populated link. But this is likely to be superseded by a move to the “project” model. In fact, the proposed change here would possibly be useful as a first step towards the “project” model (although that isn’t an explicit goal of this refactoring).

[pfmoore]

Sorry, you’re correct. What I should have said was (1) that I don’t now believe that this refactoring is a blocker for the new resolver implementation and (2) that it’s a bigger piece of work than I’d initially assumed. We can (if my latest experiments are correct) complete the implementation of the new resolver using InstallRequirement if necessary.

This refactoring is on the critical path for moving away from InstallRequirement, but what I was intending to say is that it’s not on the critical path for the new resolver implementation. (Unless, of course, further weird behaviour of InstallRequirement crawls out of the woodwork 🙄)

[pfmoore]

Ping @uranusjr @pradyunsg - also see this comment on Zulip.

Also, @chrahunt do you have any comments - I feel like I’m re-inventing thoughts you probably had when you proposed the previous set of refactorings, so I’d appreciate your input here.