Tidy e2e test Python environment

There’s various things that are pip-installed in environment.py. Ideally we shouldn’t be putting custom requirements in here. I tried removing them in #804 but it started failing some tests - looks like we need the click specification for testing on older version of kedro 0.16.

Given that the kedro requirements are going to continually keep bumping the version of click we should work out some better way of installing requirements here so that the testing environment is a clean as possible. Or maybe we should just forget about tests for 0.16 altogether?

There’s probably some other bits of tidying that would be useful here, e.g. setting PIP_DISABLE_PIP_VERSION_CHECK would be much nicer if done in the CI config, as we do on kedro.

[AntonyMilneQB]

@tynandebold I’ve re-opened this because I think the Python e2e tests still need work, even if we drop support for 0.16 - hope that’s ok.

• we should make a new e2e test for 0.17, since the current e2e test will just test latest kedro 0.18.
• now that we’ve split up the unit and e2e tests on CI it might be possible to completely remove https://github.com/kedro-org/kedro-viz/blob/main/package/features/environment.py and not use any venv at all for e2e tests
• if we can’t completely remove it, we should tidy it up to e.g. remove the custom pip installed requirements as much as possible, move PIP_DISABLE_PIP_VERSION_CHECK to CI config, etc.

[studioswong]

@AntonyMilneQB thanks for the detailed outline of this situation! I 100% agree on the point that the “officially supported kedro versions” be the same as “kedro versions we test for”.

That said, @idanov made a fair point in standup yesterday that we are supporting 2 versions of Kedro, which, given the upgrade to 0.18, it does make sense to drop support for 0.16. Also noted on the fact that Kedro 0.16 causes tests to fail on Python 3.7 and it makes sense to drop the support altogether.

Great that you brought up about this comment on the contributing guide - it’s been a while since this issue was discussed ( 14 months ago - time flies!), the main context behind this lies from a user report that mentioned Kedro-Viz not working well with older versions of Kedro ( i.e Kedro <=0.16.6), where back then we did a bunch of tests within the team ourselves to test with different Kedro 0.16 versions and concluded that Kedro-Viz will work with 0.16.6 onwards. ( you can refer to this PR here for some context, while a lot of the discussion also happened on slack).

You’re right in that it’s not a set in stone statement, it was just a result from testing where we can safely advise users that Kedro-Viz >=3.8 will work with Kedro>=0.16.6 at that time and we made the releavant announcement about this on the kedro-user channel. If we are dropping support for Kedro 0.16 altoegther, it would be good to remind our users that as well so we can prompt them to upgrade to get the latest Viz goodies ( we should definitely keep in mind to include the statement that we are dropping support for Kedro <0.17.0 in our next release announcement)