How to make the documentations works for `kedro-datasets`?

Introduction

This is part of the effort of splitting out kedro.extras.datasets to kedro-datasets

Background

How will docs work for kedro-datasets? We will need to generate documentation (at least the dataset part) from the kedro-datasets repo. What will be the CI/CD process to generate it?

There are also more subtle questions like:

• What does latest or stable means when we have two repositories in the doc?

Potential Solutions

1. Having a dedicated doc repository that add kedro and kedro-datasets into a monorepo - http://fmarco76.github.io/git and related services/readthedocs. This is quite complicated but seems to be the more obvious choice when I Google around.
2. Split the doc into two ReadTheDocs projects - may be easier technically, but not ideal for user experience and diverge our visits into two places.

Both of these solutions are far from ideal, is there any popular libraries doing similar things?

[SajidAlamQB]

The above 3 solutions were discussed in the technical design session, and we have decided to:

• Implement solution 2, use .readthedocs.yml hooks to copy over ONLY the latest release of kedro-datasets to kedro.datasets just before the document generation step on RTD. This means that our latest and stable tags will be the same for the kedro-datasets API docs. We agreed on this being a temporary solution as we will be moving away from RTD and Sphinx. We’ll mitigate the issue of the kedro-datasets docs falling behind by having frequent releases.

• We will copy just latest release of kedro-datasets on CI, and test that the docs still pass.

[merelcht]

The above solution was discussed in Tech Design on 14/9 and everyone agreed that it sounds like a good plan. We should:

• Ensure that as much of the setup as possible is automated
• Do an experiment to see if we can “have a note of which version of docs for kedro-datasets is being used in the kedro docs.” Above this is described as a nice to have, but in the discussion we agreed this is quite important and we should make an effort to make it work.