Improve running kedro as part of an automated workflow

Description

When any kedro command is executed for the first time or in a clean environment (which is often the case in CI/CD) the telemetry prompt gets run. The user then has to answer Yes or No to running telemetry. If no user is involved, e.g. in an automated CI/CD workflow a hack needs to be put in place to programatically add a .telemetry file.

Possible Implementation

The above hack works, but isn’t well known. An alternative way of solving this issue is by running the command in a form like: yes | kedro new. We need to document how people can accept/deny the telemetry tracking automatically.

[deepyaman]

In the past I think we’ve been regarding CI runs as contaminating our telemetry data, so probably it’s easiest and safest from the risk perspective to just disable telemetry in this case. But that does mean that in future if we do actually want to look at CI telemetry data we wouldn’t have anything available on it…

Another thought I had this morning was to enable telemetry by default in CI environments, but to be even more conservative in what’s collected by kedro-telemetry in those cases. Not sure that’s necessary, though; I feel like it’s quite conservative by default (but I’m also not that familiar with it).

I would advocate as capturing the CI variable in telemetry regardless of whether the default is to have it on or off.

[merelcht]

We discussed this issue in a Technical Design session and decided on the following:

• We do want to capture telemetry for projects run on CI. This gives us insights into how many Kedro projects are run as part of an automated CI workflow, which in turn will help us understand how many Kedro projects are part of production systems.
• Use the CI environment variable to indicate that data comes from a CI environment
• The user still needs to be allowed to decide whether to run telemetry on their CI Kedro projects or not, so we will not implement a system to automatically track the data if it’s part of a CI run.
• We need to document how a user can automatically give/deny consent to telemetry, by describing how to programatically create a .telemetry file
• We want to implement some logic in the kedro new command that detects if it’s being run in CI and if so, whether consent has been given/not and if not then it will advice the user on how to do this.

To do:

• Update kedro-telemetry to use the CI environment variable to indicate that data comes from a CI environment
• Document how a user can automatically give/deny consent to telemetry, by describing how to programatically create a .telemetry file
• Implement logic in the kedro new command that detects if it’s being run in CI and if so, whether consent has been given and if that’s not the case then it will advice the user on how to do this.