validate: warn about using non-tagged latest images

If a given user uses a certain image username/myanalysisimage always pointing to latest we will face the following problem:

• The user will run the workflow once and REANA will pull username/myanalysisimage:latest with the hash 4ac0e1872789
• The user realizes that there is something wrong in the image resulting in an error
• The user fixes it and builds a new image username/myanalysisimage:latest which is in reality a new image with hash 879952dab949
• REANA, when it creates a Kubernetes job, does not set an image pull policy so it falls back to the default IfNotPresent
  • This means that, if the user job is allocated in a node where username/myanalysisimage:latest with hash 4ac0e1872789 is present, the new 879952dab949 won’t be pulled
• The user will expect to have the new image (879952dab949) which was just pushed to the Docker registry but in reality, the analysis will be run with 4ac0e1872789 resulting in the same old error

Possible solutions:

• Set all Kubernetes to imagePullPolicy: Always, leading into a performance issue, all jobs will be delayed because a image pull needs to be performance, and what is more, the network would be overloaded
• User contacts us, admin makes an intervention: for node in $(kubectl get nodes); do ssh -i reana.pem fedora@$node 'sudo su - root docker rmi username/myanalysisimage'
• Allow users to specify -o image-force-pull as a workflow operational option when they face this problem

[tiborsimko]

(1) We need to create a new click option validate --environments where all this work (and some other tickets) will be plugged under.

(2) This issue is about creating check_environment_tags() function that will go through all runtime environment specified by the researcher and check whether it uses “latest” tag, or no tag, in which case it should return a warning.

(3) Note that the implementation will differ based on which workflow engine people use:

• Serial - https://github.com/reanahub/reana-client/pull/459
• Yadage - https://github.com/reanahub/reana-client/pull/462
• CWL

Take RooFit demo as an example.

For Serial, the environment is specified under “environment” label in “reana.yaml”. The code should read the strings from there, and check if there is “:sometag”, and if sometag is not latest.

For Yadage, the tag is specified in imagetag in workflow specs, which are read by the client after “reana.yaml” is read.

For CWL, the tag is specified under 'dockerPull` in workflow specification.

See rg 6.18.04 for location of all the places.

Implementation-wise, the check_environment_tags() function should support all the three engines. There could be three atomic functions implementing the check for each engine.

(4) We could later create test cases, since this is basically a string manipulation task, it should be relatively straightforward.

[mvidalgarcia]

Closed by https://github.com/reanahub/reana-client/pull/468