Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Run release in CI

See original GitHub issue

We closed #826 to moving towards a multi-module project structure. We’ve introduced release.sh, but some improvements can still be made to unify the release.

Most of the current release steps are run locally. They are:

Bump and release marquez-python
Bump marquez-python dependency in marquez-airflow
Bump and release marquez-airflow
Bump development versions for the java modules
Tag main branch with X.Y.Z
Publish docker images (api, web)
Publish java modules to maven

The proposed release steps below would run in CI to release Marquez X.Y.Z. First, we would tag main branch with X.Y.Z to kicks off the CI release workflow, then:

Publish java modules to maven
Publish docker images (api, web)
Bump and release python modules

Issue Analytics

State:
Created 2 years ago
Comments:5 (5 by maintainers)

Top GitHub Comments

1reaction

mobuchowskicommented, Apr 9, 2021

Agreed, that does make sense, and bumping version before tagging, as separate manual step does fix the issues you’re talking about. I’ll change the PR.

new-version.sh <new-version> 👌

1reaction

mobuchowskicommented, Mar 30, 2021

@wslulciuc created draft PR for this: https://github.com/MarquezProject/marquez/pull/1168

What I’ve done:

Python: split test and build phases for marquez-python and marquez-airflow. Build phases are triggered only on tag (recognized by env variable set by CI - $CIRCLE_TAG. Build phases are ment to create Python wheels and put them to shared workspace. Then, release step - also, only triggered by tag - one for all python projects, just pushes wheels created in shared workspace to pypi.
Docker: least changes - basically all I had to do is script bumping version in gradle-properties and add dependency on build-web and build-api steps.
Maven: after bit of tinkering on my own I’ve found that gradle release plugin has option -Prelease.releaseVersion=version which does all the bumping for me. Found that we can sign binaries on CI using env variables. Still, couldn’t reliably test if last step works - which is publishing to oss.sonatype.org with publishing plugin. By looking at docs I don’t see how it could differ, provided that we have all the env variables set up, but that would be on you to check 😉