Improve GitHub => JIRA reliability

When an event occurs on GitHub, it is sent to JIRA.

+----------+           +---------------+          +----------+
|          |  webhook  |               |   API    |          |
|  GitHub  ----------->|     JIRA      |---------->   JIRA   |
|          |           |  Integration  |          |          |
+----------+           |               |          +----------+
                       +---------------+                      

If delivery fails, either on it’s way to the integration or to JIRA, the event data is lost.

For example, if a customer’s JIRA instance is in maintenance mode when a commit is pushed, the JIRA issues mentioned won’t be associated back to the commit.

Because there is no retry or visibility into these failures, this leaves the user wondering why their commit isn’t appearing in JIRA.

If a webhook is missed, the integration should be able to recover.

Potential solutions:

1. Manually retrigger. Provide button to user, allowing them to manually resend the event.
2. Recurring sync. Sync hourly from last cursor point. Downside: this would overwrite “files changed” data and may not trigger transitions. This could be mitigated by checking if the data was present before writing it during the sync using the existsByProperties endpoint.
3. Retry failed webhooks. Use the delivery API to retrigger missed webhooks.

Recommended solution

If schedules allow, my preferred solution is option 3. It would be relatively straightforward to implement and would ensure the data that reaches JIRA is of the highest quality (some data, like “files changed” isn’t available via the sync).

If option 3 isn’t feasible, option 2 seems like the next best choice for a few reasons: (a) it isn’t blocked by other work, like option 1 and 3, (b) it doesn’t require interaction from the user, © it leverages the existing sync code. Ensuring we don’t override existing data introduces some complexity.

[jnraine]

@scepticulous option 1 won’t be possible for a few months yet.

Did you mean “If Option 3 isn’t feasible”?

Oops. I got mixed up. Fixed!

---

I’m going to pause this until next week when we get timelines for webhook delivery API.

[rachellerathbone]

As it has been almost 2 months with no response, closing this issue.