Proposal: Hypermedia and linking between resources

Bitbucket’s API uses HAL-style links to make things more discoverable. An example of this is how a repository object embeds a link to its list of pull request and a pull request object links to its comments and reviewers.

Embedding links has a number of disadvantages though.

Limitations of Custom Links in Schema Objects

As our API grows, so does the number of associated resources and by extension the number of links embedded in every response, which has a compute and bandwidth cost. The extensive list embedded in our repository object reflects this:

  "links": {
    "watchers": {
      "href": "https://api.bitbucket.org/2.0/repositories/evzijst/interruptingcow/watchers"
    },
    "branches": {
      "href": "https://api.bitbucket.org/2.0/repositories/evzijst/interruptingcow/refs/branches"
    },
    "tags": {
      "href": "https://api.bitbucket.org/2.0/repositories/evzijst/interruptingcow/refs/tags"
    },
    "commits": {
      "href": "https://api.bitbucket.org/2.0/repositories/evzijst/interruptingcow/commits"
    },
    "clone": [
      {
        "href": "https://bitbucket.org/evzijst/interruptingcow.git",
        "name": "https"
      },
      {
        "href": "ssh://[e-mail]/evzijst/interruptingcow.git",
        "name": "ssh"
      }
    ],
    "self": {
      "href": "https://api.bitbucket.org/2.0/repositories/evzijst/interruptingcow"
    },
    "html": {
      "href": "https://bitbucket.org/evzijst/interruptingcow"
    },
    "avatar": {
      "href": "https://bitbucket.org/evzijst/interruptingcow/avatar/32/"
    },
    "hooks": {
      "href": "https://api.bitbucket.org/2.0/repositories/evzijst/interruptingcow/hooks"
    },
    "forks": {
      "href": "https://api.bitbucket.org/2.0/repositories/evzijst/interruptingcow/forks"
    },
    "downloads": {
      "href": "https://api.bitbucket.org/2.0/repositories/evzijst/interruptingcow/downloads"
    },
    "pullrequests": {
      "href": "https://api.bitbucket.org/2.0/repositories/evzijst/interruptingcow/pullrequests"
    }
  }

Another downside is that the purpose of these links is invisible to Swagger clients. This means that even though a repository links to its pull requests, a Swagger code generator would not automatically add a method .getPullRequests() to the generated Repository class.

Adding Hyper Linking to Open API

Support to express relationships between resources could be added to Open API in a number of ways:

1. Formalize the notation of links embedded in schema objects
2. Instead of link standardization, declare the resource relationships

Option 1 would make it possible to declare the shape and purpose of the links in the Open API schema so that code generators could take advantage of them at runtime. However, it would not reduce the overhead of embedding.

Option 2 is akin to the approach followed by the JSON Hyper-Schema draft. It would eliminate the need to embed any link information at runtime, while still allowing code generators to be aware of the resource relationships and generate appropriate functions.

Suggested Hyper Link Declaration Scheme

While the concept of the expired JSON Hyper-Schema is good, it is aimed at use in JSON Schema and does not apply well to Open API schemas. Instead, we suggest an extension to Open API’s SchemaObject to declare links by adding an element links that contains an array of relationships to resource endpoints (operations, in Swagger lingo).

A link entry refers to an operationId and provides the values to use in the target path’s URI template. The parameter values are templates that use curlies for variables. The variables themselves are JSON pointers to properties of the schema object itself.

For instance, in Bitbucket, a repository object can link to its pull requests like this:

  "definitions": {
    "repository": {
      "type": "object",
      "properties": {
        "slug": {
          "type": "string",
        },
        "owner": {
          "$ref": "#definitions/user"
        }
      },
      "links": [
        {
          "rel": "pullrequests",
          "href": {
            "operation": "getPullRequestsByRepository",
            "params": {
              "username": "{owner/username}",
              "slug": "{slug}",
              "state": "open"
            }
          }
        }
      ]
    }
  }
}

This provides a code generator with sufficient information to add a getPullRequests() method to its generated repository class. This method would not take any arguments, as the URL is constructed on method invocation time using the repository object’s properties.

Since the link points to an operationId, it is clear what schemas are associated with it and so the code generator can use the correct return type in the method declaration. Similar for any POST/PUT request body object.

Example

Following is a more thorough model showing the relationships between repositories, users and pull requests in Bitbucket’s API:

{
  "paths": {
    "/2.0/users/{username}": {
      "parameters": [
        {
          "name": "username",
          "type": "string",
          "in": "path"
        }
      ],
      "get": {
        "operationId": "getUserByName"
      },
      "responses": {
        "200": {
          "Schema": {
            "$ref": "#definitions/user"
          }
        }
      }
    },
    "/2.0/repositories/{username}/{slug}": {
      "parameters": [
        {
          "name": "username",
          "type": "string",
          "in": "path"
        },
        {
          "name": "slug",
          "type": "string",
          "in": "path"
        }
      ],
      "get": {
        "operationId": "getRepository",
        "responses": {
          "200": {
            "schema": {
              "$ref": "#definitions/repository"
            }
          }
        }
      }
    },
    "/2.0/repositories/{username}/{slug}/pullrequests": {
      "parameters": [
        {
          "name": "username",
          "type": "string",
          "in": "path"
        },
        {
          "name": "slug",
          "type": "string",
          "in": "path"
        },
        {
          "name": "state",
          "type": "string",
          "enum": ["open", "merged", "declined"],
          "in": "query"
        }
      ],
      "get": {
        "operationId": "getPullRequestsByRepository",
        "responses": {
          "200": {
            "schema": {
              "type": "array",
              "items": {
                "$ref": "#definitions/pullrequest"
              }
            }
          }
        }
      }
    },
    "/2.0/repositories/{username}/{slug}/pullrequests/{pid}": {
      "parameters": [
        {
          "name": "username",
          "type": "string",
          "in": "path"
        },
        {
          "name": "slug",
          "type": "string",
          "in": "path"
        },
        {
          "name": "pid",
          "type": "string",
          "in": "path"
        }
      ],
      "get": {
        "operationId": "getPullRequestsById",
        "responses": {
          "200": {
            "schema": {
              "$ref": "#definitions/pullrequest"
            }
          }
        }
      }
    },
    "/2.0/repositories/{username}/{slug}/pullrequests/{pid}/merge": {
      "parameters": [
        {
          "name": "username",
          "type": "string",
          "in": "path"
        },
        {
          "name": "slug",
          "type": "string",
          "in": "path"
        },
        {
          "name": "pid",
          "type": "string",
          "in": "path"
        }
      ],
      "post": {
        "operationId": "mergePullRequest",
        "responses": {
          "204": {}
        }
      }
    }
  },
  "definitions": {
    "user": {
      "type": "object",
      "properties": {
        "username": {
          "type": "string"
        },
        "uuid": {
          "type": "string"
        }
      },
      "links": [
        {
          "rel": "repositories",
          "href": {
            "operation": "getRepositoriesByOwner",
            "parameters": {
              "username": "{username}"
            }
          }
        }
      ]
    },
    "repository": {
      "type": "object",
      "properties": {
        "slug": {
          "type": "string",
        },
        "owner": {
          "$ref": "#definitions/user"
        }
      },
      "links": [
        {
          "rel": "self",
          "href": {
            "operation": "getRepository",
            "params": {
              "username": "{owner/username}",
              "slug": "{slug}",
            }
          }
        },
        {
          "rel": "pullrequests",
          "href": {
            "operation": "getPullRequestsByRepository",
            "params": {
              "username": "{owner/username}",
              "slug": "{slug}",
              "state": "open"
            }
          }
        }
      ]
    },
    "pullrequest": {
      "type": "object",
      "properties": {
        "id": {
          "type": "integer"
        },
        "title": {
          "type": "string"
        },
        "repository": {
          "$ref": "#definitions/repository"
        },
        "author": {
          "$ref": "#definitions/user"
        }
      },
      "links": [
        {
          "rel": "self",
          "href": {
            "operation": "getPullRequestById",
            "parameters": {
              "username": "{repository/owner/username}",
              "slug": "{repository/slug}",
              "pid": "{id}"
            }
          }
        },
        {
          "rel": "merge",
          "href": {
            "operation": "mergePullRequest",
            "parameters": {
              "username": "{repository/owner/username}",
              "slug": "{repository/slug}",
              "pid": "{id}"
            }
          }
        }
      ]
    }
  }
}

[DavidBiesack]

Nice proposal. I agree dynamic runtime links are a better option. That is central to our HATEOAS design: the API provides links (affordances) only if an operation is available to the (authenticated) client. Not allowed to delete a resource? Then the API does not return the rel : delete link. At the beginning of a paginated collection? Then omit the rel : prev link.

I like the ability to refer to an operation by operation id. As noted in other comments, however, this alone does not allow for links to resources outside of the API.

I suggest also a top-level link-strategy annotation in OAS which tells what link representation the API uses (HAL, Collection+JSON, Atom, etc.). This is probably just a hint, but tools like Swagger UI could use this to allow navigating the resources via parsing the links in the responses and matching them to the operations. This would promote HATEOAS understanding of hypermedia APIs, rather than low-level method+url tight coupling. See my comment on #577. (So I guess I disagree with @ePaul because I think embedding the link relations in OAS is the way to go: it reveals the intended use of the API - that is, the client should be looking for links and the embedded href values in the links, in order to act on responses, rather than hard-coded transitions based on static operation+path+parameters (which is all one can do with Swagger 2.0).

Please elaborate on what params is - are they parameters for the link, or are they bindings that are substituted for the parameters on the target link? If so, I think a more appropriate name would be args or arguments (as described on SO)

Finally, some of the ideas in #445 might be useful. In our APIs, we repeat common sets of links in multiple places - for example, most simple resources have rel : self, rel : update and rel : delete – that is a (parameterized) set of links, usually differing only in the request or response content type; collections all share next, prev, first and last links, and so on. So, as #445 proposes reusable parameter groups, I think reusable link groups may also be useful. In this case, however, the reference to other operations would be more implicit via operation+path rather than explicit via an operation id.

[erikvanzijst]

Potential issues:

• After discriminator, this is yet another extension to Swagger’s SchemaObject not present in JSON Schema
• No real attention has been paid to links to external URLs
• It’s unclear how well this could support pagination links

Pagination is of particular interest, as it is a concept notoriously ill-defined and lacking in Swagger. APIs often have custom ways of dealing with it. Two common approach stand out:

1. Wrapping the response array in an envelope containing things like total size, page number, next and previous links (Bitbucket)
2. Returning a pure array and moving the pagination properties and links to response headers (GitHub)

In case of the former, the response contains the page number and so it might be possible to derive the pagination links from those values, but in case of the latter, the pagination properties lie beyond the schema.