Bug: 404 error on retrieving /api/v1/lineage

When we manually adding a dataset and then fetch data via lineage endpoint, on UI we have “Something went wrong while fetching lineage”. When we hit lineage endpoint we expect that job already exist and trying to fetch jobId, as we didn’t have job yet (only dataset), we return 404.

Requests to reproduce:

--Create a namespace
PUT http://localhost:5000/api/v1/namespaces/postgres%3A%2F%2Flocalhost%3A6432
Content-Type: application/json

{
  "ownerName": "Me"
}

###
--Create a source
PUT http://localhost:5000/api/v1/sources/postgres%3A%2F%2Flocalhost%3A6432
Content-Type: application/json

{
  "type": "DB_TABLE",
  "connectionUrl": "postgres://localhost:6432"
}

###

--Create a dataset
PUT http://localhost:5000/api/v1/namespaces/postgres%3A%2F%2Flocalhost%3A6432/datasets/dvdrental.public.actor_info
Content-Type: application/json

{
  "type": "DB_TABLE",
  "physicalName": "dvdrental.public.actor_info",
  "sourceName": "postgres://localhost:6432",
  "fields": [
    {
      "name": "value",
      "type": "string",
      "nullable": true,
      "metadata": {}
    }
  ]
}

###
--Retrieve info
GET http://localhost:3000/api/v1/lineage/?nodeId=dataset:postgres://localhost:6432:dvdrental.public.actor_info

404 response 

[collado-mike]

I think the core issue is here: https://github.com/MarquezProject/marquez/blob/main/api/src/main/java/marquez/service/LineageService.java#L40-L43 . The LineageService only queries jobs to determine lineage- if the node in the query param is a dataset, it’ll find the first job connected to that dataset, then determine lineage. If there’s a dataset with no jobs, it’ll throw that NodeIdNotFoundException, which will end up returning a 404.

[phixMe]

It appears that there are some special characters that can cause us problems with the way parse out the nodeId from the query parameters.

We won’t be able to transmit this http://localhost:3000/api/v1/lineage/?nodeId=dataset:postgres://localhost:6432:dvdrental.public.actor_info as is because we have no way of knowing what is a delimiter and what is part of the dataset name or namespace.

I will open up an issue to fix this in our web project that will encode the segments of the nodeId that contain names.