Total distance after split is higher than before split

Hello, team,

We use R5 in Beam project. It seems that split algorithm has some issue. We see that the total distance after split exceed original edge length. It seems that it is due to the double adjustment. First adjustment happens Split.java#L185 and then it happens again in Split.java#L209-L210

Here is the output from the run to build the network on https://github.com/conveyal/r5/commit/52e564f7f3aa6a34169958a0ce6bb40fbe331eea, v4.5.3 with extra patch to log some information during the split. Network and other data for R5: https://drive.google.com/open?id=1OqxwDU8JGsxGoiL6LjrfdSEKeO8hfC4j. Run it in the following way

00:00  INFO:  OsmId[50620] Original edge length: 1700.891000, total distance before: 1700.891000, total distance after: 2588.831000, ratio: 1.522044
00:15  INFO:  OsmId[50620] Original edge length: 2041.239000, total distance before: 2041.239000, total distance after: 2927.271000, ratio: 1.434066
00:15  INFO:  OsmId[50620] Original edge length: 2473.944000, total distance before: 2473.944000, total distance after: 2760.300000, ratio: 1.115749
00:15  INFO:  OsmId[50620] Original edge length: 1775.445000, total distance before: 1775.445000, total distance after: 1786.047000, ratio: 1.005971
00:15  INFO:  OsmId[50620] Original edge length: 1070.569000, total distance before: 1070.569000, total distance after: 1104.891000, ratio: 1.032060
00:15  INFO:  OsmId[50620] Original edge length: 984.855000, total distance before: 984.855000, total distance after: 1054.135000, ratio: 1.070345
00:15  INFO:  OsmId[50620] Original edge length: 715.478000, total distance before: 715.478000, total distance after: 887.050000, ratio: 1.239801
00:15  INFO:  OsmId[50620] Original edge length: 695.285000, total distance before: 695.285000, total distance after: 923.531000, ratio: 1.328277
00:15  INFO:  OsmId[50620] Original edge length: 537.778000, total distance before: 537.778000, total distance after: 776.564000, ratio: 1.444023
00:15  INFO:  OsmId[50620] Original edge length: 385.753000, total distance before: 385.753000, total distance after: 500.961000, ratio: 1.298657
00:15  INFO:  OsmId[50620] Original edge length: 376.918000, total distance before: 376.918000, total distance after: 641.994000, ratio: 1.703272
00:15  INFO:  OsmId[50620] Original edge length: 191.765000, total distance before: 191.765000, total distance after: 767.923000, ratio: 4.004500
00:15  INFO:  OsmId[50620] Original edge length: 543.048000, total distance before: 543.048000, total distance after: 1297.824000, ratio: 2.389888
00:15  INFO:  OsmId[50620] Original edge length: 377.388000, total distance before: 377.388000, total distance after: 1112.076000, ratio: 2.946771
00:15  INFO:  OsmId[50620] Original edge length: 677.217000, total distance before: 677.217000, total distance after: 1730.691000, ratio: 2.555593
00:15  INFO:  OsmId[50620] Original edge length: 1072.077000, total distance before: 1072.077000, total distance after: 2147.173000, ratio: 2.002816
00:15  INFO:  OsmId[50620] Original edge length: 288.079000, total distance before: 288.079000, total distance after: 729.049000, ratio: 2.530726
00:15  INFO:  OsmId[50620] Original edge length: 220.485000, total distance before: 220.485000, total distance after: 655.843000, ratio: 2.974547
00:15  INFO:  OsmId[50620] Original edge length: 132.538000, total distance before: 132.538000, total distance after: 1071.978000, ratio: 8.088080
00:15  INFO:  OsmId[50620] Original edge length: 469.720000, total distance before: 469.720000, total distance after: 1387.556000, ratio: 2.954007
00:15  INFO:  OsmId[50620] Original edge length: 47.794000, total distance before: 47.794000, total distance after: 119.000000, ratio: 2.489852
00:15  INFO:  OsmId[50620] Original edge length: 920.436000, total distance before: 920.436000, total distance after: 1714.522000, ratio: 1.862728
00:15  INFO:  OsmId[50620] Original edge length: 1222.735000, total distance before: 1222.735000, total distance after: 1979.587000, ratio: 1.618983
00:15  INFO:  OsmId[50620] Original edge length: 491.787000, total distance before: 491.787000, total distance after: 511.411000, ratio: 1.039903
00:15  INFO:  OsmId[50620] Original edge length: 509.456000, total distance before: 509.456000, total distance after: 702.128000, ratio: 1.378192
00:15  INFO:  OsmId[50620] Original edge length: 1593.850000, total distance before: 1593.850000, total distance after: 1912.392000, ratio: 1.199857
00:15  INFO:  OsmId[50620] Original edge length: 453.327000, total distance before: 453.327000, total distance after: 853.587000, ratio: 1.882939
00:15  INFO:  OsmId[50620] Original edge length: 658.614000, total distance before: 658.614000, total distance after: 1054.636000, ratio: 1.601296
00:15  INFO:  OsmId[50620] Original edge length: 547.592000, total distance before: 547.592000, total distance after: 593.720000, ratio: 1.084238
00:15  INFO:  OsmId[50620] Original edge length: 568.179000, total distance before: 568.179000, total distance after: 686.117000, ratio: 1.207572
00:15  INFO:  OsmId[50620] Original edge length: 1057.097000, total distance before: 1057.097000, total distance after: 1074.355000, ratio: 1.016326

I picked the link with OsmID=50620

[abyrd]

Hey @ansoncfit,

As mentioned by @abyrd, this may be intended behavior. We expect the distance to include the perpendicular distance from the street to the requested (off-street) point.

Thanks! For linking origins and destinations to the network, I understand – but does this also apply to the case where split is called from StreetLayer.getOrCreateVertexNear? Because that would mean that a street literally gets longer by having bus stops snapped to it. For everyone, not just for people using the bus stop. Is that intended?

In R5 and in OTP the term “snapping” is used frequently and informally but it doesn’t quite match the situation. The stop is not actually moved to an assumed spot on the road, but rather connected by a perpendicular line to the closest point on the road. So I tend to think of this as projection, splitting, and linking.

So indeed the use in StreetLayer#getOrCreateVertexNear looks incorrect. When it splits the edge, it shortens the existing edge and adds a new edge to represent the second half after the split point, either with permanent or temporary edges depending on the case. In all these actions, split.distance0_mm and split.distance1_mm are used as if they were the distance along the road not including the perpendicular distance. However, the Javadoc on these fields states that they do include the perpendicular distance:

    /**
     * Accumulated distance from the beginning vertex of the edge geometry up to the split point (point on the edge
     * closest to the point to be linked), plus the distance from the linked point to the split point
     */
    public int distance0_mm = 0;

This problem would be made worse by the repeated re-splitting of roads (in cases where there are multiple stops along the same road) for several reasons. First, the perpendicular distance is being added in twice for every stop along the road, which is particularly bad in cases where the stop is far away from the road. Second, as @REASY points out above there are long straight-line edges that no longer follow the actual path of the road so are very far from stops. This is probably also the result of recursive edge splitting. See this comment: https://github.com/conveyal/r5/blob/25e9e7e6c816f2a7d27a2646d5e26c0946c37b82/src/main/java/com/conveyal/r5/streets/StreetLayer.java#L1181

When an edge is split, the new pieces lose their geometry. Any subsequent splitting/linking will consider the split edges to be a straight line between their endpoints.

This problematic behavior is a result of how the code evolved. Here is what I think happened here:

Initially this splitting code was only used to non-destructively associate grids or other sets of origin or destination points with existing street networks. In these cases, the final bit of travel from the edge vertices up to the destination via the split point is appended on the fly after routing is completed. This approach is still used in Conveyal Analysis, and does preclude routing from and to points on the same road segment. In various stages this same code was adapted to attach transit stops to the street network. The perpendicular distance was initially ignored (probably for the reasons outlined here), with code later adapted to account for it.

We definitely need to work on a fix for this, and also need to add some automated tests to verify that paths remain the same length after repeated splitting.

The fact that the perpendicular distance is pre-added into the distance0 and distance1 is essentially an optimization that reduces the space required for a linkage point by 1/3 and avoids some adding. This is arguably excessive optimization, and was only done because in Analysis these values are used billions of times in hotspots containing tight loops. Our fix should probably involve storing the perpendicular distance separately in the Split objects and only pre-adding it in the LinkedPointSet class, but we may want to keep it separate even there, since we’re now applying separate speeds on the perpendicular segment.

Thanks again for the report everyone!

[ansoncfit]

As mentioned by @abyrd, this may be intended behavior. We expect the distance to include the perpendicular distance from the street to the requested (off-street) point.

See also https://github.com/conveyal/r5/blob/25e9e7e6c816f2a7d27a2646d5e26c0946c37b82/src/main/java/com/conveyal/r5/streets/LinkedPointSet.java#L378