`$page.data` friction - allow parallel loads and "waterfall" data merges

Describe the problem

Presumably, one of the main benefits to the new load architecture, specifically around the await parent() feature, is to allow all loads to run in parallel unless they explicitly don’t need to. Also presumably, the most common source of “long” loads are network calls. If all of my loads are blazingly fast – as the kids like to say – waterfalling isn’t that big of a deal, but if I throw a few network calls in there, suddenly that whole “running in parallel” thing is pretty awesome.

Unfortunately, I’m finding that I often have to throw the “running in parallel” benefits out the window because I need to be able to deeply merge data. For example, if I have a layout key in data which itself is an object being listened to by the root +layout.svelte through $page.data:

// /routes/foo/+layout.ts
export const load: LayoutLoad = () => {
  return {
    layout: {
      showFooter: true,
    }
  }
} 

// /routes/foo/bar/+layout.ts
export const load: LayoutLoad = () => {
  const things = await some_stuff_we_dont_want_to_happen_serially();

  return {
    layout: {
      showPotato: true,
    }
  }
}

// /routes/+layout.svelte
<script lang="ts">
  import { page } from '$app/stores';
</script>

<Potato show={!!$page.data?.layout?.showPotato} />
<slot />
<Footer show={!!$page.data?.layout?.showFooter} />

See the footgun? /routes/foo/bar/+layout.ts will completely overwrite the layout key in data because it’s shallow-ly merged. In order to fix it, we have to change it to:

// /routes/foo/bar/+layout.ts
export const load: LayoutLoad = async ({ parent }) => {
  const things = await some_stuff_we_dont_want_to_happen_serially();
  const { layout } = await parent();

  return {
    layout: {
      ...layout,
      showPotato: true,
    }
  }
}

But, ugh, now that load has to await parent when all I want to do is deeply merge the objects! The load itself doesn’t actually have any dependency upon the parent at all.

Describe the proposed solution

Thankfully, the solution is pretty simple. I can’t take credit – @Rich-Harris came up with it when we were whiteboarding in the maintainers’ chat. He’s smart like that. The solution is to allow data resolution to occur after all loads have finished. Rendering doesn’t start until all loads have resolved anyway, so there’s no issue here. Essentially, we allow loads to run in parallel, but data resolution to run as a (partial or complete) waterfall afterwards (if a developer needs it to):

// /routes/foo/bar/+layout.ts
export const load: LayoutLoad = () => {
  const things = await some_stuff_we_dont_want_to_happen_serially();

  // data, in this case, is the merged contents of all datas prior to this `load`, including any `load`s that also return this kind of function
  return (data) => ({
    layout: {
      ...data.layout,
      showPotato: true,
    }
  });
}

This allows the load to run in parallel, but for its returned data to depend upon prior loads’ data (which is returned, presumably, after all of their time-consuming network activities have completed).

Alternatives considered

Just get over it and deal with shallow-ly merged $page.data forever. Not a big fan. 😆

Importance

would make my life easier

Additional Information

No response

[aloker]

Looks interesting, trying to follow. What is an API waterfall? I guess this is running requests in parallel and getting a lot more data [water] quicker?

A waterfall in this context means that tasks are executed one after another and not in parallel - like a waterfall where the water strictly runs from top to bottom. In the context of Svelte Kit this can occur when parent() is called before other fetch calls inside a load function. The result of the parent call is then fully awaited before the async work in this load function is started.

Semantically, this is typically never wrong. But it is a missed opportunity for optimization by parallelization. If two async calls need to be made and the inputs of the calls do not depend on the outcome of either call, they should be started in parallel. This causes better resource usage. In the context of load functions this means: call ( or at least await) parent as late as possible.

[aradalvand]

Not sure how relevant this is but just wanted to let you know that the SEO section also needs an update, specifically the orange box 👇: