Shadow endpoints (makes `load` boilerplate unnecessary in many cases)

Describe the problem

A significant number of pages have load functions that are essentially just boilerplate:

<!-- src/routes/blog/[slug].svelte -->
<script context="module">
  /** @type {import('@sveltejs/kit').Load */
  export async function load({ params }) {
    const res = await fetch(`/blog/${params.slug}.json`);
    const { post } = await res.json();

    return {
      props: { post }
    };
  }
</script>

<script>
  export let post;
</script>

// src/routes/blog/[slug].json.js
import * as db from '$lib/db';

export function get({ params }) {
  const post = await db.get('post', params.slug);

  if (!post) {
    return { status: 404 };
  }

  return {
    body: { post }
  }  
}

The load function even contains a bug — it doesn’t handle the 404 case.

While we definitely do need load for more complex use cases, it feels like we could drastically simplify this. This topic has come up before (e.g. #758), but we got hung up on the problems created by putting endpoint code inside a Svelte file (you need magic opinionated treeshaking, and scoping becomes nonsensical).

Describe the proposed solution

I think we can solve the problem very straightforwardly: if a route maps to both a page and an endpoint, we regard the endpoint (hereafter ‘shadow endpoint’) as providing the data to the page. In other words, /blog/[slug] contains both the data (JSON) and the view of the data (HTML), depending on the request’s Accept header. (This isn’t a novel idea; this is how HTTP has always worked.)

In the example above, no changes to the endpoint would be necessary other than renaming [slug].json.js to [slug].js. The page, meanwhile, could get rid of the entire context="module" script.

One obvious constraint: handlers in shadow endpoints need to return an object of props. Since we already handle serialization to JSON, this is already idiomatic usage, and can easily be enforced.

POST requests

This becomes even more useful with POST. Currently, SvelteKit can’t render a page as a result of a POST request, unless the handler redirects to a page. At that point, you’ve lost the ability to return data (for example, form validation errors) from the handler. With this change, validation errors could be included in the page props.

We do, however, run into an interesting problem here. Suppose you have a /todos page like the one from the demo app…

import * as db from 'db';

export async function get({ locals }) {
  const todos = (await db.get('todo', locals.user)) || [];
  return {
    body: { todos }
  };
}

export async function post({ locals, request }) {
  const data = await request.formData();
  const description = data.get('description');

  // validate input
  if (description === 'eat hot chip and lie') {
    return {
      status: 400,
      body: {
        values: { description },
        errors: { description: 'invalid description' }
      }
    }
  }

  await db.post('todo', locals.user, { done: false, description });

  return {
    status: 201
  };
}

…and a page that receives the todos, values and errors props…

<script>
  export let todos;
  export let values; // used for prepopulating form fields if recovering from an error
  export let errors; // used for indicating which fields were bad
</script>

…then the initial GET would be populated with todos, but when rendering the page following a POST to /todos, the todos prop would be undefined.

I think the way to solve this would be to run the get handler after the post handler has run, and combine the props:

// pseudo-code
const post_result = await post.call(null, event);
const get_result = await get.call(null, event);

html = render_page({
  ...get_result.body,
  ...post_result.body
});

It might look odd, but there is some precedent for this — Remix’s useLoaderData and useActionData, the latter of which is only populated after a mutative request.

This feels to me like the best solution to #1711.

Combining with load

In some situations you might still need load — for example, in a photo app you might want to delay navigation until you’ve preloaded the image to avoid flickering. It would be a shame to have to reintroduce all the load boilerplate at that point.

We could get the best of both worlds by feeding the props from the shadow endpoint into load:

<script context="module">
  import { browser } from '$app/env';

  /** @type {import('@sveltejs/kit').Load */
  export async function load({ props }) {
    if (browser) {
      await new Promise((fulfil, reject) => {
        const img = new Image();
        img.onload = () => fulfil();
        img.onerror = e => reject(new Error('Failed to load image', { cause: e }));
        img.src = props.photo.src;
      });
    }

    return { props };
  }
</script>

Prerendering

One wrinkle in all this is prerendering. If you’ve deployed your app as static files, you have most likely lost the ability to do content negotiation, meaning that even though we work around filename conflicts by appending /index.html to pages, there’s no way to specify you want the JSON version of /blog/my-article.

Also, the MIME type of prerendered-pages/blog/my-article would be wrong, since static fileservers typically derive the MIME type from the file extension.

One solution: shadow endpoints are accessed via a different URL, like /_data/blog/my-article.json (hitting the endpoint directly with an Accept header would essentially just proxy to the shadow endpoint URL). App authors would need to take care to ensure that they weren’t manually fetching data from /blog/my-article in a prerendered app. In the majority of cases, shadow endpoints would eliminate the need for fetch, so this seems feasible.

Alternatives considered

The main alternative idea is #758. A similar version of this has been explored by https://svemix.com. I’m personally more inclined towards shadow endpoints, for several reasons:

• No compiler magic (e.g. tree-shaking that decides what’s server code and what’s client code) necessary
• No confusion in your typechecker about what’s in scope
• We already have endpoints, this is just an extension of them
• There’s a nice conceptual separation — data/view, server/server+client, however you want to slice it
• With co-location you’re probably going to reach a point where you start refactoring into several files anyway. With separation it’s much more likely that both data and view code will stay at a manageable size

The other alternative is to do none of this. I think that would be a mistake — the load boilerplate really is unfortunate, but moreover I don’t see any other good way to solve #1711.

Importance

would make my life easier

Additional Information

No response

[Rich-Harris]

#3679 was merged, so I’ll close this. You can see how we’ve tackled the documentation — essentially, we’re teaching shadow endpoints as the default (no mention of the word ‘shadow’), and standalone endpoints as the exception.

To @f-elix’s point:

Would there be an equivalent for layouts load functions? Something like a __layout.js endpoint that populates the layout component with the returned props?

That probably makes sense, yeah. But I think it might turn out to be a bit of a design rabbit hole because the word __layout makes very little sense in that context, since it’s really a UI word (it’s already a bit weird, since layouts can do non-UI things like redirect and populate stuff), and we probably need to come up with some better nomenclature and/or generally tidy up the conceptual landscape. Rather than try to solve all that immediately, it’s probably better to let stuff percolate for a minute and come up with proposals in a new issue.

[gkatsanos]

Sorry for posting in a closed discussion, arrived here after getting the link from the Discord server while asking for some hints on when to use load() and when to use get() (in a page endpoint). After reading the initial description, it seems to me page endpoints are a simplified version of load() - not syntactic sugar per-se, as they compile to something different, but a way to do the same thing with less code. I think this is worth clarifying in the documentation: Some clear-cut pointers on what to use and when. The question I’m trying to understand specifically is, What does load() give you access to that page endpoints don't in terms of context? (e.g. one is Server-side call, the other can be both server and client side, or access to the url params maybe or other stuff) I’d even go as far as having a table with use-cases as I predict some folks will have similar questions…

thanks!