Consistent handling of environment variables

Describe the problem

SvelteKit doesn’t do anything special with environment variables; it delegates to Vite. This is good for the sorts of environment variables Vite deals in — VITE_ prefixed values that are replaced at build time — but less good for other kinds.

Environment variables can be static (known at build time) or dynamic (not known until run time), and private (only accessible to code that runs on the server) or public (accessible to code that runs on the server or the client):

	static	dynamic
public	✅ import.meta.env.VITE_*	❌ (1)
private	⚠️ can be done with plugins (2)	⚠️ process.env.* (3)

1. To be fair, I’m not sure what sorts of things would go in this quadrant, though I would be eager to hear examples
2. You can add something like plugin-replace to selectively replace things at build time, but this is ad hoc, undiscoverable, and involves duplication between .env files and config
3. process.env is only available in Node-based environments (e.g. adapter-node, adapter-netlify and adapter-vercel — though once the Vercel adapter starts building Edge Functions that will no longer be the case)

It would be great if we had a consistent and idiomatic way of dealing with all of these.

Describe the proposed solution

I don’t actually have any proposals to make, I just didn’t want https://github.com/sveltejs/kit/pull/4293#issuecomment-1065300764 to get lost. Whatever solution we land on should

• work with .env/.env.* files
• enable dead code elimination etc at build time
• allow autocompletion/auto-imports (e.g. we could generate typings from your .env file)
• feel consistent across the four quadrants, but with clearly understandable distinctions between them (e.g. $app/env/public vs $app/env/private or whatever)

Alternatives considered

No response

Importance

would make my life easier

Additional Information

No response

[tcc-sejohnson]

Hey @Rich-Harris,

Like I said, I’m pretty passionate about this one – so here’s a full design proposal to get people talking.

To restate the points Rich laid out in the original comment, a solution needs to:

• work with .env/.env.* files
• enable dead code elimination etc at build time
• allow autocompletion/auto-imports (e.g. we could generate typings from your .env file)
• feel consistent across the four quadrants, but with clearly understandable distinctions between them (e.g. $app/env/public vs $app/env/private or whatever)

After thinking about it for some time, I think it should also:

• be composable to some degree
• guarantee from a Kit perspective that private variables are not exposed to the client
• provide the ability to validate that the environment config is valid at startup/buildtime
• provide a relatively easy-to-follow execution path

Let’s talk about those.

Be composable to some degree

The immediate use-case I can think of is when my app needs to know an environment-dependent URL for an api:

• local: http://localhost:7071
• dev: https://asdfhjkl.azurewebsites.net
• staging: https://staging.api.mywebsite.com
• prod: https://api.mywebsite.com

Whenever I need to reference this URL within my code, I often need to change behavior based on whether I’m operating in HTTP/HTTPS. Sometimes I need to know the port. Sometimes I just need the host, etc. Sure, I can just new URL(process.env['API_URL']), but that’s annoying – I have to do it every time I need the variable. I’d much rather structure my environment config as:

API_PROTOCOL=http
API_HOST=localhost
API_PORT=7071
API_URL={API_PROTOCOL}://{API_HOST}{API_PORT ? ':' + API_PORT : ''}

There are other instances where I’d like composability (or just the ability to compose a JavaScript object out of a string config value without having to do so at every single line that uses the config value). More on this in the design section. 😺

Guarantee no client exposure of private variables

Pretty simple. Obviously, Kit can’t guarantee you don’t just create a GET endpoint that returns MY_PRIVATE_VARIABLE to the frontend, but it should be able to assert that accessing a private variable from the client is invalid.

Provide the ability to validate config at startup/buildtime

I’ve wasted more time than I probably should by forgetting to include a new environment variable in .env and then having to track down why my credentialed API call isn’t working. ☹️ It’d be nice to be able to assertively validate that the environment is what is expected at buildtime and runtime.

Provide a relatively easy-to-follow execution path

There are a lot of complicated ways to implement something like this. Let’s not do one of them.

Design

I propose the addition of a few new exports from the $app route, one new root file, startup.{js|ts}, one new argument to the CLI, environment, and four new types to the App namespace. Users can define environment variables through a .env | .env.{environment} file or through environment variables present on the system at buildtime (or at runtime, if supported by whichever adapter). Environment variables will override .env variables. At both startup and buildtime, the variables will be resolved to a JavaScript Record<string, string> and passed to startup.{js|ts}, where users can validate and compose their env (more on this later). From there, environment variables will be available as import { env, privateEnv } from '$app/{buildtime|runtime}'.

With that summary, let’s follow the “execution flow” and describe in more detail how this would work.

.env | .env.environment

Same design as everyone is used to for .env files. KEY=value, supports strings. .env.{environment} is loaded when svelte-kit {do-something} --environment={environment}. (Technically, we could reuse the Vite mode here, but given that we’re building our own environment config, I would think we would want to separate it conceptually from Vite as much as possible.)

In order to support our four quadrants, we’ll need some sort of Vite-like prefixing scheme. I’m not particularly attached to any one scheme, so I’m open for suggestions, but for the sake of this proposal, we’ll say variables prefixed with ___ are ___:

• BLDPRV, buildtime private
• BLDPUB, buildtime public
• RUNPRV, runtime private
• RUNPUB, runtime public

We could also potentially allow simply PRV or PUB to indicate that the variable should be serialized during buildtime but also be available during runtime. The more specific RUNPRV/BLDPRV would take precedence.

startup.{js|ts}

Why isn’t this file named env.{js|ts}? Well, because, we can set ourselves up to kill two birds with one stone here. (Or we can just kill one bird and call it .env.{js|ts}. Up for debate.)

This file exports four methods with the following defaults:

export const resolvePublicRuntimeEnv = (publicEnv: Record<string, string>): App.PublicRuntimeEnv => publicEnv;
export const resolvePrivateRuntimeEnv = (privateEnv: Record<string, string>): App.PrivateRuntimeEnv => privateEnv;
export const resolvePublicBuildtimeEnv = (publicEnv: Record<string, string>): App.PublicBuildtimeEnv => publicEnv;
export const resolvePrivateBuildtimeEnv = (privateEnv: Record<string, string>): App.PrivateBuildtimeEnv => privateEnv;

Each of these receives the resolved variables from .env and the environment. Users can validate the values, compose them as they wish, and return the resolved environment.

New App types

You’ll notice I referenced types from the App namespace above. I propose the following be added:

declare namespace App {
  interface PublicRuntimeEnv extends Record<string, string>;

  interface PrivateRuntimeEnv extends Record<string, string>;

  interface PublicBuildtimeEnv extends Record<string, string>;

  interface PrivateBuildtimeEnv extends Record<string, string>;
}

Like the other App types, these can be set by the user.

Edit: The types should default to Record<string, string>, but users should be able to set them to anything extending Record<string, any> (and maybe we should allow numbers as indexers as well?).

User API

The objects returned from resolveXxxXxxEnv will be available to the developer throughout the app via import { env, privateEnv } from '$app/{buildtime|runtime}'. Importing privateEnv client-side (regardless of whether any of its properties are accessed) will result in an exception.

A full example, using my previous example

So, using the previous example:

// .env
BLDPUB_API_PROTOCOL=http
BLDPUB_API_HOST=localhost
BLDPUB_API_PORT=7071

// app.d.ts
declare namespace App {
  interface PublicBuildtimeEnv {
    apiUrl: URL;
  }
}

// startup.ts
const throwError = (missingKey: string) => throw Error(`Could not find ${missingKey} in .env or environment variables.`);

export const resolvePublicBuildtimeEnv = (env) => {
  if (!env.BLDPUB_API_PROTOCOL) {
    throwError('BLDPUB_API_PROTOCOL');
  }
  if (!env.BLDPUB_API_HOST) {
    throwError('BLDPUB_API_HOST');
  }

  const port = env.BLDPUB_API_PORT ? `:${env.BLDPUB_API_PORT}` : '';
  const apiUrl = new URL(`${env.BLDPUB_API_PROTOCOL}://${env.BLDPUB_API_HOST}${port}`);
  return { apiUrl }
}

// anywhere else in my code
<script lang="ts">
  import { env } from '$app/buildtime`
</script>

// typing provided by app.d.ts!
// 'http://localhost:7071' serialized into code at buildtime
<p>{env.url}</p>

Does this meet the requirements?

• work with .env/.env.* files
  • Yes
• enable dead code elimination etc at build time
  • Maybe? I’m not super familiar with how this works.
• allow autocompletion/auto-imports (e.g. we could generate typings from your .env file)
  • Absolutely.
• feel consistent across the four quadrants, but with clearly understandable distinctions between them (e.g. $app/env/public vs $app/env/private or whatever)
  • I think so, but up to the community to decide.

(My added requirements)

• be composable to some degree
  • Yes, supports resolving arbitrary environment config from Record<string, string>
• guarantee from a Kit perspective that private variables are not exposed to the client
  • Yes
• provide the ability to validate that the environment config is valid at startup/buildtime
  • Yes
• provide a relatively easy-to-follow execution path
  • To me, the .env => startup.ts => import { env, privateEnv } from '$app/{buildtime|runtime}' pipeline is pretty darn simple.

Quick note about startup.{js|ts}

Way back when, I opened #1753 to document “startup” behavior in hooks.js. Essentially, if you’ve got server-side startup code, running it in the root of hooks.js and waiting for it to finish in handle before serving any requests is the only way to do it.

To me, this behavior feels really bad. hooks.js doesn’t feel like the place it should be done. The name isn’t right, all of the other things hooks have to do have nothing to do with startup, etc. Also, having to await startup() in handle is nothing but dead code after startup is complete, but it still has to be run for every incoming server request. Ew. Introducing this startup.js file would naturally lead to a fifth export, startup, which would receive the resolved config and run (blocking) before the “server starts” (the exact behavior would likely be adapter-specific?). I’m just taking the opportunity to introduce that idea and scaffold it while we’re looking at this.

Please rip this apart, provide your controversial opinions, and hate on my design. If you notice any typos or anything that seems to stupid to be an actual suggestion, let me know so that I can correct it.

😺

[madeleineostoja]

Not sure if this is of any use since it’s already shipped, but just wanted to give some feedback that I migrated a large project over to the new env variable handling, and I liked it a lot other than the /static and /dynamic paths for the modules. I found the names muddy and I had to read the docs closely to understand what they’re doing rather than getting it intuitively out of the box.

I think just having $env/public and $env/private for the majority use-case (static), then handling dynamic variables separately (with $env/dynamic etc) would be much more ergonomic and easier to approach.