Default fetch request headers differ between Cloudflare Workers and Miniflare

When making a fetch request to an external service from a Worker, certain request headers are populated by default. The Workers runtime and miniflare send different default headers, meaning that when calling some external APIs (e.g. Github’s which requires a User-Agent header for all requests!) code that works in miniflare will fail when running on the Workers runtime.

To validate this, I set up a simple echo service running on https://http-ua-proof-of-concept.dancowell.workers.dev/ which will return a structured JSON object containing request data.

Executing the following worker in each context results in different output:

export async function handleRequest(request: Request, env: Bindings) {
  const response = await fetch(
    "https://http-ua-proof-of-concept.dancowell.workers.dev/"
  );

  const body = await response.text();

  return new Response(body);
}

const worker: ExportedHandler<Bindings> = { fetch: handleRequest };

export default worker;

Request from miniflare

{
  "method": "GET",
  "url": "https://http-ua-proof-of-concept.dancowell.workers.dev/",
  "body": "",
  "headers": {
    "accept": "*/*",
    "accept-encoding": "gzip",
    "accept-language": "*",
    "cf-connecting-ip": "126.124.253.198",
    "cf-ipcountry": "JP",
    "cf-ray": "6c6fcc948c918d01",
    "cf-visitor": "{\"scheme\":\"https\"}",
    "connection": "Keep-Alive",
    "host": "http-ua-proof-of-concept.dancowell.workers.dev",
    "sec-fetch-mode": "cors",
    "user-agent": "undici",
    "x-forwarded-proto": "https",
    "x-real-ip": "126.124.253.198"
  }
}

Request from Cloudflare

{
  "method": "GET",
  "url": "https://http-ua-proof-of-concept.dancowell.workers.dev/",
  "body": "",
  "headers": {
    "accept-encoding": "gzip",
    "cf-connecting-ip": "35.224.27.5",
    "cf-ipcountry": "US",
    "cf-ray": "6c6fdbd42d216318",
    "cf-visitor": "{\"scheme\":\"https\"}",
    "connection": "Keep-Alive",
    "host": "http-ua-proof-of-concept.dancowell.workers.dev",
    "x-forwarded-proto": "https",
    "x-real-ip": "35.224.27.5"
  }
}

Based on this output, miniflare sends 4 additional headers by default: accept, accept-language, sec-fetch-mode, user-agent

I would expect that miniflare’s default request headers should match that of the workers runtime.

[mrbbot]

Hey! 👋 I noticed undici’s fetch implementation does actually allow you to pass a custom Dispatcher, but the exported function always uses the default. I’ve switched to importing the implementation directly, and use a custom Dispatcher to remove the headers. We were already using internal imports like this, but just to be extra safe, I’ve pinned undici to an exact version too. When we need to upgrade, we can run all the tests again and update code as needed. 👍

Will be fixed in the next release. 🙂

[mrbbot]

Hey! 👋 Miniflare 2.0.0 has just been released, which includes this fix. You can find the changes since 2.0.0-rc.5 at the end of the changelog and install it with npm i miniflare -D. Please let me know if you have any other issues, and feel free to ask questions in the #miniflare channel of the Cloudflare Workers Discord server.