ky-universal fetch leads to body timeout errors

Bug report

I use ky-universal with Next.js to handle all my API calls, and I use ky’s hooks (middleware for APIs) to do things like logging, handling cookies, etc.

The library had a long-standing bug with timeout errors on large response bodies, and it was recently fixed when they switched their node-fetch dependency to 3.0.0-beta.6. Note that these are not actually timeouts from the API itself, but timeouts while trying to consume the response stream and parsing it as JSON. The linked PR was supposed to fix these issues (similar to mine) —

1. https://github.com/sindresorhus/ky-universal/issues/8
2. https://github.com/sindresorhus/ky/issues/135
3. https://github.com/sindresorhus/ky-universal/issues/18
4. https://github.com/sindresorhus/ky-universal/issues/15

After this change, I’d expected my hooks to work again, but I’m still getting the same errors —

FetchError: Response timeout while trying to fetch https://my-api.com/my-api (over 30000ms)
    at Timeout._onTimeout (/my-project/node_modules/next/dist/compiled/node-fetch/index.js:1:134853)
    at listOnTimeout (internal/timers.js:549:17)
    at processTimers (internal/timers.js:492:7) {
  type: 'body-timeout'
}

This leads me to believe Next.js’s own version of node-fetch is shadowing the one that ky-universal should use.

To Reproduce

Steps to reproduce the behavior, please provide code snippets or a repository:

1. See https://github.com/paambaati/ky-body-timeout-repro
2. Use https://github.com/paambaati/ky-body-timeout-repro/blob/234f5b9ca9af5de576b1b86ead650b29daaef66c/index.js#L28 and the associated api() method in a Next.js API route with ky-universal dependency set to version 0.7.0 - this version includes the fix for timeouts.
3. Hit the Next.js API route and watch it timeout for responses roughly over 50 KB in size.

Expected behavior

1. Should not error if we’re using the latest version of ky-universal.
2. Should prefer ky-universal’s node-fetch over Next.js’s own.

System information

• OS: macOS Mojave 10.14.6
• Browser Chrome & Firefox
• Version of Next.js: 9.4.4
• Version of Node.js: 14.1.0

Additional context

Somewhat related — https://github.com/vercel/next.js/issues/12761

[sholladay]

Hi all, Ky maintainer here! Happy to help with this.

Just to make sure we’re on the same page, Next has not yet released a version that uses ky-unversal v0.7.0 and it hasn’t updated its own node-fetch dependecy to avoid the clone() bug either. What are you doing to ensure the correct versions are being used?

Also, could someone enlighten me as to why Next uses node-fetch and ky-universal? The main purpose of ky-universal is that it ships polyfills for web standard APIs that some environments are missing (mainly Node). If you’re shipping your own polyfills, for whatever reason, then you should be able to use ky by itself without ky-universal.

I’m assuming that Next attaches its node-fetch dependency to the global scope, as that should be the only thing that would cause the updated node-fetch version within ky-universal to be ignored. If that’s the case, then it’s behaving as intended because we don’t want ky-universal to overwrite any existing fetch implementation. It only uses its own if it has to. We could change ky-universal so that it has a stronger bias towards its own fetch implementation, but then we’d have the opposite problem, if ky-universal had outdated dependencies, then it would be a pain. Given that fetch ought to be part of the environment in the first place, I personally think the “use our own as plan B” approach is the right way.

[github-actions[bot]]

This closed issue has been automatically locked because it had no new activity for a month. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.