CRITICAL: https://slate.host in production (render.com) will not finish uploads past 450mb. Everything is fine local (unless an upload fails and a bucket is bricked)

Updates

• @anurag has proven that the request timeout limit is well over what we need.
• I have updated our express server to have a 15minute timeout limit while we figure out what the problem is.
• I have expanded our memory to 2048mb from the default 512mb.
• Textile has shared their timeout limits.
• Local: 1.07 GB file upload works fine locally.
• Production: For some reason in production (render.com) 1.07 GB fails after 400mb

Original Post

I’ve opened up a conversation with render.com our provider to provide more flexibility around their request timeout, which is probably preventing many users from completing their uploads of larger files.

Since we store nothing locally on render.com, I’m just hoping to have an extended request timeout so our watermarked NodeStream (3mb) can be passed through to Textile’s servers.

API Endpoint File:

https://github.com/filecoin-project/slate/blob/main/pages/api/data/[upload].js#L27

• very simple API request, just form/multipart
• saves the success reference URL (from Textile) to our postgres DB to be picked up by another request later on.

Upload Source Code:

https://github.com/filecoin-project/slate/blob/main/node_common/upload.js

• we just work with streams, no local files get kept on render.com servers.

This has to happen on render.com because the form/multipart request that hits their servers first.

I’ve spoken to @anurag and let them know we have some flexibility on time, and just grateful for the support and quick response. ❤️

Sorry about the thrash earlier @andrewxhill @carsonfarmer, I should have checked this earlier.

[sanderpick]

Hey @anurag, Textile guy here… the only WebSocket timeouts that come to mind are from the Nginx Ingress Controller:

proxy_connect_timeout     60s;
proxy_send_timeout        3600s;
proxy_read_timeout        3600s;

See this reference for explanations on those. tl;dr, those timeouts only apply to the initial connection and successive reads/writes, not the whole request. There’s no upper limit on how long a total request can take.

[jimmylee]

Closing this issue in favor of https://github.com/filecoin-project/slate/issues/366 because I think I am getting closer to the root cause.