question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Feature: Use with folder partitioning

See original GitHub issue

I wonder if I could use this static file middleware for my situation. I need to store file uploads in a folder called blobs and since I expect a lot of files they should not be stored in the same directory. Instead I want to partition the blobs directory with 3 subdirectories based on the files content hash:

blobs/hash{0,3}/hash{3,6}/hash{6,9}/hash.ext

requests the like /hash should then fetch the file from this partitioned directory. Also after creating the file I’d like to immediately update the sirv cache with this new file. Precompressing the file to .gz and brotli would be nice too.

Any ideas if that could be accomplished?

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
lukeedcommented, May 4, 2020

Right; Node is not the best solution for file-serving. Basically sirv only exists so that people who are trying to do everything inside Node.js can still have a decent option.

Personally I use sirv for development and running production-like servers locally.

0reactions
pkecommented, May 4, 2020

Yeah, I really miss some nginx + node tutorials where they explain that serving static files with node is maybe not a good idea. Maybe I’ll write one myself when I am done.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Hands-On Tutorial: Advanced Partitioning: File-Based Using ...
Learn how to use partition redispatch with file-based data in this Dataiku tutorial.
Read more >
How does a crawler determine when to create partitions?
When an AWS Glue crawler scans Amazon S3 and detects multiple folders in a bucket, it determines the root of a table in...
Read more >
Data File Partitioning and Advanced Concepts of Hive
Using the partitioning feature of HIVE that subdivides the data, HIVE users can identify the columns, which can be used to organize the...
Read more >
Spark Partitioning & Partition Understanding
Spark/PySpark partitioning is a way to split the data into multiple partitions so that you can execute transformations on multiple partitions in parallel....
Read more >
Data partitioning guidance - Azure Architecture Center
Data access operations on each partition take place over a smaller volume of data. ... based on cost and the built-in features that...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found