Feature: Use with folder partitioning
See original GitHub issueI wonder if I could use this static file middleware for my situation.
I need to store file uploads in a folder called blobs
and since I expect a lot of files they should not be stored in the same directory. Instead I want to partition the blobs
directory with 3 subdirectories based on the files content hash:
blobs/hash{0,3}/hash{3,6}/hash{6,9}/hash.ext
requests the like /hash
should then fetch the file from this partitioned directory.
Also after creating the file I’d like to immediately update the sirv
cache with this new file. Precompressing the file to .gz and brotli would be nice too.
Any ideas if that could be accomplished?
Issue Analytics
- State:
- Created 4 years ago
- Comments:5 (3 by maintainers)
Top Results From Across the Web
Hands-On Tutorial: Advanced Partitioning: File-Based Using ...
Learn how to use partition redispatch with file-based data in this Dataiku tutorial.
Read more >How does a crawler determine when to create partitions?
When an AWS Glue crawler scans Amazon S3 and detects multiple folders in a bucket, it determines the root of a table in...
Read more >Data File Partitioning and Advanced Concepts of Hive
Using the partitioning feature of HIVE that subdivides the data, HIVE users can identify the columns, which can be used to organize the...
Read more >Spark Partitioning & Partition Understanding
Spark/PySpark partitioning is a way to split the data into multiple partitions so that you can execute transformations on multiple partitions in parallel....
Read more >Data partitioning guidance - Azure Architecture Center
Data access operations on each partition take place over a smaller volume of data. ... based on cost and the built-in features that...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Right; Node is not the best solution for file-serving. Basically
sirv
only exists so that people who are trying to do everything inside Node.js can still have a decent option.Personally I use sirv for development and running production-like servers locally.
Yeah, I really miss some nginx + node tutorials where they explain that serving static files with node is maybe not a good idea. Maybe I’ll write one myself when I am done.