question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Packager: Transform Times Out on Large Json File

See original GitHub issue

Description

Try to require a large json file, such as that found in the cmu-pronouncing-dictionary module

Reproduction

const cmu = require('cmu-pronouncing-dictionary')

Here’s the actual 4.5MB file: https://raw.githubusercontent.com/zeke/cmu-pronouncing-dictionary/master/index.json

I also tried changing the json to a regular js module but hit the same issue. Here’s the log:

Bundling `index.ios.js`
  Transforming modules  ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░  99.5% (406/407)...Error: TimeoutError: transforming XXX/build/cmu.js took longer than 301 seconds.
You can adjust timeout via the 'transformTimeoutInterval' option
    at _transform.then.catch.error (XXX/node_modules/react-native/packager/src/JSTransformer/index.js:163:30)
    at process._tickCallback (internal/process/next_tick.js:103:7)
(node:36165) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 669): Error: TimeoutError: transforming XXX/build/cmu.js took longer than 301 seconds.
You can adjust timeout via the 'transformTimeoutInterval' option

Solution

Address the performance issue that is causing the processing of this file to take so long

Additional Information

  • React Native version: 0.41
  • Platform: iOS
  • Operating System: MacOS 10.11.6

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Reactions:1
  • Comments:16 (4 by maintainers)

github_iconTop GitHub Comments

6reactions
jeayecommented, Jul 31, 2017

I’ve added a test repo, showing how simple it is to kill the packager with only a few megs of compiled CLJS code: https://github.com/jeaye/react-native-packager-bug

The repo uses a CLJS macro to generate code which compiles to a few megs; following the steps in the readme will reproduce the timeout or out-of-memory error 100% of the time.

4reactions
pesterhazycommented, Jul 31, 2017

I’ve built a work-around for this long-standing packager bug. The idea is to build an empty “husk” index.ios.husk.js file, which includes requires and a placeholder. The placeholder is then replaced with the actual bundle contents in a second step. That way the packager doesn’t run into OOMs or timeouts in the transformer/whole-program-optimization phases.

This repo shows how it works: https://github.com/pesterhazy/re-natal-husk

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to manage a large JSON file efficiently and quickly - Sease
Tips and tricks to find out efficient and fast ways to manage a large JSON file in Python using real-world applications.
Read more >
Processing large JSON files in Python without running out of ...
Loading complete JSON files into Python can use too much memory, leading to slowness or crashes. The solution: process JSON data one chunk ......
Read more >
The fastest way to convert objects from large JSON file into the ...
Do you, please, know some effective ways or some nuGet packages how to convert these data in less than 4 seconds? I tried...
Read more >
Handling JSON strings and large files in Integration Server
This service gets the JSON content and converts it to a document object (IData format). Connect the output of this service to inEmps...
Read more >
Parsing a large JSON file efficiently and easily - NGDATA
It's fast, efficient, and it's the most downloaded NuGet package out there. JSON Processing API. Another good tool for parsing large JSON files...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found