question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

row.meta.cursor in step always same value when parsing in worker mode

See original GitHub issue

Not sure if this is a bug or simply a limitation of parsing in worker mode, but when I parse a (local) file in worker mode, the row.meta.cursor always points to the end of the file:

Papa.parse(file, {
  header: true,
  worker: true, // << worker is true
  step: (row, parser) => {
    console.log(row.meta.cursor, file.size);
    // => logs "677403 677403" 8973 times
  },
});
Papa.parse(file, {
  header: true,
  worker: false, // << worker is false
  step: (row, parser) => {
    console.log(row.meta.cursor, file.size);
    // => logs increasisng number upto filesize as expected
  },
});

Is there any way to get the cursor (or anything else that can be used as a progress-indicator) when parsing in worker mode?

Issue Analytics

  • State:open
  • Created 6 years ago
  • Reactions:2
  • Comments:8

github_iconTop GitHub Comments

2reactions
billsomencommented, Jul 20, 2020

OMG, since 2019, no one has address this issue. Its pretty annoying.

1reaction
marabesicommented, Jul 18, 2019

I don’t know if this is the case BUT, I got the cursor progress + worker enable working enabling the chunk option.

The following example was giving me always the same cursor value as described by @sandergarretsen :

  Papaparse.parse(file, {
    header: true,
    worker: true,
    skipEmptyLines: true,
    step: (result, parser) => {
      console.log(result.meta.cursor) // always the same value
      stepCallback(result.data, parser)
    },
    complete: (result, file) => doneCallback(result, file)
  }

Once I enabled the chunk size, it started to work a expected:

  Papaparse.parse(file, {
    header: true,
    worker: true,
    skipEmptyLines: true,
    chunkSize: 500, // add this line
    step: (result, parser) => {
      console.log(result.meta.cursor) // shows the current cursor pointer value as expected
      stepCallback(result.data, parser)
    },
    complete: (result, file) => doneCallback(result, file)
  })

The version I am using is 5.0.0, can someone confirm that?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Setting up a Papa Parse progress bar with Web workers
The progress bar is updated, but only after the CSV file is parsed and the site is set up with data, so the...
Read more >
Basic Normalization | Airbyte Documentation
When you run your first Airbyte sync without the basic normalization, you'll notice that your data gets written to your destination as one...
Read more >
Query Processing Architecture Guide - SQL Server
Row mode execution is a query processing method used with traditional RDBMS ... If only a few rows with specific key values are...
Read more >
Browser Test Steps - Datadog Docs
Select any MFA global variables defined in Synthetic Monitoring Settings. This type of global variable stores time-based one time password (TOTP) secret keys, ......
Read more >
KeyEvent - Android Developers
getAction() value: multiple duplicate key events have occurred in a row, ... This mask is used to check whether one of the ALT...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found