question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[v7] Accessor derivation slow with 100k+ rows

See original GitHub issue

When i provide react-table v7 with good amount of rows(below case is with 100K rows and 15 columns, then getBy, makePathArray are taking long time to execute.

image

image

Is there anything we can do to optimize it ? I can see before calling getBy we are splitting by ., then in makePathArray we are doing map(d => String(d).replace('.', '_')), this thing seems to be redundant(or can be applicable at some other place also), is this good to remove ?

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:15 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
tannerlinsleycommented, Dec 6, 2019

In the latest version, the table now caches the pathObjects used in column accessors via the getBy function. I would love to see if and how this improves your speeds.

Another piece to note is that the debug and timing code was removed in the latest version. You will need to edit the source to get those exact timing logs back. Here’s the location of those loggers for reference: https://github.com/tannerlinsley/react-table/blob/176986a07ad8b25907fff435e1246962dba18e47/src/hooks/useTable.js#L174-L231

0reactions
fyodore82commented, Oct 6, 2020

I also experienced performance degradation on 25K+ rows. My code to measure performance is

console.log(new Date().toISOString(), ' useTable started')
const {...} = useTable(...)
console.log(new Date().toISOString(), ' useTable finished')

With columns like below

column = {
  accessor: 'some string'
  // ...
}

It takes 100+ seconds to execute useTable image

When I’ve written my own accessor function

const accessor = (pathArr: string[], originalRow: GridRow) => {
  let cursor = originalRow
  for (const path of pathArr) {
    if (cursor == null) return cursor
    cursor = cursor[path]
  }
  return cursor
}

column = {
  id,
  accessor: accessor.bind(null, pathArr),
  //...
}

useTable completes in couple of seconds image

My code may not be universal, I’ve developed it only for my project. But it may help someone else…

Read more comments on GitHub >

github_iconTop Results From Across the Web

[v7] Accessor derivation slow with 100k+ rows #1713 - GitHub
When i provide react-table v7 with good amount of rows(below case is with 100K rows and 15 columns, then getBy , makePathArray are...
Read more >
Filters | TanStack Table Docs
The ability for a column to be column filtered is determined by the following: The column was defined with a valid accessorKey /...
Read more >
Slow simple update query on PostgreSQL database with 3 ...
I have to update tables of 1 or 2 billion rows with various values for each rows. Each run makes ~100 millions changes...
Read more >
Essential basic functionality — pandas 1.5.2 documentation
Iterating through pandas objects is generally slow. In many cases, iterating manually over the rows is not needed and can be avoided with...
Read more >
Market Profile Westfield - JGSC Group
Growth in these neighborhoods is slow. Residents spend money for nice homes, cars, clothes, and vacations. Exercise is a priority; they work out...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found