Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Slow performance in Chrome on Mac
See original GitHub issueI’ve just starting playing with OpenRefine and it looks like a great tool for our purposes. However, the performance on our dataset is so slow as to make it basically unusable. Since I’m new to the tool, I wanted to check if I’m doing something wrong or if this is a known limitation.
My computer is a one-year-old Macbook Pro with 16Gb so it’s pretty fast. Loading our dataset (400,000 rows, 2 columns) is quite fast. Creating a text facet on the second row is pretty fast too. The facet has about 30,000 unique entries. The problems start when I click on “count” to sort the facet entries by count. That takes about 2-3 minutes. If I click on an item to view the individual entries in the righthand panel, that also takes 2-3 minutes during which time the browser is completely unresponsive.
I’m using Java v1.8.0_192. This doesn’t seem to be a memory issue since I ran with ./refine -m 4096m and it doesn’t seem to be close to the 4Gb limit (it never goes over 2Gb from what I’ve seen). When I launch a slow operation, it looks like the bottleneck is actually Chrome which is pegged at 100% CPU.
Does this seem normal to people when working with a dataset of this size or am I doing something wrong?
Issue Analytics
- State:
- Created 4 years ago
- Comments:8 (5 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
OK - more investigation and I’ve realised that the
sortaspect is a red herring. The speed issue isn’t caused by the sort - which is fast - but by redrawing the facet each time. There’s even a note in the code to the effect that there is a performance issue here!Actually reported back in 2013 as #672 but never fixed I’m afraid. This feels like it is definitely fixable 5 years on, but I’m not sure what the solution is right now.
@matthewgertner I’m going to close this issue in favour of #672, but if you want some help in thinking about whether OpenRefine can help you get the outcome you want without encountering this facet performance issue please post about what sort of outcome you are looking for in the discussion forum at https://groups.google.com/forum/#!forum/openrefine
Hopefully I’ll get some time to look at resolving #672 soon
OK thanks again @ostephens!