question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

PERF: Reintroduce a global GEOS context

See original GitHub issue

_Originally reported by @caspervdw in https://github.com/pygeos/pygeos/issues/192_

In https://github.com/pygeos/pygeos/pull/113, the global GEOS context was eliminated in favor of a context that was initialized and destroyed in every function. This was necessary for multithreading.

However, in https://github.com/pygeos/pygeos/pull/182, @jorisvandenbossche encountered a performance hit coming from this intialization/finalization of the context.

I am thinking especially of the deallocation of Geometry objects. It would probably be hard to time this, but I think that destroying a array with lots of geometries has taken a performance hit after #113.

Maybe the functions that need the GIL anyway can benefit from keeping a global GEOS context around. I would like to explore this, and maybe we can also use a global WKB reader / writer and WKT reader / writer for speeding up single-geometry construction, and repr and str functions.

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:7 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
kylebarroncommented, Oct 17, 2022

In #113, the global GEOS context was eliminated in favor of a context that was initialized and destroyed in every function. This was necessary for multithreading.

Might be helpful to edit the first comment to specifically point to https://github.com/pygeos/pygeos/pull/113. I was wondering how #113 related to multithreading 😄

0reactions
jorisvandenbosschecommented, Sep 7, 2022

We would also have to make sure to properly cleanup a theadlocal GEOS context after the thread stops. Not sure if there are hooks for that.

One option might be to define a module clean up for lib that can destroy the context (https://docs.python.org/3/c-api/module.html#c.PyModuleDef.m_free). Another option could be to put the context in a small extension type PyObject as well, and define a dealloc (although that might still require an explicit decref on module clean-up, leading to the same issue, not sure)

But indeed, on the short term only tackling the cases that already have the GIL will give the biggest benefit and is much simpler.

Read more comments on GitHub >

github_iconTop Results From Across the Web

PERF: Reintroduce a global GEOS context - PullAnswer
I suppose deallocation is one of the main cases where a method of the geometry gets called many times individually and where we...
Read more >
Feasibility of reintroducing grassland megaherbivores, the ...
Feasibility of reintroducing grassland megaherbivores, the greater one-horned rhinoceros, and swamp buffalo within their historic global range.
Read more >
A deep dive into Search Console performance data filtering ...
Search performance basics · Clicks: Count of user clicks from Google Search results to your property. · Impressions: Count of times users saw...
Read more >
Strengthening surveillance systems for malaria elimination
Surveillance is a core component of an effective system to support malaria elimination. Poor surveillance data will prevent countries from ...
Read more >
The Global Context: How Politics, Investment, and Institutions ...
The first set of risks is geo-economic: the balance in the global economy is ... Growing Doubts About the Performance of Emerging Economies....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found