question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support rasters which does not fit in memory

See original GitHub issue

As I can see from source code rio-cogeo uses MemoryFile. Does it mean that this plugin is not suit for cases when source raster is not fit in memory?

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:11 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
dionhaefnercommented, Apr 12, 2019

Yes, flake8 is very critical of lambdas (IMO to a fault, but who am I to judge). This should work though:

- dst_ctx = lambda: memfile.open(**meta)
+ dst_ctx = functools.partial(memfile.open, **meta)

If you’re not too paranoid about unclosed file descriptors in some edge cases I think it would also be perfectly fine to initialize the contexts first and enter them later:

if in_memory:
    file_ctx = MemoryFile()
    dst_ctx = memfile.open(**meta)
else:
    file_ctx = ...
    dst_ctx = ...

with file_ctx, dst_ctx as dst:
    ...
1reaction
drnextgiscommented, Apr 11, 2019

Good job @vincentsarago! Can we use contextlib2.ExitStack?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Support rasters which does not fit in memory · Issue #21 - GitHub
Works fine from my experience (raster files > 20GB, script staying below 500MB memory usage). All reactions.
Read more >
Clearing rasters from in_memory doesn't work - Esri Community
I have the need to process a very large amount of polylines against a raster using the ExtarctByMask gp. Saving the result sub-raster...
Read more >
Cell size of raster data—Help | ArcGIS for Desktop
The size of the cell (pixel) determines the level of detail that can be represented in a raster.
Read more >
NumPy memory error on large rasters - GIS Stack Exchange
A MemoryError means that you have exhausted the memory available. If your ArcGIS is 32-bit, then that limit is 4GB. You will see...
Read more >
MEM – In Memory Raster — GDAL documentation
MEM – In Memory Raster ... GDAL supports the ability to hold rasters in a temporary in-memory format. This is primarily...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found