question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Problems with memory mapped files and vectors.most_similar()

See original GitHub issue

Hello. I am on windows using python 3.6

There seems to be a problem with creating the magmap files for using the vectors.most_similar() method. Error:


  File "\pymagnitude\__init__.py", line 1074, in get_vectors_mmap
    os.rename(path_to_mmap_temp, self.path_to_mmap)

PermissionError: [WinError 32] The process cannot access the file because it is in use by another process \\AppDataTemp\\82ce74f40baf842d23c45b0e90688b9f.magmmap.tmp' -> '\AppData\\Local\\Temp\\82ce74f40baf842d23c45b0e90688b9f.magmmap'

I looked into the code and it seems you are creating and filling these files in the background. It works fine for the vectors.most_similar_approx() method. The memmap file for approx seems to be created just fine.

The problem persists even when i use blocking=True when constructing the magnitude object. Then i can’t even use the approx method. It just hangs while initializing the object.

Thank you for your time.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:12 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
yassin-taskincommented, Mar 11, 2018

Thank you very much for your time Ajay. I will post as soon as i can, most likely tomorrow.

0reactions
AjayP13commented, Mar 12, 2018

Your welcome and thanks for the help in making it Windows compatible! I added your handle as a Developer on the GitLab repository. If you come across any other issues, feel free to submit a pull request / merge request to get it merged into master and deployed to PyPI.

Would love to see what you working on / end up doing with Magnitude in your master’s thesis, e-mail us at opensource@plasticity.ai if you can share it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Memory-mapped files considered harmful (for databases)
Memory-mapped files create numerous performance and reliability issues and you should stay clear of DBMS that persist data using them.
Read more >
Inconsistent performance of memory mapped files : r/rust
I need this to happen as fast as possible, so I am experimenting: Loading the contents into a vector runs in approximately 10...
Read more >
Disadvantages of using memory mapped files - Stack Overflow
The main disadvantage of MMFs is that they consume RAM, making the file system cache less effective. Not an issue with such small...
Read more >
3.4. Shared Memory With Memory-mapped Files
Memory -mapped files provide more efficient access for initial reads. When read() is used to access a file, the file contents are first...
Read more >
Memory-mapped file - Wikipedia
A memory-mapped file is a segment of virtual memory that has been assigned a direct byte-for-byte correlation with some portion of a file...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found