question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Drastic performance issue (Binaries execution slower than Python!)

See original GitHub issue

0.6.14.4 Python: 3.5.3 (default, Nov 18 2020, 21:09:16) OS: Linux Arch: x86_64 Installed via Pip in a virtualenv.

Hallo, I am using nuitka as everybody here in order to transpile python script into binaries file and thus expect to be a bit faster than in Python. It is, normally, but I am facing right now a big issue. I developped a server containing three threads (threading module from Python) : -one for recieving messages from client via a socket and answer him -one for sending in real time events to the client. -and one, the main thread containing the two above. It controlls in a try catch, if the flag of the receiver thread is set to True (means that it received an empty string) and thus restart the server, or catch a Keyboard exception and shutdown everything.

I used this command to generate the distribution : python -m nuitka --standalone --show-progress main.py My program use many natif Python modules such as socket, threading, ElementTree, etc… and use also Redis module (https://pypi.org/project/redis/) And what is really strange is that the binary program is slowed down! It takes an incredible amount of time to do for loops. For example, this little piece of code is a bottleneck :

            for device in data.db2.lrange("historique",0,-1):
                if data.db2.exists(device) == 0:
                    data.db2.lrem("historique",0,device)

db2 is a Redis databe ; lrem return components from list in given range here 0 to -1. This piece of code is executed in a blink in Python. And as far as I know, I dont think the problem is from Redis because I already transpiled monothread program using Redis commands without any performance issues.

So the question, why the binaries are slow? Does nuitka supports multi threading? What are the things to avoid while programming in Python in order to have the most optimized binary compilation? Best Regards

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:9 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
kayhayencommented, Oct 17, 2021

It would be nice, if someone packaged this as a test case, testing support for redis would be nice.

0reactions
kayhayencommented, Feb 7, 2022

Not sure, closing for lack of feedback, the original issue seems resolved and if you need help to provide it as a test case, that’s how far we can get, as I cannot help you do that.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Issue 11114: TextIOWrapper.tell extremely slow - Python tracker
I made some minor changes to the benchmark program. Conclusions are: * setting file._CHUNK_SIZE to 20 has a dramatic effect, changing execution ......
Read more >
Will executable programmes written in Python be slower ...
Yes. In raw execution terms. Uninterrupted function-for-function C++ is much faster. However, the vast majority of programs are I/O bound meaning you're ...
Read more >
Python on Windows. Why is reading text 8x slower than ...
I am opening XML text files about 5MB in size and noticed very slow performance even using lxml. I boiled the problem down...
Read more >
Why is Python so Slow? - Tackling Python's Performance ...
The first action in solving a performance problem is to locate the bottleneck. In most use cases, Python's speed limitations won't be noticeable....
Read more >
8 Must-Know Tricks to Use S3 More Effectively in Python
AWS provides another feature that can help us upload large files called S3 Transfer Acceleration. It allows to speed up uploads (PUTs) and ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found