Nuitka takes extremely long time to compile

Problem: I tried to compile a file, Nuitka spent way more time than I expected, and process bar stuck at 301/301, while CPU utilization is 0%. After about half hour, I’ve to use Ctrl-C to manually end the process. To my surprise, after I enter Ctrl-C it magically continue and finish its compiling process.

SOFTWARE ENVIRONMENT Windows10 21H2 Nuitka 0.6.19.4 Python 3.9.10

HARDWARE ENVIRONMENT Ryzen7 1700 32GB DDR4-2666 SN750 500G

COMMAND

nuitka --mingw64 --standalone --plugin-enable=multiprocessing --plugin-enable=pyqt5 --plugin-enable=numpy --plugin-enable=tk-inter --include-module=matplotlib.pyplot --output-dir=C:\Users\home\Desktop\Python\1 C:\Users\Home\PycharmProjects\gambling_simulator\main.py

OUT-PUT

Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: gcc (gcc).
Nuitka-Scons:INFO: Slow C compilation detected, used 60s so far, this might indicate scalability problems.
Nuitka-Scons:INFO: Running 'C:\\Users\\Home\\AppData\\Local\\Nuitka\\Nuitka\\gcc\\x86_64\\11.2.0-12.0.1-9.0.0-r1\\mingw64\\bin\\gcc.exe -o "C:\\Users\\home\\Desktop\\Python\\1\\MAIN~1.DIS\\main.exe" -fuse-linker-plugin -flto=16 -fpartial-inlining -freorder-functions -Wl,--exclude-all-symbols -Wl,--out-implib,.\\import.lib -municode -O3 -Wl,--stack,9863168 -Wl,--enable-auto-import -s -static-libgcc @".\\@link_input.txt" -LC:\\Users\\home\\AppData\\Local\\Programs\\Python\\Python39\\libs -lm -lpython39' took 1725.83 seconds
Nuitka-Scons:INFO: Compiled 301 C files using ccache.
Nuitka-Scons:INFO: Cached C files (using ccache) with result 'cache miss': 294
Nuitka-Scons:INFO: Cached C files (using ccache) with result 'cache hit': 7
Nuitka-Plugins:INFO: pyqt5: Including Qt plugins 'iconengines,imageformats,mediaservice,platforms,platformthemes,printsupport,styles' below 'PyQt5\qt-plugins'.
Nuitka-Plugins:INFO: pyqt5: Found 2 files DLLs from 'OpenSSL' PyQt5installation.
Nuitka-Plugins:INFO: numpy: Found 1 file DLLs from '' numpyinstallation.
Nuitka-Plugins:INFO: data-files: Included data file 'lib2to3\Grammar3.9.10.final.0.pickle' due to package data for 'lib2to3.pgen2'.
Nuitka-Plugins:INFO: data-files: Included data file 'lib2to3\PatternGrammar3.9.10.final.0.pickle' due to package data for 'lib2to3.pgen2'.
Nuitka-Plugins:INFO: matplotlib: Included data dir 'matplotlib\\mpl-data' with 183 files due to: package data for 'matplotlib.
Nuitka-Plugins:INFO: matplotlib: Included data file 'matplotlib\mpl-data\matplotlibrc' due to Updated matplotlib config file with backend to use..
Nuitka:INFO: Keeping build directory 'C:\\Users\\home\\Desktop\\Python\\1\\main.build'.
Nuitka:INFO: Successfully created 'C:\\Users\\home\\Desktop\\Python\\1\\main.dist\\main.exe'.

Code

from matplotlib import pyplot
import random
import time
import multiprocessing
import os


class coin():
    def __init__(self,balance,bet_rate,repeat):
        self.balance = balance
        self.bet_rate = bet_rate
        self.repeat = repeat

    def get_probability(self):
        s = time.perf_counter()
        x_axis, y_axis, tasks = [], [], []
        distribution = self.get_tasks_balanced_complexity(1,self.repeat+1,os.cpu_count()-1)
        pool = multiprocessing.Pool(processes=len(distribution))
        for x in distribution:
            tasks.append(pool.apply_async(self.worker,args=(x[0],x[1])))
        pool.close()
        pool.join()
        for x in tasks:
            x_a, y_a = x.get()
            x_axis += x_a
            y_axis += y_a
        e = time.perf_counter()
        print(round(e - s, 2), 's')
        pyplot.plot(x_axis, y_axis, color='#000000')
        pyplot.show()

    def worker(self,start,end):
        x_axis, y_axis = [], []
        for x in range(start,end):
            #print('Worker-'+str(os.getpid())+':',str(round((x-start)/(end-start),2)*100)+'%')
            win = 0
            for y in range(10000):
                balance = self.run_simulation(self.balance, x)
                if balance > self.balance:
                    win += 1
            y_axis.append(win/10000)
            x_axis.append(x)
        return x_axis, y_axis

    def run_simulation(self,balance,repeat):
        for x in range(repeat):
            bet = self.bet_rate * balance
            balance -= bet
            while True:
                R = random.random()
                if R > 0.5:
                    balance += bet * 2
                    break
                elif R < 0.5:
                    break
        return balance

    @staticmethod
    def get_tasks_balanced_complexity(start,end,number):
        task = []
        sum = (start + end - 1) * (end - start) / 2
        complexity = sum / number
        while start < end:
            for x in range(start, end):
                if ((start + x) * (x - start + 1) / 2) / complexity >= 1.0:
                    task.append((start, x))
                    break
                elif ((start + end - 1) * (end - start) / 2) <= complexity:
                    task.append((start, end))
                    break
            if task[-1][0] != task[-1][1]:
                start = task[-1][1]
            else:
                start = task[-1][1] + 1
        print(task)
        return task


if __name__ == '__main__':
    coin(1000,0.05,int(input())).get_probability()

[kayhayen]

I would try anti-bloat first, then maybe --lto=no, and definitely using MSVC as well, as it’s often more scalable.

[kayhayen]

@huangxin168 posting below closed issues is a surefire way to get ignored.