Proposal about decoupling nuitka core modules from Options

Probably you won’t be interested on this one as I can understand it just works for you but after reading nuitka source code I’ve seen a bunch of modules across the whole project being coupled with Options, which is initialized/parsed on __main__.py. Let me tell you in a nice way that this is a really bad coding practice, period. By having this dependency across many of the project modules with the Options file you’ve made than those modules can’t be unit tested in isolation easily or even used independently and that’s really bad.

One example, yesterday I wanted to make some external tests using nuitka as a dependency of my project and it was a real nightmare. Let’s say I wanted to use some of Nuitka code/modules, right now that’s just not possible for few reasons, the most important being this ugly coupling with Options.py 😃

If you’re interested on this one I would be willing to refactor the code so the packages would become decoupled. I just wanted to comment this one with you first as I’m probably sure if I made a PR out of the blue refactoring a lot of your code you wouldn’t be happy with it and it’d be rejected straightaway, and it’d be a waste of time 😃 . Many coders tend to dislike other people suggestions when it comes to refactoring their projects with things that works for them, one example of one PR I’d made out of the blue few months ago and it got rejected, at that time I’ve learned the lesson providing PRs out the blue is not a good idea usually.

Let me put it differently, as I’ve mentioned in another threads the current CLI has a lot of options who are just probably interesting for you (coder) and not casual users and that’s fine, your project your rules… But… wouldn’t be great if another tools using nuitka core was also possible? An example, yesterday I wanted to make a fast test that worked only with a little subset of the current official spectrum, instead covering {python(“2.6”, “2.7”, “3.3”, “3.4”, “3.5”, “3.6”), all major operating systems, a lot of CLI options} I just wanted to a test working on {python3.6, windows, standalone_mode}, and that proved to be a nightmare… at one point, I’ve even tried to monkey patching some modules at runtime, which was a really bad idea so I’ve stopped there.

Btw, the test was going to be a little tool like:

usage = "usage: %prog module1 module2 ... moduleN"

No options at all, it would work just in standalone mode, that way I’d could create a stress test of many simple scripts and tests the new faster windows cache (got a 1st buggy version +/- working)

So… please let me know your thoughts about this proposal.

Yours, Bruno

[kayhayen]

Don’t be shy to tell me what’s bad, esp. when it’s bad.

So, essentially the way I see it, Nuitka needs to know some decisions, and a place to look it up. Lets go with standalone mode or not. If that is called options, modes, doesn’t matter much.

Places will still import something, and access that via some name or key, or function, right? Or do you want to pass an object along for them to use in functions?

The thing you are complaining about, and rightfully so, is that the way these values are to be populated, Nuitka is giving you are hard time to be doing that in code. I am totally OK for there to be a module, where values and their defaults live, and alternative ways to fill them in, then run Nuitka without the command line being even considered by Nuitka.

I really like code driven configuration. When e.g. considering which packages to include, code will often be more usable than command line. I am a fan of Buildbot and Nikola, where config is code, in a “master.py” or “conf.py”, but Nuitka isn’t like that.

Actually I wouldn’t know how to properly do it properly. Can you sketch a minimal example of how this works in practice?

[brupelo]

At #119 you wrote:

@brupelo I feel this content is misplaced in this issue. I enjoy discussions too, esp. if they have substance and are concrete. I would love to see and discuss based on commits achieving these goals. However, this should happen in the issues intended for it, limited to one specific issue, and if it’s general on the mailing list. There I have frequently explained my stance on how I develop Nuitka.

@kayhayen I think you’re actually right. As you can see, many times I tend to write long chunks of text talking about very abstract things and that’s mainly because it feels good exposing ideas based on experience/knowledge. By doing so maybe some interesting discussion emerges and if we’re lucky we both learn one or two new cool things, I’m all in to improve my skills as a coder with this type of conversations 😃

But maybe you’re right and github issues is not the best place to have this kind of long general conversations, actually from what I’ve seen so far from multiple issues I’ve opened, it seems it is not a very effective way to get things done and improving nuitka 😃 . So I guess you’ve got a point with your previous I enjoy discussions too, esp. if they have substance and are concrete. I would love to see and discuss based on commits achieving these goals. comment, it seems a legit comment 😄

On the other hand, I’m the type of coder who likes to dispatch large features all at once (I usually know beforehand how to tackle the whole thing at once) but that’s a problem here… usually if it’s my project and I’m working on it full-time and I don’t have to agree with anybody about my code I’ll deliver the feature in a very optimal and fast way.

But in this case, we’re talking about an open-source project where 1) People need to use their spare time, which usually it’s pretty limiting (at least it is over here) and 2) it’s needed to agree/discusing with others about possible commits to merge. Well, that’s really time consuming but of course, it’s definitely a necessary thing to do, otherwise If people delivered large big chunks of code/features at once and the main author/s didn’t agree those contributors would become frustrated and not willing to contribute to the project anymore… To be blunt here, I’ve always thought open-source contributions/workflows are quite tricky to manage and handle, but maybe it’s just me who’s not very used to contribute to these type of projects 😃

Anyway, coming back to this issue, I’m really interested on getting done this issue delivered as I consider it a first major step in order to sanitize a bit the whole project. But not sure how to tackle the right way, what do you think is the best way to address it? For instance, let’s say ~90files need to be changed and I’ve got like 10/15 minutes each day to spare on this. What’s the best way to progress and getting feedback from you? I was thinking about a workflow like:

• I create a feature branch from develop and I open an unfinished PR
• I’ll find little bit of spare time and eventually I’ll be progressing on the feature, although many days I won’t be able to even find not even a single minute to update the PR. I’ll tweak a little bit some files and I’ll push them, the PR will get updated and I’ll get your feedback so we’ll be able to discuss if other approaches would be better
• On many pushes, there will be times that some stuff will be missing and the project not behaving exactly like it’s behaving right now but eventually the feature will get done and we’ll have decoupled completely the Options module from all subsystems
• When the previous point is complete, I’ll be able to prove with few examples than writing tools with custom CLI parsers or GUI tools has become possible

What do you think? Does it make sense what I’m saying or it’s just a wasteful way to proceed here?

As you can see, I’m quite a newbie with git contributions so maybe what I’m proposing here is nonsense and not the best way to go… So please let me know how people tend to contribute with large features/refactorings/bug_fixings like this when they got pretty limited time.

Yours, B

Ps: https://hackernoon.com/the-art-of-pull-requests-6f0f099850f9