question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Preprocess several files

See original GitHub issue

How to preprocess several files after each other in one script ?

I had to make a litte change in the: File: __ init __.py ; Function: post_process(self) [deleted the sys.exit(0) statement if run==False ] and adding a function: Original:

        # resolve postprocess stage depending on the mode
        if self.run == False:
            sys.exit(0)
        else:

Changed:

        # resolve postprocess stage depending on the mode
        if self.run:

Added Function:

    # reseting internal things to parse a second file
    def reset_internal(self):
        self.__linenum = 0
        self.__excludeblock = False
        self.__ifblock = False
        self.__ifcondition = ''
        self.__ifconditions = []
        self.__evalsquelch = True
        self.__outputBuffer = ''

now I can do things like this: pypreprocessor.input = ‘Algorithm.py’ pypreprocessor.output = ‘Algorithm_out.py’ pypreprocessor.parse()

pypreprocessor.input = ‘Algo_2.py’ pypreprocessor.output = ‘Algo_2_out.py’ pypreprocessor.parse()

Process several file with the same #ifdef statements in one file is there another oportunity to do so ?

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
evanplaicecommented, Apr 11, 2017

self.run is False is telling the preprocessor that the output files will be produced but not run. This works more like a transpiler (ex sass, less, babel), where the files are preprocessed on-the-fly and the output is saved as new files.

Note: This can be a great approach if you setup a simple file watcher that automatically preprocesses the files when it detects that the source has been changed.

`self.run is True’ is telling the preprocessor to run the postprocessed source of the file it’s defined in. It preprocesses the source on-the-fly, imports, and launches it all at once. This works more like C where the preprocessor directives are supported natively, except python doesn’t allow preprocessor code so this requires a bit of magic to work.

Note: This approach is ideal if you just need some simple branch logic in a single source file. That doesn’t need to be imported by something else.

The 2nd approach won’t work because the python’s internal lexer/parser processes all import statements on the first pass. In this case, you’re trying to import source files that don’t exist yet (ie the preprocessor doesn’t run until after python’s internal lexer/parser). There is a way to modify the internal imports list after python’s internal lexer/parser has run, that’s the ‘magic’ that makes on-the-fly mode work. To see it in action, check out the override_imports method.

In short, if you want to load files in the same source that is responsible for preprocessing them, you have to use an alternative to the import statement. Things that seem very straight-forward and simple don’t necessarily work as you’d expect them to when you’re trying to load code that’s being dynamically generated.

0reactions
hendiolcommented, Apr 18, 2017

Thanks for the answers to evanplaice.

OK, fine.

Yes I know the function, for debugging reasons i often left removeMeta = False (so that the line numbers are consistend)

OK, I added a varaible mode with which the user can swith between: ‘Run’ (is the same as your run = True), ‘PP’ (preprocess only is the same as before run = False, PPCont (preprocess and continue: preprocess file and reset internal variables)

The nested #ifdef works with a list, you can see it in my project. I will modify my Readme and the examples and afterwards do a PR. (Some test on my version are done but if you have some cases you can test them)

Oh, i hope my commits into my PR are not to weird. At the moment I have no further ideas.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Preprocessing multiple source files with one command using ...
I have three files in my current working directory: hello.cpp; goodbye.cpp; prog.cpp. I would like to only preprocess hello.
Read more >
Multiple files and the pre-processor - Physics and Astronomy
The answer is to put our function prototypes inside a header file and #include it in all our source files. The preprocessor line:...
Read more >
An Introduction to GCC - Preprocessing source files - Linuxtopia
The preprocessed system header files usually generate a lot of output. This can be redirected to a file, or saved more conveniently using...
Read more >
Preprocess - Hugging Face
We're on a journey to advance and democratize artificial intelligence through open source and open science.
Read more >
/P (Preprocess to a File) | Microsoft Learn
The file has the same base name as the source file and an .i extension. In the process, all preprocessor directives are carried...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found