Newline characters in output capture are inappropriately translated in Python 3

This issue is specific to the captured output (i.e. in cout.txt) when SETTINGS.CAPTURE_MODE == "fd" (i.e. the default in Linux), in Python3.

In this capture mode, Sacred attempts to capture both stdout and stderr for the process and all subprocesses by forwarding these two streams to a temporary file. The contents of this temporary file are then read out every “hearbeat” and redirected to cout.txt.

However, in Python 3, the default behavior for opening a file is to convert all types of line break characters (i.e. \r, \n, and \r\n) into the newline character \n (see here for documentation of this). This causes things like progress bars to be captured incorrectly in Python 3. This is precisely the issue seen in issue 440.

A quick-and-dirty fix that works only for Python 3 would be to alter (line 119 in stdout_capturing.py) to be with NamedTemporaryFile(mode="w+", newline='') as target:

Below is a short test script that should reproduce this behavior:

import sacred                                                                   
import sys                                                                      
                                                                  
ex = sacred.Experiment("ex")                                                    
ex.observers.append(sacred.observers.FileStorageObserver.create("my_bug_test"))

# This following line doesn't work for carriage returns, because all carriage returns
# have already been converted to new lines       
ex.captured_out_filter = sacred.utils.apply_backspaces_and_linefeeds  
                                                                                
@ex.automain                                                                    
def main():                                                                     
    for _ in range(5):                                                          
        sys.stdout.write("Should not see this\r")                               
    sys.stdout.write("Should only see this\n")                                  
    sys.stdout.flush()

[amtseng]

Sorry for the delay, I hadn’t been getting these notifications for some reason. PR has been submitted!

[CarloLucibello]

friendly bump to @atseng95 for a PR