Exposure lock no longer works in Stretch

I’ve had this working for some time, then apparently with a Raspbian update in early Feb 2018 this issue starting occurring. I don’t know which update or exactly when, as my logs are on a tmpfs and wiped out on each reboot.

Here is the problem. I set up the PiCamera to lock all parameters I can, including exposure_mode=off and ISO=60 (this is v2.1 camera version). I then set shutter_speed=1000 for example and start acquiring images (I acquire raw Bayer data which I process later).

After each image is received, exposure_speed drops as if it is under control of some AGC, but the camera is completely in the dark and yet exposure_speed keeps dropping. After about 15 frames, exposure_speed is 9 and no longer falls further. See the program output below.

If I then set shutter_speed=1000 after each image is received, exposure_speed drops to 936 after the first image is received and stays there. It is as if each time shutter_speed is set, the drop is arrested for one more frame.

This issue occurs with two separate PiCamera v2.1 modules, but not with a v1.3 module. Also, using raspistill with the -ss option to produce a JPEG file, and exiftool looking for the exposure metadata indicates that the exposure was as expected with the raspistill -ss setting. I don’t have the exact command and response available at the moment because the v1.3 camera is installed, but can provide it if helpful.

I’m attaching the smallest Python file I’ve gotten working to show this. Camera setup is in the __init__() method and the frame acquisition is in the acquire() method.

OS is Raspbian (Debian stretch) on a Raspberry Pi 2B, all latest updates, and using picamera 1.13 and python 3.5.3. vcgencmd version returns:

Mar 13 2018 18:52:21 
Copyright (c) 2012 Broadcom
version 6e08617e7767b09ef97b3d6cee8b75eba6d7ee0b (clean) (release)

Program output when exposure_speed continues to drop:

Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.974, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.974, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.936, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.860, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.784, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.709, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.633, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.558, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.482, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.406, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.331, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.255, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.180, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.104, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.028, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.009, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.009, b 50, a gain 1.000, d gain 1.000
Camera: ISO 60, 5.000 fps, exp mode off, shutter 0.992 ms, exposure time 0.009, b 50, a gain 1.000, d gain 1.000

camtest.py:

from picamera.array import PiArrayOutput
from picamera import PiCamera
from time import sleep

class CamTest:
    def __init__(self):

        # Initialize the camera
        self.camera = PiCamera(sensor_mode=2)

        # Key parameters unique to each sensor type used by the camera:
        #   self.pixel_pitch: pixel pitch in microns
        #   self.iso_unity_gain: the ISO value that should put all non-programmable gains at 1
        #   self.bayer_loc_red: the location of the red pixel in each Bayer group, in form (row, col).  Row
        #               and col must either be 0 (even row or column) or 1 (odd).
        self.pixel_pitch, self.iso_unity_gain, self.bayer_loc_red = {
                    'ov5647': (1.4, 100, (1, 0)),
                    'imx219': (1.12, 60, (1, 1))
                }[self.camera.revision]

        # Image sizes of Bayer data from the camera data stream; note these are (short_dimension, long_dimension):
        #   bayer_crop is the sizes of actual image data, including the additional 5th byte in width that
        #   contains the lower 2 bits of each of the preceding 4 pixels for reconstructing 10-bit pixels.
        self.bayer_crop = (self.camera.MAX_RESOLUTION[1], int(self.camera.MAX_RESOLUTION[0] * 5 / 4))
        #   bayer_shape is the shape of the frame transferred from the camera.  It rounds bayer_crop up to the
        #   next 16 pixels in the short dimension and 32 in the long dimension. Yes, it rounds up
        #   even if the crop size is already an integer multiple of 16 or 32.
        self.bayer_shape = (16 * (self.bayer_crop[0] // 16 + 1),
                            32 * (self.bayer_crop[1] // 32 + 1))

        # Camera setup: least possible processing, no auto anything
        self.camera.resolution = self.camera.MAX_RESOLUTION
        self.camera.framerate = 5
        self.camera.awb_mode = 'off'
        self.camera.image_denoise = False
        self.camera.video_denoise = False
        self.camera.sharpness = 0
        self.camera.iso = self.iso_unity_gain      # Set ISO to force (hope) unity gains, then lock...
        self.camera.exposure_mode = 'off'                       # ... locks gains

        self.default_shutter_speed = 1000           # 1 ms

        # Capture the Bayer array as is, no processing
        self.bayer_capture = PiArrayOutput(self.camera, self.bayer_shape)
        self.stream = self.camera.capture_continuous(self.bayer_capture, format='jpeg', bayer=True, quality=1, thumbnail=None)

        self.close_now = False          # If set, stop the entire subsystem: camera, stream, buffers, and free resources

        self.camera.shutter_speed = self.default_shutter_speed

        # Allow gains to settle (?? if the sleep isn't here, the print shows gains = 0.0 ??)
        sleep(2)
        self.print_camera_stats()

    def acquire(self):
        try:
            for f in self.stream:
                self.print_camera_stats()
                # If shutter speed is forced after every image, the exposure_speed drop
                # occurs only for the first frame
                #self.camera.shutter_speed = self.default_shutter_speed

                if self.close_now:
                    self.close_all()

        except Exception as e:
            print("unable to start image stream: {}".format(repr(e)))

    def print_camera_stats(self):
        print(
            "Camera: ISO {:d}, {:.3f} fps, exp mode {}, shutter {:.3f} ms, exposure time {:.3f}, b {:d}, a gain {:.3f}, d gain {:.3f}". \
            format(self.camera.iso,
                   float(self.camera.framerate),
                   self.camera.exposure_mode,
                   self.camera.shutter_speed / 1000,
                   self.camera.exposure_speed / 1000,
                   self.camera.brightness,
                   float(self.camera.analog_gain),
                   float(self.camera.digital_gain)))

    def close(self):
        # indicate that the thread should be stopped
        self.close_now = True

    def close_all(self):
        self.stream.close()
        self.bayer_capture.close()
        self.camera.close()

try:
    cam = CamTest()
    cam.acquire()
except KeyboardInterrupt:
    cam.close()

[6by9]

Cause found and patch created. It should turn up in the firmware fairly soon, via rpi-update initially, and then via apt once proven stable.

[waveform80]

Glad to see this got fixed (and sorry I’ve been away for so long on picamera; life and piwheels have effectively eaten all my time this year - but I’m going to try and squeeze in a 1.14 release before the end of the year). Anyway, looks like this is another one I can close courtesy of 6by9! 😃