Rendering and Saving

Is there an easy way of rendering and saving the screens as videos like a wrapper in gym environments? env = SomeWrapper(save_loc="path") and env.render()

[JAEarly]

Here’s a solution in OpenCV. On the first pass it captures the frames as the environment is stepped through and saves it to a video file (mp4). On the second pass the video file is played back without needing to actually step through the environment. The second pass is not required if you merely want to save the episode to a file. The environment render call uses RGB, but OpenCV works on BGR so a conversion is required before rendering to the video file.

from dm_control import suite
import numpy as np
import cv2

def grabFrame(env):
    # Get RGB rendering of env
    rgbArr = env.physics.render(480, 600, camera_id=0)
    # Convert to BGR for use with OpenCV
    return cv2.cvtColor(rgbArr, cv2.COLOR_BGR2RGB)

# Load task:
env = suite.load(domain_name="cartpole", task_name="swingup")

# Setup video writer - mp4 at 30 fps
video_name = 'video.mp4'
frame = grabFrame(env)
height, width, layers = frame.shape
video = cv2.VideoWriter(video_name, cv2.VideoWriter_fourcc(*'mp4v'), 30.0, (width, height))

# First pass - Step through an episode and capture each frame
action_spec = env.action_spec()
time_step = env.reset()
while not time_step.last():
    action = np.random.uniform(action_spec.minimum,
                               action_spec.maximum,
                               size=action_spec.shape)
    time_step = env.step(action)
    frame = grabFrame(env)
    # Render env output to video
    video.write(grabFrame(env))

# End render to video file
video.release()

# Second pass - Playback
cap = cv2.VideoCapture(video_name)
while(cap.isOpened()):
    ret, frame = cap.read()
    cv2.imshow('Playback', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
cap.release()

# Exit
cv2.destroyAllWindows()

[Melanol]

@JAEarly Saving video works, but while playing, I get a lot of these errors: QObject::moveToThread: Current thread (0x55c6e4326da0) is not the object’s thread (0x55c6e41813d0). Cannot move to target thread (0x55c6e4326da0)

And then it ends with this (although, it manages to play the entire video): Traceback (most recent call last): File “/home/mel/PycharmProjects/dm_control_tripod/render.py”, line 57, in <module> play() File “/home/mel/PycharmProjects/dm_control_tripod/render.py”, line 46, in play cv2.imshow(‘Playback’, frame) cv2.error: OpenCV(4.6.0) /io/opencv/modules/highgui/src/window.cpp:967: error: (-215:Assertion failed) size.width>0 && size.height>0 in function ‘imshow’