[serve] Actor Ram size increases when sending large data

What is the problem?

When sending video data (binary data and numpy arrays) to a ray serve endpoint the ram size of that worker encreases on each call.

While in the same time no object storage is used:

Setup:

• Ray: 1.0.1
• OS: Ubuntu 18.04
• Python: 3.7.5

Reproduction (REQUIRED)

I was able to reproduce it with that script:

import time
import requests
from ray import serve

client = serve.start()

def echo(flask_request):
    return "hello " + flask_request.args.get("name", "serve!")

client.create_backend("hello", echo)
client.create_endpoint("hello", backend="hello", route="/hello")
url = "http://localhost:8000/hello"
payload = {}

while True:
    
    files = [
        ('test', (
            'my_video-53.webm', open('./my_video-53.webm', 'rb'),
            'video/webm'))
        ]
    headers = {
        'apikey': 'asdfasfwerqexcz'
        }

    response = requests.request("GET", url, headers=headers, data=payload, files=files)
    time.sleep(1)

The videos I used hat between 1MB and 4MB, if it is smaller the changes are not that obvious

• [x ] I have verified my script runs in a clean environment and reproduces the issue.
• I have verified the issue also occurs with the latest wheels. -> I am not able to check that because the console is broken (https://github.com/ray-project/ray/issues/11932) and does not display anything there

[simon-mo]

Hi @TanjaBayer, thanks for this report! I was able to reproduce it with the following script (swapped video file with np array) on latest master

import time
import requests
from ray import serve
import numpy as np
import io

client = serve.start()

def echo(flask_request):
    return "hello " + flask_request.args.get("name", "serve!")

client.create_backend("hello", echo)
client.create_endpoint("hello", backend="hello", route="/hello")
url = "http://localhost:8000/hello"
payload = {}

while True:
    
    files = [
        ('test', (
            'my_video-53.webm', io.BytesIO(np.zeros(4*1024*1024, dtype=np.uint8).tobytes()),
            'video/webm'))
        ]
    headers = {
        'apikey': 'asdfasfwerqexcz'
        }

    response = requests.request("GET", url, headers=headers, data=payload, files=files)
    time.sleep(1)

At start:

After a minute

(cc @edoakes )

[TanjaBayer]

@edoakes not yet but there is an open discussion in slack about it