Resumable media (chunked upload) slow

Issue

I’m getting a max of around 20Mbps on any file upload which is resumable=True and chunked (regardless of chunksize), however I get my full internet speed of around 180Mbps upload on the same file when not using this method. The problem is that if the file is too big to fit in memory (or even threading many uploads) the computer will run out of memory and I will have to use chunked upload which is just too slow.

Am I doing something wrong? How can I upload a large file without having to worry about running out of memory and not be limited to 20Mbps?

Environment details

• OS: Windows 10
• Python version: 3.5
• pip version: 19.0.1
• google-api-python-client version: 1.7.8

Steps to reproduce

1. Use the code below
2. Monitor traffic output limits

Sample Code

Shared code between both examples:

import os
from google.oauth2 import service_account
from googleapiclient.discovery import build, MediaFileUpload

def create_drive_service(user_email,SERVICE_ACCOUNT_JSON_FILE,SCOPES=None):
    if SCOPES is None: SCOPES = ['https://www.googleapis.com/auth/drive']
    credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_JSON_FILE)
    credentials = credentials.with_scopes(SCOPES)
    credentials = credentials.with_subject(user_email)
    return build('drive', 'v3', credentials=credentials)
#
# Fill in the following
#
jsonpath = 'c:\\path\\to\\service_account.json'
service = create_drive_service('user@domain',jsonpath)
filename = 'filename.ext'
filepath = 'c:\\path\\to\\folder\\' + filename
parent_id = ''

Code that is slow:

file_metadata = {"name":filename,"parents":[parent_id]}
media = MediaFileUpload(filepath,chunksize=-1,resumable=True)
request = service.files().create(body=file_metadata,media_body=media,fields='id')
while response is None:
    status, response = request.next_chunk()
file = request.execute()

Code that works full-speed but not if the file is too big to fit in memory:

file_metadata = {"name":filename,"parents":[parent_id]}
media = MediaFileUpload(filepath,resumable=False)
request = service.files().create(body=file_metadata,media_body=media,fields='id').execute()

[jfernandez04]

@acidnine I set it like this and have no problem with my upload.

media = MediaFileUpload(archivo, mimetype='text/plain', chunksize=256 * 1024, resumable=True)

[shivamsn97]

@LEChaney were you able to increase your upload speed? I am stuck at the same problem.