does this work with threads?

I tried using this library with concurrent.futures.ThreadPoolExecutor to fill up the cache like so:

import requests
import requests_cache
from requests.adapters import HTTPAdapter
adapter = HTTPAdapter(max_retries=5)
session = requests_cache.core.CachedSession()
session.mount('http://', adapter)

from concurrent.futures import ThreadPoolExecutor, as_completed

urls = [....]

with ThreadPoolExecutor(max_workers=7) as executor:
    future_to_url = {executor.submit(session.get, url): url for url in urls}
    for i, future in enumerate(as_completed(future_to_url)):
        url = future_to_url[future]
        response = future.result()

It seemed to be working pretty slowly- I wasn’t sure exactly what was going on. Do you know if this is supported?

[chanfr]

I’m having issues using request_cache with threads with the default sqlite backend.

I’m making requests that take a short time to finish with 40 threads and in some cases, I’m getting the following error:

database is locked

I’m using Pool from multiprocessing

from multiprocessing import Pool

pool = Pool(n_cores)
df = pd.concat(pool.map(func, df_split))

where func performs a get request.

[ximo1984]

@JWCook Sorry but I’m not working on that project currently so I can’t answer.