Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] - synchronize._semlock in python3.9 breaking multiprocessing.pool in AWS lambda which breaks OpenSearch

See original GitHub issue

OSError: [Errno 38] Function not implemented. I started seeing this error after upgrading to python3.9. The reason is opensearch bulk function is using multiprocessing module internally and python multiprocessing.pool.ThreadPool is breaking.

OSError: [Errno 38] Function not implemented
sl = self._semlock = _multiprocessing.SemLock(
SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)

File "/var/lang/lib/python3.9/multiprocessing/synchronize.py", line 57, in __init__
SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
File "/var/lang/lib/python3.9/multiprocessing/synchronize.py", line 162, in __init__
return Lock(ctx=self.get_context())

–

</body> </html>

It looks like:

synchronize.Lock doesn’t work in lambda for any version of Python (lambda has no /dev/shm, and no write access to /dev in lambda - see: https://aws.amazon.com/blogs/compute/parallel-processing-in-python-with-aws-lambda )
ThreadPool is now using synchronize.Lock from version 3.9

To Reproduce

Steps to reproduce the behavior:

Deploy an application using opensearch-py==1.0.0 to aws lambda
Invoke bulk function of opensearch
See error

Expected behavior The opensearch client should work as it was working fine with python3.6

Plugins opensearch-py==1.0.0

Screenshots Error screenshots

Host/Environment (please complete the following information):

OS: Aws lambda

Additional context Add any other context about the problem here.

Issue Analytics

State:
Created 2 years ago
Reactions:3
Comments:5 (2 by maintainers)

Top GitHub Comments

1reaction

jasongilmancommented, Dec 2, 2021

I’m also seeing this error with Python 3.8

[ERROR] OSError: [Errno 38] Function not implemented
Traceback (most recent call last):
....
  File "/var/task/opensearchpy/helpers/actions.py", line 469, in parallel_bulk
    pool = BlockingPool(thread_count)
  File "/var/lang/lib/python3.8/multiprocessing/pool.py", line 925, in __init__
    Pool.__init__(self, processes, initializer, initargs)
  File "/var/lang/lib/python3.8/multiprocessing/pool.py", line 196, in __init__
    self._change_notifier = self._ctx.SimpleQueue()
  File "/var/lang/lib/python3.8/multiprocessing/context.py", line 113, in SimpleQueue
    return SimpleQueue(ctx=self.get_context())
  File "/var/lang/lib/python3.8/multiprocessing/queues.py", line 336, in __init__
    self._rlock = ctx.Lock()
  File "/var/lang/lib/python3.8/multiprocessing/context.py", line 68, in Lock
    return Lock(ctx=self.get_context())
  File "/var/lang/lib/python3.8/multiprocessing/synchronize.py", line 162, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/var/lang/lib/python3.8/multiprocessing/synchronize.py", line 57, in __init__
    sl = self._semlock = _multiprocessing.SemLock(

0reactions

wbecklercommented, Oct 19, 2022

Is anyone up for contributing a patch that addresses this issue when /dev/shm isn’t available? There’s a potential drop in replacement for the multiprocessing library: https://pypi.org/project/lambda-multiprocessing/