ignoring unsupported keyword arguments for client and ExtraArgs

Problem description

Be sure your description clearly answers the following questions:

• What are you trying to achieve? Using the smart_open.open function I am trying to pass the client and ExtraArgs keywords allowed in Boto3 to my stream to an S3 bucket. boto3 upload docs
• What is the expected result? My expectation is that the data will be added to the s3 bucket without any warnings and that the ExtraArgs will be applied
• What are you seeing instead? What I am seeing instead is that while the data is still being uploaded to the s3 bucket it: 1. Kicks back a warnings re: client and ExtraArgs keywords 2. Doesn’t apply the ExtraArgs to the dataset on s3

Steps/code to reproduce the problem

In order for us to be able to solve your problem, we have to be able to reproduce it on our end. Without reproducing the problem, it is unlikely that we’ll be able to help you.

input

s3_client = boto3.client("s3")
extra_args = {'ContentType': 'text/csv'}
file_path = smart_open.open(("s3://" + bucket + "/" + key), 'w', newline="",
                                        transport_params=dict(client=s3_client, ExtraArgs=extra_args))

file = io.StringIO()
with file_path as file_out:
    writer = DictWriter(file)
    writer.writeheader()
    file_out.write(file.getvalue())

output

WARNING - ignoring unsupported keyword arguments: ['ExtraArgs', 'client']

Include full tracebacks, logs and datasets if necessary. Please keep the examples minimal (minimal reproducible example).

Versions

Please provide the output of:

import platform, sys, smart_open
print(platform.platform())
print("Python", sys.version)
print("smart_open", smart_open.__version__)

Windows-10-10.0.19041-SP0
Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:37:02) [MSC v.1924 64 bit (AMD64)]
smart_open 4.2.0

Checklist

Before you create the issue, please make sure you have:

• Described the problem clearly
• Provided a minimal reproducible example, including any required data
• Provided the version numbers of the relevant software

[patrickleclair-GORDONFN]

Thanks for all of your help. I was able to pass the parameters and have it run without warnings now.

[mpenkov]

OK, I’ve worked out what the problem is. First and foremost, the docs were for an unreleased developer version, and you had 4.2.0 installed. I’ve since released 5.0.0 - if you install that, the behavior should be more like what you expected.

Second, you’re passing the Content-Type parameter (and other parameters) incorrectly. You need to do something like this:

transport_params = {
  'client_kwargs': {
     'S3.Client.put_object': {
       'Content-Encoding': ...,
     },
  },
}

See the docs for more examples.