question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Boundary info in Content-Type multipart request header not parsed correctly because of charset [SPR-17030]

See original GitHub issue

bissorc opened SPR-17030 and commented

I’m using Spring Rest Template to upload a file. Code looks as follows:

MultipartBodyBuilder builder = new MultipartBodyBuilder();
builder.part("file", "2;3;4".getBytes());
MultiValueMap<String, HttpEntity<?>> body = builder.build();
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Collections.singletonList(MediaType.ALL));
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity(body, headers);
RestTemplate restTemplate = new RestTemplate();
restTemplate.postForLocation("***", requestEntity, String.class);

This leads to the following raw request:

POST *** HTTP/1.1
Accept: */*
Content-Type: multipart/form-data;boundary=059h2BBM-KlM_XP2rY8W1X3_jnzFLcYY;charset=UTF-8
User-Agent: Java/1.8.0_121
Host: ***:***
Connection: keep-alive
Content-Length: 187

--059h2BBM-KlM_XP2rY8W1X3_jnzFLcYY
Content-Disposition: form-data; name="file"
Content-Type: text/plain;charset=UTF-8
Content-Length: 5

2;3;4
--059h2BBM-KlM_XP2rY8W1X3_jnzFLcYY—

The problem is the following line:

Content-Type: multipart/form-data;boundary=059h2BBM-KlM_XP2rY8W1X3_jnzFLcYY;charset=UTF-8

Spring Rest Template adds ;charset=UTF-8 to the content type and this leads to problems when. I found no possibility to remove the charset. From my point of view this is a bug since the charset should not be part of the content type?!


Affects: 5.0.7

Issue Links:

  • #21599 Multipart: Invalid boundary with RestTemplate (“is duplicated by”)

Referenced from: commits https://github.com/spring-projects/spring-framework/commit/f89511e7fe9c32ad28de2c00a6197ca9ad6ce937, https://github.com/spring-projects/spring-framework/commit/390bb871d85f56aaeed98aeb45d688fcf35af8e7

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:13

github_iconTop GitHub Comments

1reaction
spring-projects-issuescommented, Jan 11, 2019

bissorc commented

Brian Clozel: for the H2O server this will work:

multipart/form-data;charset=UTF-8;boundary=5dbb2562-8a5d-48e2-bac6-3539964d75e5
Response(status=200, payload={destination_frame=upload_a794bb7e848fb4f86b31169963ca41fb, total_bytes=5}, headers=ResponseHeaders(headers={null=[HTTP/1.1 200 OK], X-h2o-context-path=[/], X-h2o-cluster-good=[true], X-h2o-rest-api-version-max=[3], X-h2o-cluster-id=[1532014457068], X-h2o-build-project-version=[3.20.0.2], Content-Length=[85], Content-Type=[application/json;charset=ISO-8859-1]}, contentLength=85, contentType=application/json;charset=ISO-8859-1, contentETag=null, lastModified=1970-01-01T00:00:00Z, contentDisposition=null, location=null), serverUnreachable=false)
0reactions
spring-projects-issuescommented, Jan 11, 2019

Joe Amoros commented

Brian Clozel - Hmm bummer. Just gave it a try. I see the change, but still getting the error. Here’s the logging and below, the error. I removed the body from the logging to save space. I’ll keep digging. Thanks again!

[org.apache.http.wire] - < >> "POST /documentUpload/multipleUpload?j_username=***&j_password=***&ert=json&uploaderEmail=*** HTTP/1.1[\r][\n]">
[org.apache.http.wire] - < >> "Accept: application/json, application/*+json[\r][\n]">
[org.apache.http.wire] - < >> "Content-Type: multipart/form-data;charset=UTF-8;boundary=DClQ6zimejfgBMQYPEmNpyOS0f6426[\r][\n]">
[org.apache.http.wire] - < >> "Content-Length: 32084[\r][\n]">
[org.apache.http.wire] - < >> "Host: ***[\r][\n]">
[org.apache.http.wire] - < >> "Connection: Keep-Alive[\r][\n]">
[org.apache.http.wire] - < >> "User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_112)[\r][\n]">
[org.apache.http.wire] - < >> "[\r][\n]">
[org.apache.http.headers] - <>> POST /documentUpload/multipleUpload?j_username=***&j_password=***&ert=json&uploaderEmail=*** HTTP/1.1>
[org.apache.http.headers] - <>> Accept: application/json, application/*+json>
[org.apache.http.headers] - <>> Content-Type: multipart/form-data;charset=UTF-8;boundary=DClQ6zimejfgBMQYPEmNpyOS0f6426>
[org.apache.http.headers] - <>> Content-Length: 32084>
[org.apache.http.headers] - <>> Host: ***>
[org.apache.http.headers] - <>> Connection: Keep-Alive>
[org.apache.http.headers] - <>> User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_112)>
[org.apache.http.wire] - < >> "--DClQ6zimejfgBMQYPEmNpyOS0f6426[\r][\n]">
[org.apache.http.wire] - < >> "Content-Disposition: form-data; name="files"; filename="99999888_inv_LP1003.99999888_invStmt_20161001_20161231_invStmt_.pdf"[\r][\n]">
[org.apache.http.wire] - < >> "Content-Type: application/pdf[\r][\n]">
[org.apache.http.wire] - < >> "Content-Length: 31830[\r][\n]">
[org.apache.http.wire] - < >> "[\r][\n]">
[[BODY]]
[org.apache.http.wire] - < >> "--DClQ6zimejfgBMQYPEmNpyOS0f6426--[\r][\n]">
[org.apache.http.wire] - < << "HTTP/1.1 400 Bad Request[\r][\n]">
[org.apache.http.wire] - < << "Date: Fri, 20 Jul 2018 20:12:04 GMT[\r][\n]">
[org.apache.http.wire] - < << "Server: ***[\r][\n]">
[org.apache.http.wire] - < << "Content-Length: 226[\r][\n]">
[org.apache.http.wire] - < << "Connection: close[\r][\n]">
[org.apache.http.wire] - < << "Content-Type: text/html; charset=iso-8859-1[\r][\n]">
[org.apache.http.wire] - < << "[\r][\n]">[org.apache.http.impl.conn.DefaultClientConnection] - <Receiving response: HTTP/1.1 400 Bad Request>
[org.apache.http.headers] - <<< HTTP/1.1 400 Bad Request>
[org.apache.http.headers] - <<< Date: Fri, 20 Jul 2018 20:12:04 GMT>
[org.apache.http.headers] - <<< Server: ***>
[org.apache.http.headers] - <<< Content-Length: 226>
[org.apache.http.headers] - <<< Connection: close>
[org.apache.http.headers] - <<< Content-Type: text/html; charset=iso-8859-1>

From Apache/ModSecurity

[Fri Jul 20 16:12:04.012821 2018] [:error] [pid 31042:tid 140693741733632] [client 10.139.48.106] ModSecurity: Multipart parsing error (init): Multipart: Invalid boundary in C-T (malformed). [hostname "***"] [uri "/restless/documentUpload/multipleUpload"] [unique_id "W1JCFH8AAAEAAHlC1@8AAABM"]

[Fri Jul 20 16:12:04.247460 2018] [:error] [pid 31042:tid 140693741733632] [client 10.139.48.106] ModSecurity: Access denied with code 400 (phase 2). Match of "eq 0" against "REQBODY_ERROR" required. [file "/etc/modsecurity/modsecurity.conf"] [line "57"] [id "200001"] [msg "Failed to parse request body."] [data "Multipart: Invalid boundary in C-T (malformed)."] [severity "CRITICAL"] [hostname "***"] [uri "/restless/documentUpload/multipleUpload"] [unique_id "W1JCFH8AAAEAAHlC1@8AAABM"] 

A little extra info, this is the config that is failing in modsecurity

1. Verify that we've correctly processed the request body.
1. As a rule of thumb, when failing to process a request body
1. you should reject the request (when deployed in blocking mode)
1. or log a high-severity alert (when deployed in detection-only mode).
#
SecRule REQBODY_ERROR "!@eq 0" \
"id:'200001', phase:2,t:none,log,deny,status:400,msg:'Failed to parse request body.',logdata:'%{reqbody_error_msg}',severity:2" 

One last bit, if I add a whitelist in the modsecurity config like so:

SecRule REQUEST_URI "/multipleUpload" "allow,id:999999"

The file uploads, but we still get a warning:

 [Mon Jul 23 11:56:06.654191 2018] [:error] [pid 15106:tid 139627239241472] [client 10.139.48.106] ModSecurity: Multipart parsing error (init): Multipart: Invalid boundary in C-T (malformed). [hostname "****"] [uri "/restless/documentUpload/multipleUpload"] [unique_id "W1X6ln8AAAEAADsC@EIAAAAT"]

[Mon Jul 23 11:56:06.702321 2018] [:error] [pid 15106:tid 139627239241472] [client 10.139.48.106] ModSecurity: Access allowed (phase 2). Pattern match "/multipleUpload" at REQUEST_URI. [file "/etc/modsecurity/modsecurity.conf"] [line "20"] [id "999999"] [hostname "***"] [uri "/restless/documentUpload/multipleUpload"] [unique_id "W1X6ln8AAAEAADsC@EIAAAAT"]

Cheers!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Content type 'multipart/form-data;boundary=----...;charset=UTF ...
In case of MultipartFile you can't use JSON data so you can't use @RequestBody. Try with @ModelAttribute annotation. Working sample :
Read more >
Charset Content-Type Option not respected when using ...
When invoking Oracle Service Bus (OSB) using a multipart request where the charset is only specified in the part headers, OSB does not...
Read more >
multipart/form-data text/plain resttemplate - You.com
How to send Multipart form data with restTemplate Spring-mvc ... multipart request header not parsed correctly because of charset [SPR-17030]#21568.
Read more >
Description of multipart/mixed Internet message format
A multipart/mixed MIME message is composed of a mix of different data types. Each body part is delineated by a boundary. The boundary...
Read more >
Troubleshooting issues with adding attachments to rest ...
The Content-Type header should be set to multipart/form-data or multipart/mixed. ... does not recognize the boundary with quotes in multipart/form-data.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found