Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Requests missing UTF8 support

See original GitHub issue

Category

  • Bug

Describe the bug

Encoding error when trying to execute batch request with utf-8 characters, e.g., å, ø.

Steps to reproduce

I’ve been trying to create navigation nodes within batch, and several of them contain UTF8 characters. Function to create navigation level:

        public async Task CreateNavigationNodeLevel(IList<SharePointNavigationNodeModel> newNavigation)
        {
            Batch batch = _ctx.NewBatch();
            var requests = new List<IBatchSingleResult<BatchResultValue<string>>>();
            foreach (var item in newNavigation)
            {
                var URI = "_api/web/navigation/TopNavigationBar";
                var body = JsonConvert.SerializeObject(item, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore });
                requests.Add(await _ctx.Web.ExecuteRequestBatchAsync(
                    batch,
                    new ApiRequest(HttpMethod.Post, ApiRequestType.SPORest, URI, body)
                ));
            }
            await _ctx.ExecuteAsync(batch);
        }

Where SharePointNavigationNodeModel is:

    public class SharePointNavigationNodeModel
    {
        public int Id { get; set; }
        public string Url { get; set; }
        public string Title { get; set; }
        public List<string> AudienceIds { get; set; }
        public ItemType __metadata = new ItemType();
    }

    public class ItemType
    {
        public string type = "SP.NavigationNode";
    }

Execute as following:

await CreateNavigationNodeLevel(new List<SharePointNavigationNodeModel>{new SharePointNavigationNodeModel{Url="http://localhost",Title="Forretningsområder"}});

This is an example with one navigation link being put into batch. In additional context you can see batch requests with 5 links where 3 are with UTF8 characters and you can see what error message each response gives.

Expected behavior

Links are getting created.

Environment details (development & target environment)

  • SDK version: 1.4.0
  • OS: Windows 10
  • SDK used in: .NET Core Azure Function App v3
  • Framework: .NET Core v3.1
  • Browser(s): N/A
  • Tooling: both VS Code and Visual Studio 2019
  • Additional details: N/A

Additional context

Request information

"--batch_632b1836-ee40-44e4-aa64-ba0993cb0dc6
Content-Type: multipart/mixed; boundary=\"changeset_e599ca54-ad0f-416b-b0d6-bee5952a883b\"

--changeset_e599ca54-ad0f-416b-b0d6-bee5952a883b
Content-Type: application/http
Content-Transfer-Encoding:binary

POST https://contoso.sharepoint.com/sites/Norway/_api/web/navigation/TopNavigationBar HTTP/1.1
Accept: application/json;odata=verbose
Content-Type: application/json;odata=verbose
Content-Length: 111
If-Match: *

{\"__metadata\":{\"type\":\"SP.NavigationNode\"},\"Id\":0,\"Url\":\"http://linkless.header/\",\"Title\":\"Medarbeider i Atea\"}

--changeset_e599ca54-ad0f-416b-b0d6-bee5952a883b--
--batch_632b1836-ee40-44e4-aa64-ba0993cb0dc6
Content-Type: multipart/mixed; boundary=\"changeset_14230860-620b-409e-b815-63d157f2379c\"

--changeset_14230860-620b-409e-b815-63d157f2379c
Content-Type: application/http
Content-Transfer-Encoding:binary

POST https://contoso.sharepoint.com/sites/Norway/_api/web
/navigation/TopNavigationBar HTTP/1.1
Accept: application/json;odata=verbose
Content-Type: application/json;odata=verbose
Content-Length: 110
If-Match: *

{\"__metadata\":{\"type\":\"SP.NavigationNode\"},\"Id\":0,\"Url\":\"http://linkless.header/\",\"Title\":\"Salg og tjenester\"}

--changeset_14230860-620b-409e-b815-63d157f2379c--
--batch_632b1836-ee40-44e4-aa64-ba0993cb0dc6
Content-Type: multipart/mixed; boundary=\"changeset_f7d9d356-302d-4e35-a559-a8d2c684d0aa\"

--changeset_f7d9d356-302d-4e35-a559-a8d2c684d0aa
Content-Type: application/http
Content-Transfer-Encoding:binary

POST https://contoso.sharepoint.com/sites/Norway/_api/web/navigation/TopNavigationBar HTTP/1.1
Accept: application/json;odata=verbose
Content-Type: application/json;odata=verbose
Content-Length: 111
If-Match: *

{\"__metadata\":{\"type\":\"SP.NavigationNode\"},\"Id\":0,\"Url\":\"http://linkless.header/\",\"Title\":\"Forretningsområder\"}

--changeset_f7d9d356-302d
-4e35-a559-a8d2c684d0aa--
--batch_632b1836-ee40-44e4-aa64-ba0993cb0dc6
Content-Type: multipart/mixed; boundary=\"changeset_79734a9b-ee5a-4784-bd25-23c1eb4f6cc7\"

--changeset_79734a9b-ee5a-4784-bd25-23c1eb4f6cc7
Content-Type: application/http
Content-Transfer-Encoding:binary

POST https://contoso.sharepoint.com/sites/Norway/_api/web/navigation/TopNavigationBar HTTP/1.1
Accept: application/json;odata=verbose
Content-Type: application/json;odata=verbose
Content-Length: 114
If-Match: *

{\"__metadata\":{\"type\":\"SP.NavigationNode\"},\"Id\":0,\"Url\":\"http://linkless.header/\",\"Title\":\"IT-støtte og systemer\"}

--changeset_79734a9b-ee5a-4784-bd25-23c1eb4f6cc7--
--batch_632b1836-ee40-44e4-aa64-ba0993cb0dc6
Content-Type: multipart/mixed; boundary=\"changeset_e0d95e1e-123e-48cb-adb6-d4f90a59de56\"

--changeset_e0d95e1e-123e-48cb-adb6-d4f90a59de56
Content-Type: application/http
Content-Transfer-Encoding:binary

POST https://contoso.share
point.com/sites/Norway/_api/web/navigation/TopNavigationBar HTTP/1.1
Accept: application/json;odata=verbose
Content-Type: application/json;odata=verbose
Content-Length: 109
If-Match: *

{\"__metadata\":{\"type\":\"SP.NavigationNode\"},\"Id\":0,\"Url\":\"http://linkless.header/\",\"Title\":\"Vår organisasjon\"}

--changeset_e0d95e1e-123e-48cb-adb6-d4f90a59de56--
--batch_632b1836-ee40-44e4-aa64-ba0993cb0dc6--
"

Batch response:

"--batchresponse_1133751b-b1cf-4524-a7ee-a11ab82301c0
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 201 Created
CONTENT-TYPE: application/json;odata=verbose;charset=utf-8
LOCATION: https://contoso.sharepoint.com/sites/Norway/_api/Web/Navigation/GetNodeById(3812)

{\"d\":{\"__metadata\":{\"id\":\"https://contoso.sharepoint.com/sites/Norway/_api/Web/Navigation/GetNodeById(3812)\",\"uri\":\"https://contoso.sharepoint.com/sites/Norway/_api/Web/Navigation/GetNodeById(3812)\",\"type\":\"SP.NavigationNode\"},\"Children\":{\"__deferred\":{\"uri\":\"https://contoso.sharepoint.com/sites/Norway/_api/Web/Navigation/GetNodeById(3812)/Children\"}},\"TitleResource\":{\"__deferred\":{\"uri\":\"https://contoso.sharepoint.com/sites/Norway/_api/Web/Navigation/GetNodeById(3812)/TitleResource\"}},\"AudienceIds\":null,\"CurrentLCID\":1033,\"Id\":3812,\"IsDocLib\":true,\"IsExternal\":true,\"IsVisible\":true,\"ListTemplateType\":0,\"Title\":\"Medarbeider i Atea\",\"Url\":
\"http://linkless.header/\"}}
--batchresponse_1133751b-b1cf-4524-a7ee-a11ab82301c0
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 201 Created
CONTENT-TYPE: application/json;odata=verbose;charset=utf-8
LOCATION: https://contoso.sharepoint.com/sites/Norway/_api/Web/Navigation/GetNodeById(3813)

{\"d\":{\"__metadata\":{\"id\":\"https://contoso.sharepoint.com/sites/Norway/_api/Web/Navigation/GetNodeById(3813)\",\"uri\":\"https://contoso.sharepoint.com/sites/Norway/_api/Web/Navigation/GetNodeById(3813)\",\"type\":\"SP.NavigationNode\"},\"Children\":{\"__deferred\":{\"uri\":\"https://contoso.sharepoint.com/sites/Norway/_api/Web/Navigation/GetNodeById(3813)/Children\"}},\"TitleResource\":{\"__deferred\":{\"uri\":\"https://contoso.sharepoint.com/sites/Norway/_api/Web/Navigation/GetNodeById(3813)/TitleResource\"}},\"AudienceIds\":null,\"CurrentLCID\":1033,\"Id\":3813,\"IsDocLib\":true,\"IsExternal\":true,\"IsVisible\":true,\"ListTemplateType\":0,\"Title\"
:\"Salg og tjenester\",\"Url\":\"http://linkless.header/\"}}
--batchresponse_1133751b-b1cf-4524-a7ee-a11ab82301c0
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 400 Bad Request
CONTENT-TYPE: application/json;odata=verbose;charset=utf-8

{\"error\":{\"code\":\"-1, Microsoft.SharePoint.Client.InvalidClientQueryException\",\"message\":{\"lang\":\"en-US\",\"value\":\"Invalid JSON. Unexpected end of input was found in JSON content. Not all object and array scopes were closed.\"}}}
--batchresponse_1133751b-b1cf-4524-a7ee-a11ab82301c0
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 400 Bad Request
CONTENT-TYPE: application/json;odata=verbose;charset=utf-8

{\"error\":{\"code\":\"-1, Microsoft.SharePoint.Client.InvalidClientQueryException\",\"message\":{\"lang\":\"en-US\",\"value\":\"Invalid JSON. Unexpected end of input was found in JSON content. Not all object and array scopes were closed.\"}}}
--batchresponse_1
133751b-b1cf-4524-a7ee-a11ab82301c0
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 400 Bad Request
CONTENT-TYPE: application/json;odata=verbose;charset=utf-8

{\"error\":{\"code\":\"-1, Microsoft.SharePoint.Client.InvalidClientQueryException\",\"message\":{\"lang\":\"en-US\",\"value\":\"Invalid JSON. Unexpected end of input was found in JSON content. Not all object and array scopes were closed.\"}}}
--batchresponse_1133751b-b1cf-4524-a7ee-a11ab82301c0--
"

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
jansenbecommented, Nov 2, 2021

@wonderplayer : docs are updated (see https://pnp.github.io/pnpcore/using-the-sdk/basics-customapirequests.html#making-a-custom-spo-rest-request). Closing this one now. Feel free to re-open if you still have issues.

Note: we plan to add navigation support in the future, it’s a matter or resourcing but eventually we’ll get that done.

1reaction
wonderplayercommented, Nov 2, 2021

Oh, wow. I thought that there is something I should be able to do, but for some reason this thought didn’t pop up in my mind 😄 Maybe this information could be included in custom request documentation?

Read more comments on GitHub >

github_iconTop Results From Across the Web

utf 8 - python requests.get() returns improperly decoded ...
When you make a request, Requests makes educated guesses about the encoding of the response based on the HTTP headers. The text encoding...
Read more >
Can't POST UTF-8 encoded data · Issue #3476 · psf/requests
I'm using Requests to send data to a Firebase database. Data contained in the JSON file is UTF-8 encoded. I'm sending that data...
Read more >
UTF-8 support for uniform resource locators
If any applications fail with Bad Request errors, try the application with the utf8-url-support-enabled stanza entry set to no . If this step...
Read more >
requests 2.12.5
Requests is the only Non-GMO HTTP library for Python, safe for human consumption. Warning: Recreational use of other HTTP libraries may result in...
Read more >
UTF-8
UTF-8 is a variable-length character encoding standard used for electronic communication. ... Any bytes missing from UTF-8 will still allow the text to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Error: Task Canceled when loading items by Caml query

Next page

Improve support for working with fields via a "typed" field collection