question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

String size exceeds the maximum allowed size of 131072 in U-SQL script

See original GitHub issue

Hello, I am hitting the error of string data type size limitation of 128KB during extract phase.

I tried the option of Extractors.Tsv(silent:true) to omit the corresponding rows which does not fit the 128KB but that doesn’t seem to solve the problem.

I do not want to parse using custom extractors and want to consume the entire line and then process if necessary.

Code snippet: @eventLogs = EXTRACT Event string FROM @path USING Extractors.Tsv(silent:true);

Error: MESSAGE: String size 215710 exceeds the maximum allowed size of 131072

Questions: • Is there a way to store larger literals than 128KB during extraction? • If not is there a way to omit those lines instead of run-time failure?

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:11

github_iconTop GitHub Comments

2reactions
narensoftcommented, Jan 2, 2018

Can you please share the sample with us @MikeRys and @ullasm

0reactions
anbudhanushcommented, Nov 28, 2018

Hi Mike, Big thank you for this piece. It is a life saver. Is it mandatory to have the start the document with “[” and end with “]”. Sometimes I have to churn bigdata sets of json through this extractor and it is bit of work to add these brackets in those massive files. I do it with emeditor.

Read more comments on GitHub >

github_iconTop Results From Across the Web

What is the maximum allowed size for String in U-SQL?
Currently the maximum allowed size for a string in U-SQL is 128 KB. If you need to handle larger sizes than that for...
Read more >
String size exceeds than maximum allowed
to assign the body text. Problem is that my body text exceeds the maximum character of 8000 (which is allowed in varchar(max)).
Read more >
Taking a byte out of U-SQL
This now produced a different error: message":"Row size exceeds the maximum allowed size of 4194304 bytes","description": It seemed some of the ...
Read more >
but its maximum row size exceeds the allowed ...
Hi Experts, I am facing this issue of "but its maximum row size exceeds the allowed maximum of 8060 bytes. INSERT or UPDATE...
Read more >
Overview
Found strings which match to known social media urls ... Report size exceeded maximum capacity and may have missing network information.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found