Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Azure Cosmos Error when exceeding the number of columns in select statement
See original GitHub issueWe are continuously addressing and improving the SDK, if possible, make sure the problem persist in the latest SDK version.
Describe the bug When trying to execute the following code, the error is thrown (Full error text is in additional context section):
public async STT.Task<List<CheetahReading>> GetWarriorReadingsAsync(int partitionKey, double fromDepth, double toDepth)
{
var container = await _containerFactory
.GetDefaultCosmosContainer()
.GetAsync();
List<CheetahReading> result = new List<CheetahReading>();
string sql = string.Empty;
double diff = toDepth - fromDepth;
// Big and ugly if statement, but still WIP
// Cosmos has limitiation on query language, so therefor the weird SQL.
// Using MIN and AVG depth to align the curves better.
sql = @"SELECT
AVG(c.DEPTH) MaxDEPTH,
MAX(c.LTEN) MaxLTEN,
MAX(c.LSPD) MaxLSPD,
MAX(c.CCL) MaxCCL,
MAX(c.MITROT) MaxMITROT,
MAX(c.MITDEV) MaxMITDEV,
MAX(c.MINDIA) MaxMINDIA,
MAX(c.MAXDIA) MaxMAXDIA,
MAX(c.AVEDIA) MaxAVEDIA,
MAX(IS_DEFINED(c.QP) = true ? c.QP : 0) MaxQP,
MAX(IS_DEFINED(c.CFB) = true ? c.CFB : 0) MaxCFB,
MAX(IS_DEFINED(c.CFS) = true ? c.CFS : 0) MaxCFS,
MAX(IS_DEFINED(c.TEMP) = true ? c.TEMP : 0) MaxTEMP,
MAX(IS_DEFINED(c.ILS) = true ? c.ILS : 0) MaxILS,
MIN(c.DEPTH) MinDEPTH,
MIN(c.LTEN) MinLTEN,
MIN(c.LSPD) MinLSPD,
MIN(c.CCL) MinCCL,
MIN(c.MITROT) MinMITROT,
MIN(c.MITDEV) MinMITDEV,
MIN(c.MINDIA) MinMINDIA,
MIN(c.MAXDIA) MinMAXDIA,
MIN(c.AVEDIA) MinAVEDIA,
MIN(IS_DEFINED(c.QP) = true ? c.QP : 0) MinQP,
MIN(IS_DEFINED(c.CFB) = true ? c.CFB : 0) MinCFB,
MIN(IS_DEFINED(c.CFS) = true ? c.CFS : 0) MinCFS,
MIN(IS_DEFINED(c.TEMP) = true ? c.TEMP : 0) MinTEMP,
MIN(IS_DEFINED(c.ILS) = true ? c.ILS : 0) MinILS
FROM c
WHERE c.JobId = @id
AND c.DEPTH >= @from
AND c.DEPTH <= @to";
if (diff < 1000)
{
sql += " GROUP BY TRUNC(c.DEPTH)";
}
else if (diff >= 1000)
{
sql += " GROUP BY TRUNC(c.DEPTH / 10)";
}
QueryDefinition query = new QueryDefinition(sql)
.WithParameter("@id", partitionKey)
.WithParameter("@from", fromDepth)
.WithParameter("@to", toDepth);
List<WarriorDepthReading> tmpResult = new List<WarriorDepthReading>();
using (FeedIterator<WarriorDepthReading> resultSetIterator = container.GetItemQueryIterator<WarriorDepthReading>(
query,
requestOptions: new QueryRequestOptions()
{
PartitionKey = new PartitionKey(partitionKey),
}))
{
while (resultSetIterator.HasMoreResults)
{
FeedResponse<WarriorDepthReading> response = await resultSetIterator.ReadNextAsync();
tmpResult.AddRange(response);
}
}
foreach (var tmp in tmpResult)
{
// Min value
result.Add(new CheetahReading
{
DEPTH = tmp.MinDEPTH,
LSPD = tmp.MinLSPD,
LTEN = tmp.MinLTEN,
CCL = tmp.MinCCL,
MITDEV = tmp.MinMITDEV,
MITROT = tmp.MinMITROT,
AVEDIA = tmp.MinAVEDIA,
MINDIA = tmp.MinMINDIA,
MAXDIA = tmp.MinMAXDIA,
QP = tmp.MinQP,
CFB = tmp.MinCFB,
CFS = tmp.MinCFS,
TEMP = tmp.MinTEMP,
ILS = tmp.MinILS,
});
//Max value
result.Add(new CheetahReading
{
DEPTH = tmp.MaxDEPTH,
LSPD = tmp.MaxLSPD,
LTEN = tmp.MaxLTEN,
CCL = tmp.MaxCCL,
MITDEV = tmp.MaxMITDEV,
MITROT = tmp.MaxMITROT,
AVEDIA = tmp.MaxAVEDIA,
MINDIA = tmp.MaxMINDIA,
MAXDIA = tmp.MaxMAXDIA,
QP = tmp.MaxQP,
CFB = tmp.MaxCFB,
CFS = tmp.MaxCFS,
TEMP = tmp.MaxTEMP,
ILS = tmp.MaxILS,
});
}
return result.OrderBy(x => x.DEPTH).ToList();
}
If we lower the number of select columns and the rest of the method GetWarriorReadingsAsync stays the same, the query gets executed successfully, for example,:
public async STT.Task<List<CheetahReading>> GetWarriorReadingsAsync(int partitionKey, double fromDepth, double toDepth)
{
...
sql = @"SELECT
AVG(c.DEPTH) MaxDEPTH,
MAX(c.LTEN) MaxLTEN,
MAX(c.LSPD) MaxLSPD,
MAX(c.CCL) MaxCCL,
MAX(c.MITROT) MaxMITROT,
MAX(c.MITDEV) MaxMITDEV,
MAX(c.MINDIA) MaxMINDIA,
MAX(c.MAXDIA) MaxMAXDIA,
MAX(c.AVEDIA) MaxAVEDIA,
MIN(c.DEPTH) MinDEPTH,
MIN(c.LTEN) MinLTEN,
MIN(c.LSPD) MinLSPD,
MIN(c.CCL) MinCCL,
MIN(c.MITROT) MinMITROT,
MIN(c.MITDEV) MinMITDEV,
MIN(c.MINDIA) MinMINDIA,
MIN(c.MAXDIA) MinMAXDIA,
MIN(c.AVEDIA) MinAVEDIA
FROM c
WHERE c.JobId = @id
AND c.DEPTH >= @from
AND c.DEPTH <= @to";
...
}
To Reproduce
Cosmos data looks like this:
Expected behavior There should be a warning on the maximum number of columns if that is applicable to this problem.
Actual behavior Non descriptive error thrown: An unknown error occurred while processing this request.
Environment summary SDK Version: Microsoft.Azure.Cosmos 3.20.1, .NET Standard 2.0.3 OS Version (Windows 10)
Additional context
Response status code does not indicate success: InternalServerError (500); Substatus: 0; ActivityId: dbf0edbd-7d80-49fb-9115-ce56f84ed14e; Reason: (Response status code does not indicate success: InternalServerError (500); Substatus: 0; ActivityId: dbf0edbd-7d80-49fb-9115-ce56f84ed14e; Reason: (Response status code does not indicate success: InternalServerError (500); Substatus: 0; ActivityId: dbf0edbd-7d80-49fb-9115-ce56f84ed14e; Reason: (Message: {"Errors":["An unknown error occurred while processing this request. If the issue persists, please contact Azure Support: http://aka.ms/azure-support"]}
ActivityId: dbf0edbd-7d80-49fb-9115-ce56f84ed14e, Request URI: /apps/36bb182d-27f9-412c-b700-da88dd239ac7/services/15ff138b-64ae-49d9-930b-a619b132a874/partitions/732df60d-8c6f-4d4c-ad1e-27390a4ce980/replicas/132660486141625479p/, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Windows/10.0.19042 cosmos-netstandard-sdk/3.19.3);););
Issue Analytics
- State:
- Created 2 years ago
- Reactions:1
- Comments:7 (3 by maintainers)
Top Results From Across the Web
How to increase the limit of columns per select statement in ...
I have dynamic query which has more than 4096 columns in select statement and I am getting this error while executing this. The...
Read more >Azure Cosmos DB Number of Columns Limit
According to my test(I tired to add 260 properties into an entity), Azure Cosmos DB Table API accept that properties exceed 255. enter...
Read more >Increasing or Decreasing Scale for Azure Cosmos DB
We cannot set a scale of 750 RUs for our SQL API database in our Azure Cosmos DB. We see that the error...
Read more >Features and pitfalls of Azure Cosmos DB | by Yuriy Ivon
⚠️ In case a database exceeds the provisioned throughput while running a query — it throttles incoming requests by immediately throwing special HTTP...
Read more >Azure Cosmos Query | Matillion ETL Docs
Provides an upper limit on the number of rows retrieved from the Azure Cosmos DB server. Blank means fetch all records. Type, Select,...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Hi, sorry for late response this is the full exception:
Hey @zejtin, let me know if you were able to get this issue resolved. Based on the query you shared, I suspect that you’ve hit the limit in number of aggregates per GROUP BY query. This is documented here: https://docs.microsoft.com/azure/cosmos-db/sql/sql-query-group-by#remarks.
We will work on making this error message more specific. Thanks!