question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

`ListGroup` throws `OverflowException`

See original GitHub issue

Description

ListGroup intermittently throws OverflowException from the librdkafka bindings. It appears to happen when group membership information reaches a certain size.

How to reproduce

With Confluent.Kafka 2.0.2 on .NET 6:

using Confluent.Kafka;
using System;
using System.Collections.Generic;
using System.Threading.Tasks;

var broker = $"bootstrap:9092";
var groupId = $"test-group-{Guid.NewGuid()}";
var topicName = "test-topic";

var adminConfig = new AdminClientConfig
{
    BootstrapServers = broker,
};

var consumerConfig = new ConsumerConfig
{
    BootstrapServers = broker,
    EnableAutoCommit = false,
    GroupId = groupId,
};

var admin = new AdminClientBuilder(adminConfig)
    .Build();

for (var i = 0; i < 10; i++)
{
    var consumer = new ConsumerBuilder<Ignore, Ignore>(consumerConfig).Build();

    consumer.Subscribe(topicName);
    await Task.Delay(TimeSpan.FromSeconds(5));

    var info = admin.ListGroup(groupId, TimeSpan.FromSeconds(5));
    Console.WriteLine($"{i}: {groupId} has {info.Members.Count} members");
}

After a few members (locally, 3 seems to do it), this program crashes:

0: test-group-496fec82-92e4-4d54-9d51-ce769e149134 has 1 members
1: test-group-496fec82-92e4-4d54-9d51-ce769e149134 has 2 members
Unhandled exception. System.OverflowException: Arithmetic operation resulted in an overflow.
   at System.IntPtr.op_Explicit(IntPtr value)
   at Confluent.Kafka.Impl.SafeKafkaHandle.CopyBytes(IntPtr ptr, IntPtr len)
   at Confluent.Kafka.Impl.SafeKafkaHandle.<>c.<ListGroupsImpl>b__73_3(rd_kafka_group_member_info mi)
   at System.Linq.Utilities.<>c__DisplayClass2_0`3.<CombineSelectors>b__0(TSource x)
   at System.Linq.Enumerable.SelectRangeIterator`1.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Confluent.Kafka.Impl.SafeKafkaHandle.<>c.<ListGroupsImpl>b__73_1(rd_kafka_group_info gi)
   at System.Linq.Utilities.<>c__DisplayClass2_0`3.<CombineSelectors>b__0(TSource x)
   at System.Linq.Enumerable.SelectRangeIterator`1.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Confluent.Kafka.Impl.SafeKafkaHandle.ListGroupsImpl(String group, Int32 millisecondsTimeout)
   at Confluent.Kafka.Impl.SafeKafkaHandle.ListGroup(String group, Int32 millisecondsTimeout)
   at Confluent.Kafka.AdminClient.ListGroup(String group, TimeSpan timeout)
   at Program.<Main>$(String[] args) in /workspaces/ubuntu/Program.cs:line 35
   at Program.<Main>(String[] args)

Full logs here with Debug = "all" on adminConfig.

The problem is not reproducible with Confluent.Kafka 1.9.3.

Checklist

Please provide the following information:

  • A complete (i.e. we can run it), minimal program demonstrating the problem: above
  • Confluent.Kafka NuGet version: 2.0.2
  • Apache Kafka version: 2.7 (Confluent Platform 6.1.2)
  • Client configuration: above
  • Operating system: macOS (M1), Ubuntu 22.10 (x64)
  • Provide logs (with “debug” : “…” as necessary in configuration): run.log
  • Provide broker log excerpts: n/a
  • Critical issue.

Issue Analytics

  • State:closed
  • Created 7 months ago
  • Reactions:7
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

6reactions
anchitjcommented, Feb 20, 2023

This should be fixed in the coming release.

3reactions
anchitjcommented, Mar 9, 2023

@BEagle1984 Most likely end of this month of early next month.

Read more comments on GitHub >

github_iconTop Results From Across the Web

c# - List sum too large, throwing overflow exception
I'm getting a "Arithmetic operation resulted in an overflow" exception. I guess the sum is too large, but converting it to Int64 didn't...
Read more >
OverflowException Class (System)
The exception that is thrown when an arithmetic, casting, or conversion operation in a checked context results in an overflow.
Read more >
[Solved]-List sum too large, throwing overflow exception-LINQ ...
Coding example for the question List sum too large, throwing overflow exception-LINQ,C#.
Read more >
SharePoint Search BDC / BCS Model with Item Level ...
OverflowException : Length of the access control list exceed the allowed maximum. at System.Security.AccessControl.RawAcl.
Read more >
Netjs/mscorlib.ts at master · praeclarum/Netjs
... class OverflowException extends Exception { constructor() { super("Overflow"); } ... Groups: List<Group> = new List<Group> (); Success: boolean = false; } ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found