Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Go client Reader panics when passing custom StartMessageID

See original GitHub issue

Expected behavior

I can pass arbitrary type implementing the pulsar.MessageID interface in pulsar.ReaderOptionswhen creating a Reader instance.

Actual behavior

When I pass a custom type the client library panics:

panic: interface conversion: pulsar.MessageID is mypackage.pulsarMessageID, not *pulsar.messageID
	/usr/local/go/src/runtime/panic.go:513 +0x1b9
github.com/apache/pulsar/pulsar-client-go/pulsar.createReaderAsync(0xc0002229a0, 0xc0001fe29a, 0x9, 0x0, 0x0, 0xb800a0, 0xc0002fb6e0, 0xc00018dd40, 0x0, 0x0, ...)
	/go/pkg/mod/github.com/apache/pulsar@v2.2.0+incompatible/pulsar-client-go/pulsar/c_reader.go:115 +0x504
github.com/apache/pulsar/pulsar-client-go/pulsar.(*client).CreateReaderAsync(0xc0002229a0, 0xc0001fe29a, 0x9, 0x0, 0x0, 0xb800a0, 0xc0002fb6e0, 0x0, 0x0, 0x0, ...)
	/go/pkg/mod/github.com/apache/pulsar@v2.2.0+incompatible/pulsar-client-go/pulsar/c_client.go:220 +0x65
github.com/apache/pulsar/pulsar-client-go/pulsar.(*client).CreateReader(0xc0002229a0, 0xc0001fe29a, 0x9, 0x0, 0x0, 0xb800a0, 0xc0002fb6e0, 0x0, 0x0, 0x0, ...)
	/go/pkg/mod/github.com/apache/pulsar@v2.2.0+incompatible/pulsar-client-go/pulsar/c_client.go:207 +0xca

The culprit is the following line:

// Unchecked type assertion assuming that StartMessageID is of type *messageID
C._pulsar_client_create_reader_async(client.ptr, topic, options.StartMessageID.(*messageID).ptr,
	conf, savePointer(&readerAndCallback{reader, conf, callback}))

Steps to reproduce

See above

System configuration

Pulsar version: 2.2 github.com/apache/pulsar v2.2.0+incompatible h1:i2dfg67DNdHRgdzdIiSObTb9VRavAtZStxC4YIySxAI= github.com/apache/pulsar v2.2.0+incompatible/go.mod h1:7npmYEEu+jEeIMdpGSJwrTxUB86OW/UM5IkJCAyfl5Y= g

Issue Analytics

State:
Created 5 years ago
Comments:7 (4 by maintainers)

Top GitHub Comments

1reaction

nitishvcommented, Jul 5, 2020

@wolfstudy the same panic happens with native go client

Here’s what I tried.

Added a unit test which creates custom message ID type, satisfying this interface https://github.com/apache/pulsar-client-go/blob/master/pulsar/message.go#L108

$ git diff pulsar/reader_test.go                                                                                                         
diff --git a/pulsar/reader_test.go b/pulsar/reader_test.go                                                                                                                        
index c98771d..988f0cf 100644                                                                                                                                                     
--- a/pulsar/reader_test.go                                                                                                                                                       
+++ b/pulsar/reader_test.go                   
@@ -362,3 +362,58 @@ func TestReaderHasNext(t *testing.T) {
 
        assert.Equal(t, 10, i)
 }
+
+type myMessageID struct {
+       data []byte
+}
+
+func (id *myMessageID) Serialize() []byte {
+       return id.data
+}
+
+func TestReaderWithCustomMessageID(t *testing.T) {
+       client, err := NewClient(ClientOptions{
+               URL: lookupURL,
+       })
+
+       assert.Nil(t, err)
+       defer client.Close()
+
+       topic := newTopicName()
+       ctx := context.Background()
+
+       // create producer
+       producer, err := client.CreateProducer(ProducerOptions{
+               Topic:           topic,
+               DisableBatching: true,
+       })
+       assert.Nil(t, err)
+       defer producer.Close()
+
+       // send 10 messages
+       msgIDs := [10]MessageID{}
+       for i := 0; i < 10; i++ {
+               msgID, err := producer.Send(ctx, &ProducerMessage{
+                       Payload: []byte(fmt.Sprintf("hello-%d", i)),
+               })
+               assert.NoError(t, err)
+               assert.NotNil(t, msgID)
+               msgIDs[i] = msgID
+       }
+
+       myMsg := &myMessageID{
+               data: msgIDs[4].Serialize(),
+       }
+
+       myMsg := &myMessageID{
+               data: msgIDs[4].Serialize(),
+       }
+
+       // attempt to create reader on 5th message (not included)
+       var reader Reader
+       assert.NotPanics(t, func() {
+               reader, err = client.CreateReader(ReaderOptions{
+                       Topic:          topic,
+                       StartMessageID: myMsg,
+               })
+       })
+
+       assert.Nil(t, err)
+       defer reader.Close()
+}

Ran the modified UT using the CI bash script

NVASHISH-M-93PK:pulsar-client-go nvashish$ git diff run-ci.sh 
diff --git a/run-ci.sh b/run-ci.sh
index 2de0eee..7507b68 100755
--- a/run-ci.sh
+++ b/run-ci.sh
@@ -31,7 +31,7 @@ go build -o pulsar-perf ./perf
 
 ./pulsar-test-service-start.sh
 
-go test -race -coverprofile=/tmp/coverage -timeout=20m ./...
+go test -race -coverprofile=/tmp/coverage -timeout=20m ./... -run=TestReaderWithCustomMessageID
 go tool cover -html=/tmp/coverage -o coverage.html
 
 ./pulsar-test-service-stop.sh

Test case failed with panic, even though it compiles.

-- Ready to start tests                                                                                                                                                           
+ go test -race -coverprofile=/tmp/coverage -timeout=20m ./... -run=TestReaderWithCustomMessageID                                                                                 
?       github.com/apache/pulsar-client-go/examples/consumer    [no test files]                                                                                                   
?       github.com/apache/pulsar-client-go/examples/consumer-listener   [no test files]                                                                                           
?       github.com/apache/pulsar-client-go/examples/producer    [no test files]                                                                                                   
?       github.com/apache/pulsar-client-go/examples/reader      [no test files]                                                                                                   
ok      github.com/apache/pulsar-client-go/integration-tests    1.022s  coverage: 0.0% of statements [no tests to run]                                                            
?       github.com/apache/pulsar-client-go/perf [no test files]                                                                                                                   
time="2020-07-05T05:54:05Z" level=info msg="Connecting to broker" remote_addr="pulsar://localhost:6650"                                                                           
time="2020-07-05T05:54:05Z" level=info msg="TCP connection established" local_addr="127.0.0.1:49144" remote_addr="pulsar://localhost:6650"                                        
time="2020-07-05T05:54:05Z" level=info msg="Connection is ready" local_addr="127.0.0.1:49144" remote_addr="pulsar://localhost:6650"                                               
time="2020-07-05T05:54:06Z" level=info msg="Created producer" cnx="127.0.0.1:49144 -> 127.0.0.1:6650" producer_name=standalone-0-0 topic="persistent://public/default/my-topic-501
323100"                                                                                                                                                                           
time="2020-07-05T05:54:06Z" level=info msg="Closing producer" producer_name=standalone-0-0 topic="persistent://public/default/my-topic-501323100"                                 
time="2020-07-05T05:54:06Z" level=info msg="Closed producer" producer_name=standalone-0-0 topic="persistent://public/default/my-topic-501323100"                                  
--- FAIL: TestReaderWithCustomMessageID (1.14s)                                                                                                                                   
    reader_test.go:410:                                                                                                                                                           
                Error Trace:    reader_test.go:410                                                                                                                                
                Error:          func (assert.PanicTestFunc)(0xba3710) should not panic                                                                                            
                                        Panic value:    interface conversion: pulsar.MessageID is *pulsar.myMessageID, not *pulsar.messageID                                      
                Test:           TestReaderWithCustomMessageID                                                                                                                     
panic: runtime error: invalid memory address or nil pointer dereference [recovered]                                                                                               
        panic: runtime error: invalid memory address or nil pointer dereference                                                                                                   
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xb9e8f2]                                                                                                            
                                                                                                                                                                                  
goroutine 20 [running]:                                                                                                                                                           
testing.tRunner.func1(0xc000124200)                                                                                                                                               
        /usr/local/go/src/testing/testing.go:830 +0x69d                                                                                                                           
panic(0xcf0820, 0x13abde0)                                                                                                                                                        
        /usr/local/go/src/runtime/panic.go:522 +0x1b5                                                                                                                             
github.com/apache/pulsar-client-go/pulsar.TestReaderWithCustomMessageID(0xc000124200)                                                                                             
        /pulsar-client-go/pulsar/reader_test.go:418 +0x9c2                                                                                                                        
testing.tRunner(0xc000124200, 0xda0408)                                                                                                                                           
        /usr/local/go/src/testing/testing.go:865 +0x164                                                                                                                           
created by testing.(*T).Run                                                                                                                                                       
        /usr/local/go/src/testing/testing.go:916 +0x65b                                                                                                                           
FAIL    github.com/apache/pulsar-client-go/pulsar       1.164s                                                                                                                    
ok      github.com/apache/pulsar-client-go/pulsar/internal      1.029s  coverage: 0.0% of statements [no tests to run]                                                            
ok      github.com/apache/pulsar-client-go/pulsar/internal/auth 1.021s  coverage: 0.0% of statements [no tests to run]                                                            
ok      github.com/apache/pulsar-client-go/pulsar/internal/compression  1.025s  coverage: 17.9% of statements [no tests to run]                                                   
?       github.com/apache/pulsar-client-go/pulsar/internal/pulsar_proto [no test files]

Should I open an issue for native go client repo referencing this issue?

0reactions

wolfstudycommented, Jul 6, 2020

Cool Thanks to @nitishv the https://github.com/apache/pulsar-client-go/pull/305 has been merged to master, will close the issue.

Top Results From Across the Web

Go client Reader panics when passing custom StartMessageID ...

Reader client's CreateReader API should not panic at runtime when using a custom type ... Go client Reader panics when passing custom StartMessageID...

pulsar - Go Packages

NackBackoffPolicy is a interface for custom message negativeAcked policy, users can specify a NackBackoffPolicy for a consumer. > Notice: the ...

Pulsar Go client

Download the library of Go client to local environment: ... MessageRouter set a custom message routing policy by passing an ... StartMessageID: pulsar....

Accessing Kubernetes CRDs from the client-go package

The Kubernetes API server is easily extendable by Custom Resource Defintions. However, actually accessing these resources from the popular ...

CHANGELOG.md · Gitee 极速下载/pulsar-client-go - Gitee.com

由于 Go 语言与生俱来的优势以及 Go 社区的不断壮大，Pulsar 社区对 Go Client 的呼声 ... Fix panic in CreateReader API using custom MessageID for ReaderOptions, ...