Pulsar CPP client mem leak
See original GitHub issueDescribe the bug
pulsar host can’t connect successfuly, send pulsar will mem leak
To Reproduce Steps to reproduce the behavior:
- Config pulsar host can’t connect successfully
- Send a msg
- ClientConnection will mem leak
Expected behavior If Pulsar fails to connect, each message will leak memory
Code:
m_conf = pulsar_client_configuration_create();
pulsar_client_configuration_set_logger(m_conf, PulsarProducer::pulsar_logger, (void*)this);
m_client = pulsar_client_create(service_url.c_str(), m_conf);
pulsar_result err = pulsar_client_create_producer(m_client, topic.c_str(), producer_conf, &producer);
if (err != pulsar_result_Ok) {
LOG_ERROR("Failed to create producer: %s\n", pulsar_result_str(err));
pulsar_producer_configuration_free(producer_conf);
return NULL;
}
Screenshots valgrind
==00:00:41:04.793 10873== 592 bytes in 1 blocks are possibly lost in loss record 2,589 of 2,878
==00:00:41:04.793 10873== at 0x4C2BFD9: calloc (vg_replace_malloc.c:762)
==00:00:41:04.793 10873== by 0x4012774: _dl_allocate_tls (in /usr/lib64/ld-2.17.so)
==00:00:41:04.793 10873== by 0x505588B: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
==00:00:41:04.793 10873== by 0x13AE17E: __gthread_create (gthr-default.h:662)
==00:00:41:04.793 10873== by 0x13AE17E: std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>) (thread.cc:142)
==00:00:41:04.793 10873== by 0xCDB4B1: thread<std::_Bind<std::_Mem_fn<void (pulsar::ExecutorService::*)(std::shared_ptr<boost::asio::io_service>)>(pulsar::ExecutorService*, std::shared_ptr<boost::asio::io_service>)> > (thread:135)
==00:00:41:04.793 10873== by 0xCDB4B1: pulsar::ExecutorService::ExecutorService() (ExecutorService.cc:30)
==00:00:41:04.793 10873== by 0xCDB716: construct<pulsar::ExecutorService> (shared_ptr_base.h:396)
==00:00:41:04.793 10873== by 0xCDB716: _S_construct<pulsar::ExecutorService> (alloc_traits.h:254)
==00:00:41:04.793 10873== by 0xCDB716: construct<pulsar::ExecutorService> (alloc_traits.h:393)
==00:00:41:04.793 10873== by 0xCDB716: _Sp_counted_ptr_inplace<> (shared_ptr_base.h:399)
==00:00:41:04.793 10873== by 0xCDB716: construct<std::_Sp_counted_ptr_inplace<pulsar::ExecutorService, std::allocator<pulsar::ExecutorService>, (__gnu_cxx::_Lock_policy)2u>, const std::allocator<pulsar::ExecutorService> > (new_allocator.h:120)
==00:00:41:04.793 10873== by 0xCDB716: _S_construct<std::_Sp_counted_ptr_inplace<pulsar::ExecutorService, std::allocator<pulsar::ExecutorService>, (__gnu_cxx::_Lock_policy)2u>, const std::allocator<pulsar::ExecutorService> > (alloc_traits.h:254)
==00:00:41:04.793 10873== by 0xCDB716: construct<std::_Sp_counted_ptr_inplace<pulsar::ExecutorService, std::allocator<pulsar::ExecutorService>, (__gnu_cxx::_Lock_policy)2u>, const std::allocator<pulsar::ExecutorService> > (alloc_traits.h:393)
==00:00:41:04.793 10873== by 0xCDB716: __shared_count<pulsar::ExecutorService, std::allocator<pulsar::ExecutorService> > (shared_ptr_base.h:502)
==00:00:41:04.793 10873== by 0xCDB716: __shared_ptr<std::allocator<pulsar::ExecutorService> > (shared_ptr_base.h:957)
==00:00:41:04.793 10873== by 0xCDB716: shared_ptr<std::allocator<pulsar::ExecutorService> > (shared_ptr.h:316)
==00:00:41:04.793 10873== by 0xCDB716: allocate_shared<pulsar::ExecutorService, std::allocator<pulsar::ExecutorService> > (shared_ptr.h:598)
==00:00:41:04.793 10873== by 0xCDB716: make_shared<pulsar::ExecutorService> (shared_ptr.h:614)
==00:00:41:04.793 10873== by 0xCDB716: pulsar::ExecutorServiceProvider::get() (ExecutorService.cc:91)
==00:00:41:04.793 10873== by 0xCC5BE9: pulsar::ConnectionPool::getConnectionAsync(std::string const&, std::string const&) (ConnectionPool.cc:83)
==00:00:41:04.793 10873== by 0xDA61FF: pulsar::BinaryProtoLookupService::getPartitionMetadataAsync(std::shared_ptr<pulsar::TopicName> const&) (BinaryProtoLookupService.cc:72)
==00:00:41:04.793 10873== by 0xCB7C84: pulsar::ClientImpl::createProducerAsync(std::string const&, pulsar::ProducerConfiguration, std::function<void (pulsar::Result, pulsar::Producer)>) (ClientImpl.cc:158)
==00:00:41:04.793 10873== by 0xCB2074: pulsar::Client::createProducerAsync(std::string const&, pulsar::ProducerConfiguration, std::function<void (pulsar::Result, pulsar::Producer)>) (Client.cc:65)
==00:00:41:04.793 10873== by 0xCB27D9: pulsar::Client::createProducer(std::string const&, pulsar::ProducerConfiguration const&, pulsar::Producer&) (Client.cc:53)
==00:00:41:04.793 10873== by 0xCA6AED: pulsar_client_create_producer (c_Client.cc:37)
==00:00:41:04.793 10873==
==00:00:41:04.793 10873== 592 bytes in 1 blocks are possibly lost in loss record 2,590 of 2,878
==00:00:41:04.793 10873== at 0x4C2BFD9: calloc (vg_replace_malloc.c:762)
==00:00:41:04.793 10873== by 0x4012774: _dl_allocate_tls (in /usr/lib64/ld-2.17.so)
==00:00:41:04.793 10873== by 0x505588B: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
==00:00:41:04.793 10873== by 0xDC28CC: start_thread (posix_thread.ipp:50)
==00:00:41:04.793 10873== by 0xDC28CC: posix_thread<boost::asio::detail::resolver_service_base::work_io_service_runner> (posix_thread.hpp:45)
==00:00:41:04.793 10873== by 0xDC28CC: start_work_thread (resolver_service_base.ipp:122)
==00:00:41:04.793 10873== by 0xDC28CC: start_resolve_op (resolver_service_base.ipp:111)
==00:00:41:04.793 10873== by 0xDC28CC: void boost::asio::detail::resolver_service<boost::asio::ip::tcp>::async_resolve<std::_Bind<std::_Mem_fn<void (pulsar::ClientConnection::*)(boost::system::error_code const&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>)> (std::shared_ptr<pulsar::ClientConnection>, std::_Placeholder<1>, std::_Placeholder<2>)> >(std::shared_ptr<void>&, boost::asio::ip::basic_resolver_query<boost::asio::ip::tcp> const&, std::_Bind<std::_Mem_fn<void (pulsar::ClientConnection::*)(boost::system::error_code const&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>)> (std::shared_ptr<pulsar::ClientConnection>, std::_Placeholder<1>, std::_Placeholder<2>)>&) (resolver_service.hpp:83)
==00:00:41:04.793 10873== by 0xDAD709: async_resolve<std::_Bind<std::_Mem_fn<void (pulsar::ClientConnection::*)(const boost::system::error_code&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>)>(std::shared_ptr<pulsar::ClientConnection>, std::_Placeholder<1>, std::_Placeholder<2>)> > (resolver_service.hpp:117)
==00:00:41:04.793 10873== by 0xDAD709: async_resolve<std::_Bind<std::_Mem_fn<void (pulsar::ClientConnection::*)(const boost::system::error_code&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>)>(std::shared_ptr<pulsar::ClientConnection>, std::_Placeholder<1>, std::_Placeholder<2>)> > (basic_resolver.hpp:167)
==00:00:41:04.793 10873== by 0xDAD709: pulsar::ClientConnection::tcpConnectAsync() (ClientConnection.cc:470)
==00:00:41:04.793 10873== by 0xCC5D69: pulsar::ConnectionPool::getConnectionAsync(std::string const&, std::string const&) (ConnectionPool.cc:92)
==00:00:41:04.793 10873== by 0xDA61FF: pulsar::BinaryProtoLookupService::getPartitionMetadataAsync(std::shared_ptr<pulsar::TopicName> const&) (BinaryProtoLookupService.cc:72)
==00:00:41:04.793 10873== by 0xCB7C84: pulsar::ClientImpl::createProducerAsync(std::string const&, pulsar::ProducerConfiguration, std::function<void (pulsar::Result, pulsar::Producer)>) (ClientImpl.cc:158)
==00:00:41:04.793 10873== by 0xCB2074: pulsar::Client::createProducerAsync(std::string const&, pulsar::ProducerConfiguration, std::function<void (pulsar::Result, pulsar::Producer)>) (Client.cc:65)
==00:00:41:04.793 10873== by 0xCB27D9: pulsar::Client::createProducer(std::string const&, pulsar::ProducerConfiguration const&, pulsar::Producer&) (Client.cc:53)
==00:00:41:04.793 10873== by 0xCA6AED: pulsar_client_create_producer (c_Client.cc:37)
==00:00:41:04.793 10873== by 0xBAE2EA: PulsarProducer::newProducer(std::string const&) (pulsar_producer.cpp:108)
Desktop (please complete the following information):
- OS: [e.g. iOS]
Additional context Add any other context about the problem here.
Issue Analytics
- State:
- Created 2 years ago
- Comments:24 (12 by maintainers)
Top Results From Across the Web
[Pulsar Client Cpp] memory leak when receive async #8846
Describe the bug Currently when the consumer invoke receiveAsync(), there will a callback push to pendingReceives_ ...
Read more >[GitHub] [pulsar] ltamber commented on issue #8846: [Pulsar Client ...
[GitHub] [pulsar] ltamber commented on issue #8846: [Pulsar Client Cpp] memory leak when receive async · Previous message · View by thread ·...
Read more >Client Java 2.9.2 - Apache Pulsar
Client Java 2.9.2 ... Fix resources leak when create producer failed #13505; [Java] Fix semaphore and memory leak when chunks failed to enqueue...
Read more >apache/pulsar v2.8.1 on GitHub - NewReleases.io
Pending ack set managed ledger config true #11494 · Add getTxnID method in Transaction. · Fix direct memory leak related to commit and...
Read more >e7b779c1746fc2e0bf1e9284a0df8f3e871ccea2 - pulsar-client-go ...
Fixed memory leak of connection (#165) Signed-off-by: xiaolong.ran ... a pure-Go client library for Pulsar that does not depend on the C++ Pulsar...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Any progress / solution / work-around for this?
@Mayaag02 You can see the labels of https://github.com/apache/pulsar/pull/15316, which means 2.8.4, 2.9.3, 2.10.1 include this fix. The latest 3.0.0 release of the separated repo also includes it.