question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Pulsar CPP client mem leak

See original GitHub issue

Describe the bug

pulsar host can’t connect successfuly, send pulsar will mem leak

To Reproduce Steps to reproduce the behavior:

  1. Config pulsar host can’t connect successfully
  2. Send a msg
  3. ClientConnection will mem leak

Expected behavior If Pulsar fails to connect, each message will leak memory

Code:

    m_conf = pulsar_client_configuration_create();
    pulsar_client_configuration_set_logger(m_conf, PulsarProducer::pulsar_logger, (void*)this);
    m_client = pulsar_client_create(service_url.c_str(), m_conf);


      pulsar_result err = pulsar_client_create_producer(m_client, topic.c_str(), producer_conf, &producer);
      if (err != pulsar_result_Ok) {
          LOG_ERROR("Failed to create producer: %s\n", pulsar_result_str(err));
          pulsar_producer_configuration_free(producer_conf);
          return NULL;
      }

    

Screenshots valgrind

==00:00:41:04.793 10873== 592 bytes in 1 blocks are possibly lost in loss record 2,589 of 2,878
==00:00:41:04.793 10873==    at 0x4C2BFD9: calloc (vg_replace_malloc.c:762)
==00:00:41:04.793 10873==    by 0x4012774: _dl_allocate_tls (in /usr/lib64/ld-2.17.so)
==00:00:41:04.793 10873==    by 0x505588B: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
==00:00:41:04.793 10873==    by 0x13AE17E: __gthread_create (gthr-default.h:662)
==00:00:41:04.793 10873==    by 0x13AE17E: std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>) (thread.cc:142)
==00:00:41:04.793 10873==    by 0xCDB4B1: thread<std::_Bind<std::_Mem_fn<void (pulsar::ExecutorService::*)(std::shared_ptr<boost::asio::io_service>)>(pulsar::ExecutorService*, std::shared_ptr<boost::asio::io_service>)> > (thread:135)
==00:00:41:04.793 10873==    by 0xCDB4B1: pulsar::ExecutorService::ExecutorService() (ExecutorService.cc:30)
==00:00:41:04.793 10873==    by 0xCDB716: construct<pulsar::ExecutorService> (shared_ptr_base.h:396)
==00:00:41:04.793 10873==    by 0xCDB716: _S_construct<pulsar::ExecutorService> (alloc_traits.h:254)
==00:00:41:04.793 10873==    by 0xCDB716: construct<pulsar::ExecutorService> (alloc_traits.h:393)
==00:00:41:04.793 10873==    by 0xCDB716: _Sp_counted_ptr_inplace<> (shared_ptr_base.h:399)
==00:00:41:04.793 10873==    by 0xCDB716: construct<std::_Sp_counted_ptr_inplace<pulsar::ExecutorService, std::allocator<pulsar::ExecutorService>, (__gnu_cxx::_Lock_policy)2u>, const std::allocator<pulsar::ExecutorService> > (new_allocator.h:120)
==00:00:41:04.793 10873==    by 0xCDB716: _S_construct<std::_Sp_counted_ptr_inplace<pulsar::ExecutorService, std::allocator<pulsar::ExecutorService>, (__gnu_cxx::_Lock_policy)2u>, const std::allocator<pulsar::ExecutorService> > (alloc_traits.h:254)
==00:00:41:04.793 10873==    by 0xCDB716: construct<std::_Sp_counted_ptr_inplace<pulsar::ExecutorService, std::allocator<pulsar::ExecutorService>, (__gnu_cxx::_Lock_policy)2u>, const std::allocator<pulsar::ExecutorService> > (alloc_traits.h:393)
==00:00:41:04.793 10873==    by 0xCDB716: __shared_count<pulsar::ExecutorService, std::allocator<pulsar::ExecutorService> > (shared_ptr_base.h:502)
==00:00:41:04.793 10873==    by 0xCDB716: __shared_ptr<std::allocator<pulsar::ExecutorService> > (shared_ptr_base.h:957)
==00:00:41:04.793 10873==    by 0xCDB716: shared_ptr<std::allocator<pulsar::ExecutorService> > (shared_ptr.h:316)
==00:00:41:04.793 10873==    by 0xCDB716: allocate_shared<pulsar::ExecutorService, std::allocator<pulsar::ExecutorService> > (shared_ptr.h:598)
==00:00:41:04.793 10873==    by 0xCDB716: make_shared<pulsar::ExecutorService> (shared_ptr.h:614)
==00:00:41:04.793 10873==    by 0xCDB716: pulsar::ExecutorServiceProvider::get() (ExecutorService.cc:91)
==00:00:41:04.793 10873==    by 0xCC5BE9: pulsar::ConnectionPool::getConnectionAsync(std::string const&, std::string const&) (ConnectionPool.cc:83)
==00:00:41:04.793 10873==    by 0xDA61FF: pulsar::BinaryProtoLookupService::getPartitionMetadataAsync(std::shared_ptr<pulsar::TopicName> const&) (BinaryProtoLookupService.cc:72)
==00:00:41:04.793 10873==    by 0xCB7C84: pulsar::ClientImpl::createProducerAsync(std::string const&, pulsar::ProducerConfiguration, std::function<void (pulsar::Result, pulsar::Producer)>) (ClientImpl.cc:158)
==00:00:41:04.793 10873==    by 0xCB2074: pulsar::Client::createProducerAsync(std::string const&, pulsar::ProducerConfiguration, std::function<void (pulsar::Result, pulsar::Producer)>) (Client.cc:65)
==00:00:41:04.793 10873==    by 0xCB27D9: pulsar::Client::createProducer(std::string const&, pulsar::ProducerConfiguration const&, pulsar::Producer&) (Client.cc:53)
==00:00:41:04.793 10873==    by 0xCA6AED: pulsar_client_create_producer (c_Client.cc:37)
==00:00:41:04.793 10873==
==00:00:41:04.793 10873== 592 bytes in 1 blocks are possibly lost in loss record 2,590 of 2,878
==00:00:41:04.793 10873==    at 0x4C2BFD9: calloc (vg_replace_malloc.c:762)
==00:00:41:04.793 10873==    by 0x4012774: _dl_allocate_tls (in /usr/lib64/ld-2.17.so)
==00:00:41:04.793 10873==    by 0x505588B: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
==00:00:41:04.793 10873==    by 0xDC28CC: start_thread (posix_thread.ipp:50)
==00:00:41:04.793 10873==    by 0xDC28CC: posix_thread<boost::asio::detail::resolver_service_base::work_io_service_runner> (posix_thread.hpp:45)
==00:00:41:04.793 10873==    by 0xDC28CC: start_work_thread (resolver_service_base.ipp:122)
==00:00:41:04.793 10873==    by 0xDC28CC: start_resolve_op (resolver_service_base.ipp:111)
==00:00:41:04.793 10873==    by 0xDC28CC: void boost::asio::detail::resolver_service<boost::asio::ip::tcp>::async_resolve<std::_Bind<std::_Mem_fn<void (pulsar::ClientConnection::*)(boost::system::error_code const&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>)> (std::shared_ptr<pulsar::ClientConnection>, std::_Placeholder<1>, std::_Placeholder<2>)> >(std::shared_ptr<void>&, boost::asio::ip::basic_resolver_query<boost::asio::ip::tcp> const&, std::_Bind<std::_Mem_fn<void (pulsar::ClientConnection::*)(boost::system::error_code const&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>)> (std::shared_ptr<pulsar::ClientConnection>, std::_Placeholder<1>, std::_Placeholder<2>)>&) (resolver_service.hpp:83)
==00:00:41:04.793 10873==    by 0xDAD709: async_resolve<std::_Bind<std::_Mem_fn<void (pulsar::ClientConnection::*)(const boost::system::error_code&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>)>(std::shared_ptr<pulsar::ClientConnection>, std::_Placeholder<1>, std::_Placeholder<2>)> > (resolver_service.hpp:117)
==00:00:41:04.793 10873==    by 0xDAD709: async_resolve<std::_Bind<std::_Mem_fn<void (pulsar::ClientConnection::*)(const boost::system::error_code&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>)>(std::shared_ptr<pulsar::ClientConnection>, std::_Placeholder<1>, std::_Placeholder<2>)> > (basic_resolver.hpp:167)
==00:00:41:04.793 10873==    by 0xDAD709: pulsar::ClientConnection::tcpConnectAsync() (ClientConnection.cc:470)
==00:00:41:04.793 10873==    by 0xCC5D69: pulsar::ConnectionPool::getConnectionAsync(std::string const&, std::string const&) (ConnectionPool.cc:92)
==00:00:41:04.793 10873==    by 0xDA61FF: pulsar::BinaryProtoLookupService::getPartitionMetadataAsync(std::shared_ptr<pulsar::TopicName> const&) (BinaryProtoLookupService.cc:72)
==00:00:41:04.793 10873==    by 0xCB7C84: pulsar::ClientImpl::createProducerAsync(std::string const&, pulsar::ProducerConfiguration, std::function<void (pulsar::Result, pulsar::Producer)>) (ClientImpl.cc:158)
==00:00:41:04.793 10873==    by 0xCB2074: pulsar::Client::createProducerAsync(std::string const&, pulsar::ProducerConfiguration, std::function<void (pulsar::Result, pulsar::Producer)>) (Client.cc:65)
==00:00:41:04.793 10873==    by 0xCB27D9: pulsar::Client::createProducer(std::string const&, pulsar::ProducerConfiguration const&, pulsar::Producer&) (Client.cc:53)
==00:00:41:04.793 10873==    by 0xCA6AED: pulsar_client_create_producer (c_Client.cc:37)
==00:00:41:04.793 10873==    by 0xBAE2EA: PulsarProducer::newProducer(std::string const&) (pulsar_producer.cpp:108)

Desktop (please complete the following information):

  • OS: [e.g. iOS]

Additional context Add any other context about the problem here.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:24 (12 by maintainers)

github_iconTop GitHub Comments

1reaction
batilakcommented, Mar 21, 2022

Any progress / solution / work-around for this?

0reactions
BewareMyPowercommented, Nov 19, 2022

@Mayaag02 You can see the labels of https://github.com/apache/pulsar/pull/15316, which means 2.8.4, 2.9.3, 2.10.1 include this fix. The latest 3.0.0 release of the separated repo also includes it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

[Pulsar Client Cpp] memory leak when receive async #8846
Describe the bug Currently when the consumer invoke receiveAsync(), there will a callback push to pendingReceives_ ...
Read more >
[GitHub] [pulsar] ltamber commented on issue #8846: [Pulsar Client ...
[GitHub] [pulsar] ltamber commented on issue #8846: [Pulsar Client Cpp] memory leak when receive async · Previous message · View by thread ·...
Read more >
Client Java 2.9.2 - Apache Pulsar
Client Java 2.9.2 ... Fix resources leak when create producer failed #13505; [Java] Fix semaphore and memory leak when chunks failed to enqueue...
Read more >
apache/pulsar v2.8.1 on GitHub - NewReleases.io
Pending ack set managed ledger config true #11494 · Add getTxnID method in Transaction. · Fix direct memory leak related to commit and...
Read more >
e7b779c1746fc2e0bf1e9284a0df8f3e871ccea2 - pulsar-client-go ...
Fixed memory leak of connection (#165) Signed-off-by: xiaolong.ran ... a pure-Go client library for Pulsar that does not depend on the C++ Pulsar...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found