Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Cache check_ is_ active_ rm

See original GitHub issue

rm = ResourceManager() Spend a lot of time on check_ is_ active_ rm function , could i cache it ?

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:10 (5 by maintainers)

github_iconTop GitHub Comments

dimon222commented, Jun 22, 2022

@lxorc thanks for this information. Indeed, it sounds plausible to achieve it when cluster page takes long to load. I still have to review available endpoints, but if there’s one that can play the role of active mode health check, it’s good idea to consider replacement. That is also in top of above suggestions (optimization-wise)

dimon222commented, Jun 21, 2022

I had some thinking about this question myself few years ago, but here’s why I still haven’t done so:

  • Hadoop is considered cluster nature environment. That means that to achieve HA on clientside when running against direct namenodes you need to hit multiple and be able to fallback easily in chances when one or another gets down. If you cache the result, then you risk getting stuck on same broken node for n-time. We could consider short living cache for few minutes, but it might still breach the concept of HA.

Now, enterprise distributions of Hadoop usually include Knox gateway that typically deals with HA concept on its own. If you have single direct Knox url you, however, don’t need to check if cluster is active at all, because it always considered so.

So in my opinion what we could consider:

  1. Knox mode - don’t check active RM. Its always same single URL and it should be alive by design
  2. Non-knox mode - as is what we have today - default
Read more comments on GitHub >

github_iconTop Results From Across the Web

Apache Hadoop 2.7.3 – ResourceManager High Availability
Admin commands. yarn rmadmin has a few HA-specific command options to check the health/state of an RM, and transition to Active/Standby.
Read more >
Solved: Resource Manager HA and Yarn service-check is fail...
Solved: We enabled Yarn Resource Manager HA on our cluster ( HDP 2.3.2 ... File "/var/lib/ambari-agent/cache/common-services/YARN/ ...
Read more >
13.2.28. Managing the SSSD Cache Red Hat Enterprise Linux 6
Deleting Domain Cache Files · Deleting the cache file deletes all user data, both identification and cached credentials. · If the configuration is...
Read more >
How To Clear The SSSD Cache In Linux - RootUsers
The System Security Services Daemon (SSSD) provides access to identity and authentication providers. Basically rather than relying on ...
Read more >
QRadar: How to clear the Tomcat cache - IBM
Before you attempt to clear the Tomcat cache check if the issue ... Restarting Tomcat ends all active user sessions and logs out...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found