question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Pinot Broker logs to show source_ip of host/user who executed the query

See original GitHub issue

Currently in Pinot broker we see logs as below : <Sample log>

requestId=130,table<redacted>,timeMs=23,docs=72/108615312,entries=2741105/792,segments(queried/processed/matched/consuming/unavailable):256/253/8/32/0,consumingFreshnessTimeMs=1654707653215,servers=6/6,groupLimitReached=false,brokerReduceTimeMs=0,exceptions=0,serverStats=(Server=SubmitDelayMs,ResponseDelayMs,ResponseSize,DeserializationTimeMs,RequestSentDelayMs);pinot-server-2_R=0,4,1817,0,1;pinot-server-5_R=0,5,1817,1,1;pinot-server-1_R=0,20,1820,0,1;pinot-server-4_R=1,21,1820,0,1;pinot-server-0_R=1,4,1818,0,1;pinot-server-3_R=1,5,1817,0,1,offlineThreadCpuTimeNs(total/thread/sysActivity/resSer):0/0/0/0,realtimeThreadCpuTimeNs(total/thread/sysActivity/resSer):0/0/0/0,query=<redacted>

Another useful param to add here is source_ip of host/user who executed the query. This will be useful to track usage of Pinot from a particular user.

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
Jackie-Jiangcommented, Jun 13, 2022

@abhs50 You may take a look at PinotClientRequest class where the HttpRequesterIdentity is created from the request context. This identity is passed to the BrokerRequestHandler.handleRequest() where you can log the source info

0reactions
mcvsubbucommented, Jun 16, 2022

I would like to make sure that this is a MUST have feature. We have worked hard to keep to exactly ONE log per happy-path query on the broker, and kept the log in a parseable fashion. Increasing log size is a concern both while debugging production problems and storage as well.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Command-Line Interface (CLI) - Apache Pinot Docs
Pinot provides a rich CLI to perform almost every operation on the cluster. You can execute all the commands using the pinot-admin.sh ....
Read more >
Apache Pinot SQLi and RCE Cheat Sheet - Doyensec's Blog
In this post, we will show how a classic SQL-injection (SQLi) bug in a Pinot-backed API can be escalated to Remote Code Execution...
Read more >
Ingesting OSQuery Logs To Apache Pinot - Decodable
Previously, we demonstrated how to use an osquery extension to send event logs to Apache Pulsar. We then routed, cleansed, and filtered the ......
Read more >
Getting Started With Apache Pinot | StarTree
Also, it comes with a UI to query data in Pinot. Controller works closely with Zookeeper for cluster coordination. Pinot Broker. Brokers are...
Read more >
A Complete Guide to Pino Logging in Node.js - Better Stack
Due to its impressive performance and wealth of features, it is included by default in the open-source Fastify web server to handle all...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found