Pinot Broker logs to show source_ip of host/user who executed the query
See original GitHub issueCurrently in Pinot broker we see logs as below : <Sample log>
requestId=130,table<redacted>,timeMs=23,docs=72/108615312,entries=2741105/792,segments(queried/processed/matched/consuming/unavailable):256/253/8/32/0,consumingFreshnessTimeMs=1654707653215,servers=6/6,groupLimitReached=false,brokerReduceTimeMs=0,exceptions=0,serverStats=(Server=SubmitDelayMs,ResponseDelayMs,ResponseSize,DeserializationTimeMs,RequestSentDelayMs);pinot-server-2_R=0,4,1817,0,1;pinot-server-5_R=0,5,1817,1,1;pinot-server-1_R=0,20,1820,0,1;pinot-server-4_R=1,21,1820,0,1;pinot-server-0_R=1,4,1818,0,1;pinot-server-3_R=1,5,1817,0,1,offlineThreadCpuTimeNs(total/thread/sysActivity/resSer):0/0/0/0,realtimeThreadCpuTimeNs(total/thread/sysActivity/resSer):0/0/0/0,query=<redacted>
Another useful param to add here is source_ip of host/user who executed the query. This will be useful to track usage of Pinot from a particular user.
Issue Analytics
- State:
- Created a year ago
- Comments:6 (6 by maintainers)
Top GitHub Comments
@abhs50 You may take a look at
PinotClientRequest
class where theHttpRequesterIdentity
is created from the request context. This identity is passed to theBrokerRequestHandler.handleRequest()
where you can log the source infoI would like to make sure that this is a MUST have feature. We have worked hard to keep to exactly ONE log per happy-path query on the broker, and kept the log in a parseable fashion. Increasing log size is a concern both while debugging production problems and storage as well.