Log messages not pushed to Logstash
See original GitHub issueWe using logstash-logback-encoder version 4.11 with Spring Boot 1.5.6-RELEASE, and jHipster, deployed in Cloud Foundry.
We have configured the logger programmatically. While the logs are seen on the console we notice that many are simply not being TCP-ed to the Logstash server. Moreover, there is no error or message indicating that there have been drops although the TCP connection itself looks perfectly healthy. In fact, we have a heartbeat, which is working just fine.
Logger configuration:
//Additional constants...
private static final String ASYNC_APPENDER_NAME = "ASYNC";
private static final String CONSOLE_APPENDER_NAME = "CONSOLE";
private static final Integer ASYNC_QUEUE_SIZE = 1000;
private static final Integer ASYNC_FLUSH_SIZE = 2000;
private static final Integer MAX_HISTORY = 30;
private static final Integer DISCARDING_THRESHOLD = 0;
public static void configure() {
LoggerContext loggerContext = loggerContext();
StatusManager statusManager = loggerContext.getStatusManager();
if (statusManager != null) {
statusManager.add(new InfoStatus("Setting Logger configuration from " + LoggerConfiguration.class.getName(), loggerContext));
statusManager.add(new OnErrorConsoleStatusListener());
}
loggerContext.addListener(levelChangePropogator());
setCustomLogLevels(loggerContext);
StatusPrinter.print(statusManager);
StatusPrinter.print(loggerContext);
StatusPrinter.printInCaseOfErrorsOrWarnings(loggerContext);
}
@Bean
public static LoggerContext loggerContext() {
return (LoggerContext) LoggerFactory.getILoggerFactory();
}
@Bean
public static Logger rootLogger(LoggerContext loggerContext) {
Logger rootLogger = loggerContext.getLogger(Logger.ROOT_LOGGER_NAME);
rootLogger.setLevel(Level.INFO);
rootLogger.addAppender(consoleAppender(loggerContext, rootLogger));
rootLogger.addAppender(asyncAppender(loggerContext, tcpSocketAppender(loggerContext, rootLogger)));
rootLogger.addAppender(rollingFileAppender(loggerContext));
return rootLogger;
}
@Bean
protected static LevelChangePropagator levelChangePropogator() {
LevelChangePropagator levelChangePropagator = new LevelChangePropagator();
levelChangePropagator.setResetJUL(true);
return levelChangePropagator;
}
@Bean
@Primary
protected static LogstashTcpSocketAppender tcpSocketAppender(LoggerContext loggerContext, Logger rootLogger) {
LogstashTcpSocketAppender logstashTcpSocketAppender = new LogstashTcpSocketAppender();
logstashTcpSocketAppender.setName(TCP_APPENDER_NAME);
logstashTcpSocketAppender.setContext(loggerContext);
logstashTcpSocketAppender.addDestination(TCP_DESTINATION);
logstashTcpSocketAppender.setWriteBufferSize(TCP_WRITE_BUFFER_SIZE);
logstashTcpSocketAppender.setEncoder(logstashEncoder(loggerContext, rootLogger));
logstashTcpSocketAppender.setKeepAliveDuration(Duration.buildByDays(100));
return logstashTcpSocketAppender;
}
@Bean
protected static LogstashEncoder logstashEncoder(LoggerContext loggerContext, Logger rootLogger) {
LogstashEncoder encoder = new LogstashEncoder();
encoder.setContext(loggerContext);
encoder.setIncludeContext(true);
encoder.setCustomFields(setEnvironmentAttributes());
encoder.start();
return encoder;
}
@Bean
protected static ConsoleAppender<ILoggingEvent> consoleAppender(LoggerContext loggerContext, Logger rootLogger) {
ConsoleAppender<ILoggingEvent> consoleAppender = new ConsoleAppender<>();
consoleAppender.setContext(loggerContext);
consoleAppender.setName(CONSOLE_APPENDER_NAME);
consoleAppender.setEncoder(patternLayoutEncoder(loggerContext));
consoleAppender.start();
return consoleAppender;
}
The logstash input configuration is below:
tcp {
port => 9202
type => "some_type"
tags => [ "some_tag" ]
codec => json_lines
}
Any thoughts why this could be occurring? Importantly, how best to debug such an issue? Relevant code snippets are included - please let me know if additional information is required. Any inputs will be of great help.
Thanks!
Issue Analytics
- State:
- Created 6 years ago
- Comments:8
Top GitHub Comments
Hi @philsttr thank you for your inputs and valuable suggestions.
TCP_WRITE_BUFFER_SIZE
: Tried different values here, including letting the default be. This does not seem to matter at this point.start()
is being called explicitlysetCustomLevels()
is explicitly setting log levels for libraries, and also the root package for the service in questionStarting the nc command has helped narrow down the problem in the sense that logs are “selectively” being TCP-ed. Specifically, all the library logs (spring, etc.) are going through fine but the application-level are not.
Closing due to inactivity