Some logs are missing in elk


The problem we are facing is some logs are missing in elk while few are available. We cannot find the root cause for this issue because the logs which are missing and those which are available look similar. We cannot see any errors(only warnings and those warning logs are also available) in logstash logs too. We are pretty sure that logs are not entering into elk machine itself as we queried in elasticsearch. We also tried changing the output of logstash to file and checked there, but we cannot find the missing logs there too. Logstash seems to be in running state but we cannot get all logs.

Below are the configurations we use in both production elk as well as non-production elk (prod elk and non-prod elk has set up in different machine). In prod it works fine but in non-prod we are facing this issue. In production there are only 4-5 applns running whereas in non-prod there are about 5 different environment and each environment contains 10-13 applns.

logstash version"=>"6.8.7"
elasticsearch "6.8.7",
Kibana 6.8.7


    <appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
	<destination>${LOGSTASH_DESTINATION}</destination> <!-- ip:port -->
			<mdc /> 		<!-- MDC variables on the Thread will be written as JSON fields -->
			<context /> 	<!--Outputs entries from logback's context -->
			<version /> 	<!-- Logstash json format version, the @version field in the output -->
			<logLevel />
			<loggerName />
			<threadName />
					"indexName": "xxx"   <!-- dev/qa/uat/preuat-->
			<message />
			<logstashMarkers />
			<arguments />
			<stackTrace />
<root level="INFO">
    <appender-ref ref="LOGSTASH" />

logstash conf
tcp {
port => 5000 codec => json
elasticsearch {
hosts => {"localhost:5000"} index => "yy-%{indexName}"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.