Creates multiple messages

[Honda_fred_elk (Fredrick Yessaian)]

Hi,

I'm new to logstash exploring EKL As a starting.. I have simple config setup for logstash.

input {
file {
path => "/appl/log/TestLogs/*.log"
start_position => "beginning"
}
}

output {
elasticsearch { hosts => ["localhost:9200"] }
stdout{}
file{
path => "/appl/log/TestLogs/LogStash_output.log"
}
}

but when I start logstash.. it reads .log file and creates nested messages like below

{"message":"{"message":"2016-07-18 13:01:55,996 DEBUG org.springframework
{"message":"{"message":"2016-07-18 13:01:55,997 DEBUG org.springframework
{"message":"{"message":"2016-07-18 13:01:55,999 DEBUG org.springframework
{"message":"{"message":"2016-07-18 13:01:56,000 DEBUG org.springframework
{"message":"{"message":"{\"message\":\"2016-07-18 13:01:55,734 INFO
{"message":"{"message":"{\"message\":\"2016-07-18 13:01:55,761 DEBUG
{"message":"{"message":"{\"message\":\"2016-07-18 13:01:55,794 DEBUG
{"message":"{"message":"{\"message\":\"2016-07-18 13:01:55,796 DEBUG
{"message":"{"message":"{\"message\":\"2016-07-18 13:01:55,797 DEBUG
{"message":"{"message":"{\"message\":\"2016-07-18 13:01:55,831 DEBUG

It worked a day before.. but all of sudden.. the output message created like this
What could be the reason.?

Thanks in advance
Fredrick

[warkolm (Mark Walkom)]

They all have different timestamps from what I can see, so LS considers them different events.

[Honda_fred_elk (Fredrick Yessaian)]

Helo Mark, thanks for your response..

I should have given proper output.. check below message.. this is how I get.. just copy and paste it in a notepad

{"message":"2016-07-18 13:01:55,734 INFO org.springframework.beans.factory.xml.XmlBeanDefinitionReader:
{"message":"2016-07-18 13:01:55,761 DEBUG org.springframework.beans.factory.xml.DefaultDocumentLoader:
{"message":"2016-07-18 13:01:55,794 DEBUG org.springframework.beans.factory.xml.PluggableSchemaResolver:
{"message":"2016-07-18 13:01:55,796 DEBUG org.springframework.beans.factory.xml.PluggableSchemaResolver:
{"message":"2016-07-18 13:01:55,797 DEBUG org.springframework.beans.factory.xml.PluggableSchemaResolver:

{"message":"{"message":"2016-07-18 13:01:55,734 INFO org.springframework.beans.factory.xml.XmlBeanDefinitionReader:
{"message":"{"message":"2016-07-18 13:01:55,761 DEBUG org.springframework.beans.factory.xml.DefaultDocumentLoader:
{"message":"{"message":"2016-07-18 13:01:55,794 DEBUG org.springframework.beans.factory.xml.PluggableSchemaResolver:
{"message":"{"message":"2016-07-18 13:01:55,796 DEBUG org.springframework.beans.factory.xml.PluggableSchemaResolver:
{"message":"{"message":"2016-07-18 13:01:55,797 DEBUG org.springframework.beans.factory.xml.PluggableSchemaResolver:

{"message":"{"message":"{\"message\":\"2016-07-18 13:01:55,734 INFO org.springframework.beans.factory.xml.XmlBeanDefinitionReader:
{"message":"{"message":"{\"message\":\"2016-07-18 13:01:55,761 DEBUG org.springframework.beans.factory.xml.DefaultDocumentLoader:
{"message":"{"message":"{\"message\":\"2016-07-18 13:01:55,794 DEBUG org.springframework.beans.factory.xml.PluggableSchemaResolver:
{"message":"{"message":"{\"message\":\"2016-07-18 13:01:55,796 DEBUG org.springframework.beans.factory.xml.PluggableSchemaResolver:
{"message":"{"message":"{\"message\":\"2016-07-18 13:01:55,797 DEBUG org.springframework.beans.factory.xml.PluggableSchemaResolver:

the same nested pattern repeats for more than 10 time.. I don't know why?..

[Honda_fred_elk (Fredrick Yessaian)]

Sorry.. Its a mistake from my side.. I have kept the input file as /*.log and output file location as same .log.. so once the output log file written.. then that was considered as input file and was processed by logstash again.. Finally it ended as OOM issue by logstash..

I changed the input and output location.. works fine..