How to monitor logstash logs

[Dan]

Please suggest me how to monitor Logstash logs.

I am trying below solution but it's not working...

Generating my logstash logs in json format in logstash log folder...

By using File input to logstash ingesting this logs into elastic search to see it in kibana reports..

But it is not indexing properly...

Can you please someone suggest me better way to do this...

Thanks in Advance...

[magnusbaeck (Magnus Bäck)]

And what does your configuration look like?

Be careful when having a Logstash instance monitor its own logs, otherwise you might end up with a self-amplifying system.

[Dan]

Hi Magnus, Below is my conf file...

input{
file {

    path => "D:\ELK5.6.1\logstash-5.6.1\bin\logstash-json.log"
    sincedb_path => "D:\ELK5.6.1\logstash-5.6.1\bin\xyz.txt"
    start_position => "beginning"
    #type => "json"

codec => multiline
{
pattern => '^{'
negate => true
what => previous
}
}
}
filter {

}

output {
stdout { codec => rubydebug }
elasticsearch {
action => "index"
hosts => "127.0.0.1:9200"
index => "logstash-logs"
workers => 1
}
}

My plan is first it should ingest full log file as one message and i will parse that message..
But this is not working

What wrong i am doing here..
I have 1700 rows in log file...
In kibana it is showing 1700 records....

[magnusbaeck (Magnus Bäck)]

My plan is first it should ingest full log file as one message and i will parse that message..

That sounds like a bad idea. Why would you want to do that?

I have 1700 rows in log file...
In kibana it is showing 1700 records....

If you have one log message per row then that's the expected result.

[Dan]

My log file like below

{
"level" : "DEBUG",
"loggerName" : "logstash.runner",
"timeMillis" : 1526632605062,
"thread" : "LogStash::Runner",
"logEvent" : {
"message" : "-------- Logstash Settings (* means modified) ---------"
}
}{
"level" : "DEBUG",
"loggerName" : "logstash.runner",
"timeMillis" : 1526632605192,
"thread" : "LogStash::Runner",
"logEvent" : {
"message" : "node.name: "Admin-PC""
}
}{
"level" : "DEBUG",
"loggerName" : "logstash.runner",
"timeMillis" : 1526632605192,
"thread" : "LogStash::Runner",
"logEvent" : {
"message" : "*path.config: "jdbc_to_ES.conf""
}
}

in kibana i am getting
{ as one record
"level" : "DEBUG", as one record
and so on

With this information i cant make out error right..

[magnusbaeck (Magnus Bäck)]

That's very odd. According to the documentation Logstash's JSON log format has one logrecord per line. If that isn't the case it's a bug.

[Dan]

Thanks Magnus.....

My mistake it is generating one log record per line...

But in the log file i am seeing all "level" : "DEBUG", and "level" : "ERROR",
I want only the error one, so that I want to parse this log

[magnusbaeck (Magnus Bäck)]

Use a json filter to parse the JSON string, then wrap a drop filter in a conditional that selects the messages you don't want to keep.