# How to monitor logstash logs **URL:** \ **Category:** Logstash\ **Created:** [May 24, 2018, 6:03am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-logs/133077 "2018-05-24T06:03:07Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Dan](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@Dan](https://discuss.elastic.co/u/Dan)\ **Post date:** [May 24, 2018, 6:03am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-logs/133077/1 "2018-05-24T06:03:07Z")
Please suggest me how to monitor Logstash logs. I am trying below solution but it's not working... Generating my logstash logs in json format in logstash log folder... By using File input to logstash ingesting this logs into elastic search to see it in kibana reports.. But it is not indexing properly... Can you please someone suggest me better way to do this... Thanks in Advance... ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 24, 2018, 6:19am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-logs/133077/2 "2018-05-24T06:19:21Z")
And what does your configuration look like? Be careful when having a Logstash instance monitor its own logs, otherwise you might end up with a self-amplifying system. ---
**Author:** ![Dan](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@Dan](https://discuss.elastic.co/u/Dan)\ **Post date:** [May 24, 2018, 6:57am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-logs/133077/3 "2018-05-24T06:57:53Z")
Hi Magnus, Below is my conf file... > input{ > file { > > ``` > path => "D:\ELK5.6.1\logstash-5.6.1\bin\logstash-json.log" > sincedb_path => "D:\ELK5.6.1\logstash-5.6.1\bin\xyz.txt" > start_position => "beginning" > #type => "json" > > ``` > > codec =\> multiline > { > pattern =\> '^{' > negate =\> true > what =\> previous > } > } > } > filter { > > } > > output { > stdout { codec =\> rubydebug } > elasticsearch { > action =\> "index" > hosts =\> "127.0.0.1:9200" > index =\> "logstash-logs" > workers =\> 1 > } > } My plan is first it should ingest full log file as one message and i will parse that message.. But this is not working What wrong i am doing here.. I have 1700 rows in log file... In kibana it is showing 1700 records.... ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 24, 2018, 8:22am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-logs/133077/4 "2018-05-24T08:22:51Z")
> My plan is first it should ingest full log file as one message and i will parse that message.. That sounds like a bad idea. Why would you want to do that? > I have 1700 rows in log file... > In kibana it is showing 1700 records.... If you have one log message per row then that's the expected result. ---
**Author:** ![Dan](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@Dan](https://discuss.elastic.co/u/Dan)\ **Post date:** [May 24, 2018, 8:42am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-logs/133077/5 "2018-05-24T08:42:29Z")
My log file like below > { > "level" : "DEBUG", > "loggerName" : "logstash.runner", > "timeMillis" : 1526632605062, > "thread" : "LogStash::Runner", > "logEvent" : { > "message" : "-------- Logstash Settings (\* means modified) ---------" > } > }{ > "level" : "DEBUG", > "loggerName" : "logstash.runner", > "timeMillis" : 1526632605192, > "thread" : "LogStash::Runner", > "logEvent" : { > "message" : "node.name: "Admin-PC"" > } > }{ > "level" : "DEBUG", > "loggerName" : "logstash.runner", > "timeMillis" : 1526632605192, > "thread" : "LogStash::Runner", > "logEvent" : { > "message" : "\*path.config: "jdbc\_to\_ES.conf"" > } > } in kibana i am getting **{** as one record **"level" : "DEBUG",** as one record and so on With this information i cant make out error right.. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 24, 2018, 9:27am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-logs/133077/6 "2018-05-24T09:27:50Z")
That's very odd. According to the documentation Logstash's JSON log format has one logrecord per line. If that isn't the case it's a bug. ---
**Author:** ![Dan](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@Dan](https://discuss.elastic.co/u/Dan)\ **Post date:** [May 25, 2018, 5:51am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-logs/133077/7 "2018-05-25T05:51:04Z")
Thanks Magnus..... My mistake it is generating one log record per line... But in the log file i am seeing all **"level" : "DEBUG",** and **"level" : "ERROR",** I want only the error one, so that I want to parse this log ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 25, 2018, 6:16am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-logs/133077/8 "2018-05-25T06:16:33Z")
Use a json filter to parse the JSON string, then wrap a drop filter in a conditional that selects the messages you don't want to keep. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 22, 2018, 6:16am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-logs/133077/9 "2018-06-22T06:16:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.