Logstash not indexing data

Elastic Stack Logstash
1

Trying input some log files into my elastic+logstash+kibana, but i have some problems.
I make my pattern in http://grokdebug.herokuapp.com, in the web test all works perfect, but when i run the logstash the server not work.
Below the logstash log after execute , not show anything.

My input file is in this sintaxe:

May 01 13:26:23 - denyhosts : INFO new denied hosts: ['185.103.252.233']
May 01 14:19:26 - denyfileutil: INFO purging entries older than: Sun Apr 3 14:19:26 2016
May 01 14:19:26 - denyfileutil: INFO num entries purged: 0

I want get only denyhosts tag.

This is my patterns created:

DATAFULL %{MONTH} %{MONTHDAY} %{TIME}
INFO ( - denyhosts : INFO )
DENY (new denied hosts: )
GETIP ([(?[^]]*)])

This is my logstash config to execute

input {
file {
path => "/home/elk/denyhosts.txt"
#type => "denyhosts"
start_position => "beginning"
}
}
filter{
grok {
#match => { "message" => "%{DATAFULL}%{INFO}%{DENY}%{GETIP}"}
tag_on_failure => [ "failedPattern" ]
patterns_dir => "/home/elk/dennyHosts"
}
geoip {
source => "IPBLOCK"
}
}
output {
elasticsearch {}
stdout { codec => rubydebug }
}

Using:

  • Linux Ubuntu 14
  • JDK 1.8
  • elasticsearch-2.3.2
  • kibana-4.5.0-linux-x64
  • logstash-2.3.2

Some can help-me ?
Thanks

2

What does this mean exactly?

Also, why not just use existing patterns rather than creating your own?

3

Hi warkolm.

when i run the logstash the server not work.

After execute logstash nothing is displayed in kibana or in logstash verbose log.
About my pattern, was created with existing demos, only adpted for my demand

4

Can you expand those patterns out into the grok, to see if you can make any progress there?

You might also try a pattern at a time. So get the DATAFULL piece working (%{DATAFULL}.*). And then work through each pattern.

5

Hi Jclose.

I will be test your solution..
One question:
For example i run now the logstashDemo.conf and generated de indexes xxx in elastic, if i run again the same logstashDemo.conf, elastic will indentify that have the indexes and not will indexes again? Or a new indexes will generated?