Not able to create index with logstash.conf

Elastic Stack Logstash
1

Doing a poc need some help asap.

I have logstash.conf file with following stuff.

input {
file {
path => "D:/elasticsearch/logstash-tutorial.log/nginxAccess.log"
start_position => "beginning"
type => "logs"
}
}
filter {
grok{
match=>{
"message"=>"%{IP:clientip} - - [%{NOTSPACE:date} -%{INT}] "%{WORD:action} /%{WORD}/%{WORD}/%{NOTSPACE:login} %{WORD:protocol}/%{NUMBER:protocolNum}" %{NUMBER:status} %{NUMBER} "%{NOTSPACE}" "%{NOTSPACE:client} (%{WORD}; %{WORD:clientOs}%{GREEDYDATA}"
}
add_field=>{
"eventName"=>"groke"
}
}
geoip {
source => "clientip"
}
}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
index =>[ "test-index" ]
}
}

when I am running this file through cmd using this command ..../bin>logstash -f logstash.conf

It gets start with the message
Settings: Default pipeline workers: 8
Pipeline main started

But i dont see any index with this name test-index is created in http://localhost:9200/_cat/indices?v

Please help me with this as this is urgent.

2

This is an extremely common difficulty that people have and the question is asked at least a few times per week. Please consult the archives of this forum for clues. Good keywords include sincedb, sincedb_path, and ignore_older.

3

Hi,

Thanks for the reply but could not get anything from the existing questions.
The problem is I have one other file as well from where i am able to create the index in Elastic Search ::

input {
twitter {
consumer_key => "GgxvFNgtahQwrbnN4hE5bo8wr"
consumer_secret => "d10jYAbsym6T4b3qi0brUDFRyKq6ccZbOuUlFTqEaFoGZEMnqv"
keywords => ["#IndiasBiggestSale"]
oauth_token => "111380564-GSbQ1vin3w1MZtJsefdT1NXr7G9nzCQlHeCntURA"
oauth_token_secret => "i4a4gnB5lRxXPRPivq0pxiY0ApibPv0olJwDFsPOUKRDJ"
}
beats {
port => "5043"
}
}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
index => [ "twitter-index" ]
}
file {
path => "D:/elasticsearch/logstash-tutorial.log/twitter.log"
}
}

Here everything is working fine index created, twitter.log file is being populated.
But in other file I have used filter with regex, tested this file as well using command
..bin>logstash logstash.conf --configtest

It outputs Configuration OK. But the index is not created.