How to insert Data to Elasticsearch using Logstash?

Hi, I am trying to insert data to Elasticsearch using Logstash. I created an index 'twitter' and my config file is as follows,

input{
file{
path=>"C:/Users/Sample.log"
codec=>json
}
}
output{
stdout{
codec=>rubydebug
}

elasticsearch {  
hosts => "13.10.256.10:9200"
 index => "twitter"
}

}

I could see the logs in command prompt but when I checked elasticsearch with the API 'http://13.10.256.10:9200/_cat/indices?v', it's showing docs.count as 0. What wrong am I doing? Do I need to create any other fields other than index? Please help me on this issue.

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

I moved your question to #logstash.
May be share the output of logstash and of your curl command?

The file input plugin by default tails the file, so if it was created before Logstash was started you will need to set start_position to beginning and also delete the sincedb file, as this will have recorded the current file as processed.

Hi David,

Thanks for your quick reply. Please see my logstash output and output of elasticsearch.

Logstash:

{
"CheckTS" => 1496264400085,
"path" => "C:/Users/Sample.log",
"@timestamp" => 2017-06-13T13:49:01.335Z,
"VersionTag" => 0,
"@version" => "1",
"host" => "localhost",
"location" => "abc",
"Component" => "mongodb",
"Application" => "Test"
}
{
"CheckTS" => 1496264400085,
"path" => "C:/Users/Sample.log",
"@timestamp" => 2017-06-13T13:49:01.335Z,
"VersionTag" => 0,
"@version" => "1",
"host" => "localhost",
"location" => "abc",
"Component" => "sql",
"Application" => "Test1"
}

Elasticsearch:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open twitter BcDQCs7uRWunz39AavLAVg 3 2 0 0 477b 477b

Hi Christian, I don't have problem with start_position, because I am able to see all my results in the command prompt output. The only problem is I can't see when I check it in the elasticsearch index.

I got it solved. Logstash is running from 2 different folders and that made the mess.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.