# Logstash not indexed to Elasticsearch

**URL:** <https://discuss.elastic.co/t/logstash-not-indexed-to-elasticsearch/103909>\
**Category:** Logstash\
**Created:** [October 13, 2017, 1:53pm UTC](https://discuss.elastic.co/t/logstash-not-indexed-to-elasticsearch/103909 "2017-10-13T13:53:18Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![johntadd](https://avatars.discourse-cdn.com/v4/letter/j/ecae2f/32.png) [@johntadd](https://discuss.elastic.co/u/johntadd)\
**Post date:** [October 13, 2017, 1:53pm UTC](https://discuss.elastic.co/t/logstash-not-indexed-to-elasticsearch/103909/1 "2017-10-13T13:53:18Z")

</div>

Searching the forum, but I don't find any threads that actually fixes my problem. So posting the question here.

The main problem is I don't find data processed by logstash gets indexed to elasticsearch.

```
curl 'localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open .kibana hIm_g8DDQPu9HC3aOAiZKQ 1 1 1 0 3.2kb 3.2kb

```

My setup flow is logstash read local gz files, indexing those data to elasticsearch. Config contents include:

```
input {
  file {
    type => "gzip"
    path => "/path/to/log/*.log.gz"
    start_position => beginning
    sincedb_path => "/dev/null"
    codec => "gzip_lines"
    max_open_files => 1000000
  }
}
output {
  elasticsearch {
    hosts => ["localhost:9200"]
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

```

I am sure elasticsearch is up because it responses message when posting with curl command `curl localhost:9200`

```
{
  "name" : "-6IWhCo",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "J6OXwQmcSUS6R9q0nRmEvg",
  "version" : {
    "number" : "5.5.0",
    "build_hash" : "260387d",
    "build_date" : "2017-06-30T23:16:05.735Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  }
  "tagline" : "You Know, for Search"
}

```

Also my ELK stack runs on the same EC2 server.

In addition, logstash does process local gz files but sometimes it complains `observe_read_file: general error reading /path/to/....gz - error: java.lang.IllegalArgumentException: Object ¿´óxY^T<U+0A8D>.Áæ^^Àf{ú@^GESCi·Mé<U+0094># n`, though more often it just prints `Received line {:path =>"/path/to/....gz", :text=>"\xBF\xB4\xF3LY\x14\x8D...."}` when debug mode is enabled (e.g. by adding --debug flag in /etc/systemd/system/logstash.service file)

There is no firewall rules that might block routing message between logstash and easticsearch, and I am able to `telnet localhost 9200` as well

```
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

```

The only warning I found it might be weird is `[logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<Java::JavaNet::URI:0x3aa8a5e>}` but that seems to be normal as I notice other people's logstash log file (when googling or searching in the forum) also contains that line. And elasticsearch log also prints `][o.e.c.r.a.AllocationService] [-6IWhCo] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.kibana][0]] ...]).` that looks up and running.

What other places I can check that might prevent elasticsearch from indexing data?

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 10, 2017, 1:53pm UTC](https://discuss.elastic.co/t/logstash-not-indexed-to-elasticsearch/103909/2 "2017-11-10T13:53:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.