# Logstash спорадически перестаёт отправлять логи в elastic

**URL:** <https://discuss.elastic.co/t/logstash-elastic/204231>\
**Category:** Вопросы на русском языке\
**Created:** [October 18, 2019, 12:55pm UTC](https://discuss.elastic.co/t/logstash-elastic/204231 "2019-10-18T12:55:43Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![111167](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111167/32/48997_2.png) [@111167](https://discuss.elastic.co/u/111167)\
**Post date:** [October 18, 2019, 12:55pm UTC](https://discuss.elastic.co/t/logstash-elastic/204231/1 "2019-10-18T12:55:43Z")

</div>

Коллеги, доброго дня. Конфигурация: elasticsearch: 6.4.2, logstash: 6.6.1

Cluster health:  
{  
"cluster\_name" : "elastic-kibana",  
"status" : "yellow",  
"timed\_out" : false,  
"number\_of\_nodes" : 1,  
"number\_of\_data\_nodes" : 1,  
"active\_primary\_shards" : 606,  
"active\_shards" : 606,  
"relocating\_shards" : 0,  
"initializing\_shards" : 0,  
"unassigned\_shards" : 605,  
"delayed\_unassigned\_shards" : 0,  
"number\_of\_pending\_tasks" : 0,  
"number\_of\_in\_flight\_fetch" : 0,  
"task\_max\_waiting\_in\_queue\_millis" : 0,  
"active\_shards\_percent\_as\_number" : 50.0412881915772  
}

Конфигурация логстеша:

```
input {
  file {
    start_position => "beginning"
    path => "/logs/log.json"
    type => json
  }
}
filter {
mutate {
rename => ["message","log_message"]
}
grok {
match => { log_message => "{\"timestamp\":\"%{TIMESTAMP_ISO8601:date}\",\"severity\":\"%{WORD:severity}\",\"order_id\":%{GREEDYDATA:order_id},\"data\":%{GREEDY
}
mutate {
remove_field => ["log_message"]
gsub => ["client", "\"", ""]
gsub => ["friendly_order_id", "\"", ""]
gsub => ["order_id", "\"", ""]
add_field => { "newtimestamp" => "%{date}"}
}
}

output {
        elasticsearch {
                index => "cluster-blue-%{+YYYY.MM.dd}"
                hosts => ["172.17.0.1:9200"]
        }

}

```

Проблема: периодически перестают парситься логи с логстеша в эластик, лечится рестартом логстеша. Один из моментов остановки наступает после выполнения сервисных задач в эластике, но не всегда, проблема носит спорадический характер:

```
[2019-10-18T01:55:00,000][INFO][o.e.x.m.MlDailyMaintenanceService] triggering scheduled [ML] maintenance tasks
[2019-10-18T01:55:00,000][INFO][o.e.x.m.a.TransportDeleteExpiredDataAction] [Beoh7RO] Deleting expired data
[2019-10-18T01:55:00,005][INFO][o.e.x.m.a.TransportDeleteExpiredDataAction] [Beoh7RO] Completed deletion of expired data
[2019-10-18T01:55:00,005][INFO][o.e.x.m.MlDailyMaintenanceService] Successfully completed [ML] maintenance tasks

```

На момент прекращения записи в эластик сам лог логстеша без ошибок:

```
[2019-10-17T22:00:28,605][INFO][logstash.pipeline] Starting pipeline {:pipeline_id=>".monitoring-logstash", "pipeline.workers"=>1, "pipeline.batch.size"=>2, "pipeline.batch.delay"=>50}
[2019-10-17T22:00:28,657][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
[2019-10-17T22:00:28,662][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://elasticsearch:9200/"}
[2019-10-17T22:00:28,666][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2019-10-17T22:00:28,667][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2019-10-17T22:00:28,705][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://elasticsearch:9200"]}
[2019-10-17T22:00:28,737][INFO][logstash.pipeline] Pipeline started successfully {:pipeline_id=>".monitoring-logstash", :thread=>"#<Thread:0x626548eb run>"}
[2019-10-17T22:00:28,742][INFO][logstash.agent] Pipelines running {:count=>2, :running_pipelines=>[:main, :".monitoring-logstash"], :non_running_pipelines=>[]}
[2019-10-17T22:00:29,073][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}

```

Подскажите, в чём может быть проблема ?

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [October 18, 2019, 4:49pm UTC](https://discuss.elastic.co/t/logstash-elastic/204231/2 "2019-10-18T16:49:10Z")

</div>

Во сколько прекратилась запись? В полночь?

---

<div class="post-metadata">

**Author:** ![111167](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111167/32/48997_2.png) [@111167](https://discuss.elastic.co/u/111167)\
**Post date:** [October 21, 2019, 8:42am UTC](https://discuss.elastic.co/t/logstash-elastic/204231/3 "2019-10-21T08:42:01Z")

</div>

После 2 часов ночи, после выполнения сервисных задач

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 18, 2019, 8:42am UTC](https://discuss.elastic.co/t/logstash-elastic/204231/4 "2019-11-18T08:42:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.