# Seems that Logstash stop shipping logs to ES / Problem into the logstash config

**URL:** <https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946>\
**Category:** Logstash\
**Created:** [May 14, 2019, 7:52am UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946 "2019-05-14T07:52:04Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rodolphe\_Ghio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rodolphe_ghio/32/46130_2.png) [@Rodolphe\_Ghio](https://discuss.elastic.co/u/Rodolphe_Ghio)\
**Post date:** [May 14, 2019, 7:52am UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/1 "2019-05-14T07:52:04Z")

</div>

Hello everyone, I'm new in the ELK community. I'm trying to setup a kibana sandbox (linux vm). So I've put data into the logstash. I've created index / mapping thanks to the Dev tool.  
But when I'm trying to set a default index, kibana don't find any index

 ![Capture](https://us1.discourse-cdn.com/elastic/original/3X/8/c/8c623f9327245317f89077546df2457aacc54083.png)  
So here's the two index I've created. ![Capture2](https://us1.discourse-cdn.com/elastic/original/3X/a/9/a9118d3bb4a628abb954e0dc9956955153072f15.png).  
Can anyone help me ? or give me a clue ?

EDITED:  
It seems that something is misconfigured on the Logstash end  
Here's my Input file and my output

> **Input**
>
> ```
> input {
> beats {
> port => 5044
> tags => "fromBeats"
> }
> file {
> path => [
> "/home/secunix/logs/*",
> "/tech/*"
> ]
> start_position => "beginning"
> 
> sincedb_path => "/dev/null"
> }
> tcp {
> port => 5514
> type => "syslog"
> tags => "from Syslog-ng"
> } 
> }
> 
> filter {
> grok {
> match => ["message", "<(?<sys_priority>\d+?)>(?<syslog_timestamp>%{CISCOTIMESTAMP})\s(?<logsource>%{URIHOST})(\s(?:(?<application>.*?)(%(?<project>.*?))?))?:(?:\s)?(?<logmessage>.*$)"]
> }
> if [logmessage] {
> mutate {
> replace => ["message", "%{logmessage}"]
> remove_field => ["logmessage"]
> }
> }
> if [project] {
> mutate {
> replace => ["type", "%{project}"]
> remove_field => ["project"]
> }
> }else if [application] {
> mutate {
> lowercase => ["application"]
> }
> mutate {
> gsub => ["application", " ", "_"]
> }
> mutate {
> replace => ["type", "%{application}"]
> }
> }else {
> mutate {
> replace => ["type", "uknapp"]
> add_field => { "application" => "uknapp" }
> }
> }
> }
> 
> ```

> **Output**
>
> ```
> output {
> elasticsearch {
> hosts => ["localhost:9200"] //host pour push l’index
> index => "logstash-%{+YYYY.MM.dd}" //Nom de l’index à la sortie
> #+++ sa Added by scr-sop-af-config-elksandbox. Do not remove this line.
> user => "logstash" //Logs de connexion, ne pas toucher
> #--- sa Added by scr-sop-af-config-elksandbox. Do not remove this line.
> #+++ sa Added by scr-sop-af-config-elksandbox. Do not remove this line.
> password => "logstash"
> #--- sa Added by scr-sop-af-config-elksandbox. Do not remove this line.
> 
> }
> }
> 
> ```

and this is the kinf of logs i've :

```auto
{"index":{"_index":"shakespeare","_id":0}}
{"type":"act","line_id":1,"play_name":"Henry IV", "speech_number":"","line_number":"","speaker":"","text_entry":"ACT I"}

```

---

<div class="post-metadata">

**Author:** ![azasypkin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/azasypkin/32/42216_2.png) [@azasypkin](https://discuss.elastic.co/u/azasypkin)\
**Post date:** [May 14, 2019, 8:10am UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/2 "2019-05-14T08:10:58Z")

</div>

Hi @Rodolphe_Ghio,

I see the indices you're referring to don't have any docs in them, that's the reason why you don't see them in Kibana UI. That means you need to have at least on document in the index to be able to create a pattern for it in Kibana.

You can follow [https://github.com/elastic/kibana/issues/35481](https://github.com/elastic/kibana/issues/35481) where possible improvements are discussed.

--  
Oleg

---

<div class="post-metadata">

**Author:** ![Rodolphe\_Ghio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rodolphe_ghio/32/46130_2.png) [@Rodolphe\_Ghio](https://discuss.elastic.co/u/Rodolphe_Ghio)\
**Post date:** [May 14, 2019, 8:20am UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/3 "2019-05-14T08:20:56Z")

</div>

> [@azasypkin](#):
>
> have any doc

Thanks for yyour answer but how can I add or link data to those index.  
Cause in my point of view I've added some logs into the folder wich is into the input file in logstash.

---

<div class="post-metadata">

**Author:** ![azasypkin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/azasypkin/32/42216_2.png) [@azasypkin](https://discuss.elastic.co/u/azasypkin)\
**Post date:** [May 14, 2019, 8:27am UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/4 "2019-05-14T08:27:14Z")

</div>

> [@Rodolphe\_Ghio](#):
>
> Cause in my point of view I've added some logs into the folder wich is into the input file in logstash.

If you don't see any documents in logstash indices, probably something in misconfigured on the Logstash end or there is nothing to ingest. Let me forward your question to the `Logstash` hub. In the meantime please share more info on your logstash setup/configuration.

--  
Oleg

---

<div class="post-metadata">

**Author:** ![Rodolphe\_Ghio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rodolphe_ghio/32/46130_2.png) [@Rodolphe\_Ghio](https://discuss.elastic.co/u/Rodolphe_Ghio)\
**Post date:** [May 14, 2019, 8:34am UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/5 "2019-05-14T08:34:04Z")

</div>

Thanks a lot.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 14, 2019, 1:01pm UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/6 "2019-05-14T13:01:04Z")

</div>

> [@Rodolphe\_Ghio](#):
>
> ```
> file {
> path => [
> "/home/secunix/logs/",
> "/tech/"
> ]
> start_position => "beginning"
> sincedb_path => "/dev/null"
> }
> 
> ```

If you are really giving the file input directories it will not do anything, you need to give it paths to files, not directories.

Please edit your post, select the configuration and click on \</\> in the toolbar above the edit pane to blockquote it. Then make sure that the configuration in the preview panel on the right matches your configuration.

---

<div class="post-metadata">

**Author:** ![Rodolphe\_Ghio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rodolphe_ghio/32/46130_2.png) [@Rodolphe\_Ghio](https://discuss.elastic.co/u/Rodolphe_Ghio)\
**Post date:** [May 14, 2019, 1:10pm UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/8 "2019-05-14T13:10:25Z")

</div>

Ok, but that's why I've put the "\*" behind the "/" cause I think it select all the files in the directories. What do you think about this ? Do you think it will not work ?

```auto
file {
    path => [
        "/home/secunix/logs/*",
        "/tech/*"
    ]
    start_position => "beginning"
    sincedb_path => "/dev/null"
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 14, 2019, 1:12pm UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/9 "2019-05-14T13:12:28Z")

</div>

I would expect that to work. You could run with "--log.level trace" on the logstash command line and see what filewatch has to say.

---

<div class="post-metadata">

**Author:** ![Rodolphe\_Ghio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rodolphe_ghio/32/46130_2.png) [@Rodolphe\_Ghio](https://discuss.elastic.co/u/Rodolphe_Ghio)\
**Post date:** [May 14, 2019, 1:14pm UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/10 "2019-05-14T13:14:36Z")

</div>

Sorry, but I'm not sure to understand, where is the logstash command line, or where I've to type "--log.level trace" cause I've only access to the dev tool and my linux vm.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 14, 2019, 1:21pm UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/11 "2019-05-14T13:21:28Z")

</div>

If you cannot modify the command line used to start logstash you could follow the instructions at the tail end of [this](https://discuss.elastic.co/t/logstash-6-5-0-stops-shipping-logs-to-es/175477/5) post. Read the whole post.

---

<div class="post-metadata">

**Author:** ![Rodolphe\_Ghio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rodolphe_ghio/32/46130_2.png) [@Rodolphe\_Ghio](https://discuss.elastic.co/u/Rodolphe_Ghio)\
**Post date:** [May 14, 2019, 2:09pm UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/12 "2019-05-14T14:09:43Z")

</div>

![Capture](https://us1.discourse-cdn.com/elastic/original/3X/8/e/8ee9806801a475518db833f4626a1a9484dc3882.png) As you can see I don't have any TRACE loggers

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 14, 2019, 2:14pm UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/13 "2019-05-14T14:14:51Z")

</div>

It would be up to you to use the API to modify the filewatch loggers to TRACE, as the post I linked to explained.

The strange thing is you do not have any loggers for inputs, filters, or outputs, which suggests you do not have any pipelines configured.

---

<div class="post-metadata">

**Author:** ![Rodolphe\_Ghio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rodolphe_ghio/32/46130_2.png) [@Rodolphe\_Ghio](https://discuss.elastic.co/u/Rodolphe_Ghio)\
**Post date:** [May 16, 2019, 7:06am UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/15 "2019-05-16T07:06:55Z")

</div>

Bump

---

<div class="post-metadata">

**Author:** ![Rodolphe\_Ghio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rodolphe_ghio/32/46130_2.png) [@Rodolphe\_Ghio](https://discuss.elastic.co/u/Rodolphe_Ghio)\
**Post date:** [May 29, 2019, 1:41pm UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/16 "2019-05-29T13:41:49Z")

</div>

My conf file was wrong

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 26, 2019, 1:48pm UTC](https://discuss.elastic.co/t/seems-that-logstash-stop-shipping-logs-to-es-problem-into-the-logstash-config/180946/17 "2019-06-26T13:48:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.