# Filebeat not trasnferring messages to Logstash

**URL:** <https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 24, 2018, 6:59am UTC](https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311 "2018-07-24T06:59:56Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nithani25](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nithani25/32/33457_2.png) [@Nithani25](https://discuss.elastic.co/u/Nithani25)\
**Post date:** [July 24, 2018, 6:59am UTC](https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311/1 "2018-07-24T06:59:57Z")

</div>

HI Team,  
I have completed ELK setup completed and successful in creating many visualtisation, however, offlate my filebeat is not sending any messages to my logstash system. I am getting below message when trying to publish

## 

```auto
  ./filebeat -e -c filebeat.yml -d "publish"
2018-07-23T23:46:15.212-0700 INFO instance/beat.go:468 Home path: [/opt/bea/ELKSTACK/filebeat-6.2.4-linux-x86_64] Config path: [/opt/bea/ELKSTACK/filebeat-6.2.4-linux-x86_64] Data path: [/opt/bea/ELKSTACK/filebeat-6.2.4-linux-x86_64/data] Logs path: [/opt/bea/ELKSTACK/filebeat-6.2.4-linux-x86_64/logs]
2018-07-23T23:46:15.212-0700 INFO instance/beat.go:475 Beat UUID: 57d1129b-8370-49a0-80ca-15f15e15e377
2018-07-23T23:46:15.212-0700 INFO instance/beat.go:213 Setup Beat: filebeat; Version: 6.2.4
2018-07-23T23:46:15.213-0700 INFO pipeline/module.go:76 Beat name: STOXXXXX.corp.prk.com
2018-07-23T23:46:15.213-0700 INFO [monitoring] log/log.go:97 Starting metrics logging every 30s
2018-07-23T23:46:15.213-0700 INFO instance/beat.go:301 filebeat start running.
2018-07-23T23:46:15.213-0700 INFO registrar/registrar.go:73 No registry file found under: /opt/bea/ELKSTACK/filebeat-6.2.4-linux-x86_64/data/registry. Creating a new registry file.
2018-07-23T23:46:15.216-0700 INFO registrar/registrar.go:110 Loading registrar data from /opt/bea/ELKSTACK/filebeat-6.2.4-linux-x86_64/data/registry
2018-07-23T23:46:15.216-0700 INFO registrar/registrar.go:121 States Loaded from registrar: 0
2018-07-23T23:46:15.216-0700 WARN beater/filebeat.go:261 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2018-07-23T23:46:15.216-0700 INFO crawler/crawler.go:48 Loading Prospectors: 1
2018-07-23T23:46:15.218-0700 INFO log/prospector.go:111 Configured paths: [/opt/bea/Logwarhouse/Genacess/messages]
2018-07-23T23:46:15.218-0700 INFO crawler/crawler.go:82 Loading and starting Prospectors completed. Enabled prospectors: 1
2018-07-23T23:46:15.218-0700 INFO cfgfile/reload.go:127 Config reloader started
2018-07-23T23:46:15.218-0700 INFO cfgfile/reload.go:219 Loading of config files completed.
2018-07-23T23:46:45.216-0700 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":10,"time":11},"total":{"ticks":20,"time":26,"value":20},"user":{"ticks":10,"time":15}},"info":{"ephemeral_id":"e80fa1bc-b620-4750-bd55-e78defa06843","uptime":{"ms":30009}},"memstats":{"gc_next":4194304,"memory_alloc":1394592,"memory_total":2918704,"rss":14323712}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"type":"logstash"},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0},"writes":1},"system":{"cpu":{"cores":4},"load":{"1":0.54,"15":0.25,"5":0.37,"norm":{"1":0.135,"15":0.0625,"5":0.0925}}}}}}
------------------------------------------------------------------------------------

Filebeat Input:
 -----------------

paths:
    - /opt/bea/Logwarhouse/Genacess/messages
include_lines: ['^Apigee']
close_inactive: 20m
multiline.pattern: '^\w{3}\s\s\d{2}'
multiline.negate: true
multiline.match: after

output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5089"]

```

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [July 24, 2018, 1:28pm UTC](https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311/2 "2018-07-24T13:28:28Z")

</div>

Tried to format your above config but I don't think I was successful. Could you paste your exact config? If the above is correct, there are issues with the indentation.

---

<div class="post-metadata">

**Author:** ![Nithani25](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nithani25/32/33457_2.png) [@Nithani25](https://discuss.elastic.co/u/Nithani25)\
**Post date:** [July 24, 2018, 1:46pm UTC](https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311/3 "2018-07-24T13:46:19Z")

</div>

Yes the above details are right, can you help me by highlighting the mistakes(indentation misatkes)

---

<div class="post-metadata">

**Author:** ![Nithani25](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nithani25/32/33457_2.png) [@Nithani25](https://discuss.elastic.co/u/Nithani25)\
**Post date:** [July 24, 2018, 1:51pm UTC](https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311/4 "2018-07-24T13:51:32Z")

</div>

My log file looks below

\< Jul 2 02:30:03 10.66.9.201 [] 2018-07-02T09:30:03.003+0000 fez8b5754-125c-47g-850c-21ae56f6c080 Apigee-Edge - - 1530523803518|US-Common-Accounts|/v1-log/us/common/accounts/middlewareSOAP|POST|Debug|Response-Target|[Connection: close][Co  
ntent-Length: 1174][Content-Type: text/xml;charset=utf-8][Date: Mon, 02 Jul 2018 09:30:03 GMT],|200|x4Jt3ak41J|GetAccount|10x.2xx.1xx.xx|c7373307-3457-6bff-a5a6-3e4ad69171e1|PostmanRuntime/6.1.6|\<soapenv:Envelope xmlns:soapenv="[http://sc](http://sc)  
[hemas.xmlsoap.org/soap/envelope/](http://hemas.xmlsoap.org/soap/envelope/)" xmlns:oas="[http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd](http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd)" xmlns:add="[http://schemas.xmlsoap.org/ws/2004/08/addressing](http://schemas.xmlsoap.org/ws/2004/08/addressing)" xmlns:v260="[http://middleware.prk.com/XMLSchemas](http://middleware.prk.com/XMLSchemas)  
/AccountGetDetails/v260"\>soapenv:Headeradd:Action[http://sharedservices.prk.com/AccountGetDetailsv260/sendSynchronousResponse](http://sharedservices.prk.com/AccountGetDetailsv260/sendSynchronousResponse)\</add:Action\>add:To[http://test.GWSBETA.frk.com/](http://test.GWSBETA.frk.com/)\</add:To\>\<add:RelatesTo RelationshipType="wsa:Reply"\>c737330  
7-3457-4bff-a5a6-3e4ad69171e1\</add:RelatesTo\>add:MessageID7fb537427dda11e8a0210a415f66000013185c3277000000\</add:MessageID\>\</soapenv:Header\>soapenv:Bodysoapenv:FaultReceiverSystem error\</faultstri  
ng\>COMN0025Account Not FoundAccount.GetDetailsHigh2018-07-02  
T02:30:03.374-07:00FdNbr-AccNbr111-000000234512345\</soapenv:Fault\>\</soapenv:Body\>\</soapenv:Envelope\> /\>

Since i am looking for messages which as Apigee in them to be indexed, i have mentioned include pattern.

---

<div class="post-metadata">

**Author:** ![Nithani25](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nithani25/32/33457_2.png) [@Nithani25](https://discuss.elastic.co/u/Nithani25)\
**Post date:** [July 24, 2018, 2:00pm UTC](https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311/5 "2018-07-24T14:00:58Z")

</div>

Another important point, i am working on two other log patterns and filebeat is still not sending the input.

Also everything was fine tine till last night, i created one index pattern.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [July 24, 2018, 6:29pm UTC](https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311/6 "2018-07-24T18:29:25Z")

</div>

You find a few valid examples with the correct indentation here in the getting started guide: [https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration.html](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration.html)

---

<div class="post-metadata">

**Author:** ![Nithani25](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nithani25/32/33457_2.png) [@Nithani25](https://discuss.elastic.co/u/Nithani25)\
**Post date:** [July 25, 2018, 5:17am UTC](https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311/7 "2018-07-25T05:17:08Z")

</div>

Hi,  
I dont think i have made any intendation errors this time, but i still get the same error.

\<  
filebeat.prospectors:

# Each - is a prospector. Most options can be set at the prospector level, so

# you can use different prospectors for various configurations.

# Below are the prospector specific configurations.

- type: log

>

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [July 25, 2018, 6:20am UTC](https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311/8 "2018-07-25T06:20:48Z")

</div>

Could you format your config above with 3 ticks to make readable?

Also please run the beat with `-d` option which will enable debug output. This should tell us more.

---

<div class="post-metadata">

**Author:** ![Nithani25](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nithani25/32/33457_2.png) [@Nithani25](https://discuss.elastic.co/u/Nithani25)\
**Post date:** [July 25, 2018, 7:06am UTC](https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311/9 "2018-07-25T07:06:20Z")

</div>

Hi Ruflin,  
I am able to fix it by myself, big thanks for identifying the problem with indentation though!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 22, 2018, 7:06am UTC](https://discuss.elastic.co/t/filebeat-not-trasnferring-messages-to-logstash/141311/10 "2018-08-22T07:06:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.