# Problem with Filebeat sending data to logstash

**URL:** https://discuss.elastic.co/t/problem-with-filebeat-sending-data-to-logstash/108733
**Category:** Beats
**Tags:** filebeat
**Created:** [November 22, 2017, 2:57pm UTC](https://discuss.elastic.co/t/problem-with-filebeat-sending-data-to-logstash/108733 "2017-11-22T14:57:51Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Tal\_Druckmann](https://avatars.discourse-cdn.com/v4/letter/t/b9e5f3/32.png) [@Tal\_Druckmann](https://discuss.elastic.co/u/Tal_Druckmann)
#### Post date: [November 22, 2017, 2:57pm UTC](https://discuss.elastic.co/t/problem-with-filebeat-sending-data-to-logstash/108733/1 "2017-11-22T14:57:52Z")

</div>

After I successfully made it to connect FileBeat with Logstash and Elasticsearch I am now having troubles with it, I tried to adjust my filter in Logstash and even after getting it back to the initial state I am unable to connect them.  
I'm running all as windows services. My operation system is VMWare workstation windows 2016 and here are my config files:  
filebeat.yaml:  
#=========================== Filebeat prospectors =============================  
filebeat.prospectors:  
- type: log

```
	enabled: true

	paths:
		- c:\programdata\filebeat\Applogs\AxesPlus\*.txt
	
#============================= Filebeat modules ===============================

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml

  reload.enabled: false
  
#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 3
  
#================================ General =====================================

tags: ["AxesPlus"]

#============================== Kibana =====================================

setup.kibana:

  host: "localhost:5601"
#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
    hosts: ["localhost:5044"]

```

logstash config:

```
input {
  beats {
   port => 5044
   type => "log"
  }
}

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

```

Logstash log:  
[2017-11-22T16:11:19,430][INFO][logstash.pipeline] Pipeline started {"[pipeline.id](http://pipeline.id)"=\>"main"}  
[2017-11-22T16:11:19,444][INFO][org.logstash.beats.Server] Starting server on port: 5044  
[2017-11-22T16:11:19,546][INFO][logstash.agent] Pipelines running {:count=\>1, :pipelines=\>["main"]}

filebeat:  
2017-11-22T16:32:12+02:00 INFO Home path: [C:\Program Files\Filebeat] Config path: [C:\Program Files\Filebeat] Data path: [C:\Program Files\Filebeat\data] Logs path: [C:\Program Files\Filebeat\logs]  
2017-11-22T16:32:12+02:00 INFO Beat UUID: 9b0b57c2-7109-471c-9295-d0bbc9f21ab7  
2017-11-22T16:32:12+02:00 INFO Metrics logging every 30s  
2017-11-22T16:32:12+02:00 INFO Setup Beat: filebeat; Version: 6.0.0  
2017-11-22T16:32:12+02:00 INFO Elasticsearch url: [http://localhost:9200](http://localhost:9200)  
2017-11-22T16:32:12+02:00 INFO Beat name: WIN-E56I4JOUMH9  
2017-11-22T16:32:12+02:00 INFO Elasticsearch url: [http://localhost:9200](http://localhost:9200)  
2017-11-22T16:32:12+02:00 INFO Connected to Elasticsearch version 6.0.0  
2017-11-22T16:32:12+02:00 INFO Template already exists and will not be overwritten.

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [November 22, 2017, 8:35pm UTC](https://discuss.elastic.co/t/problem-with-filebeat-sending-data-to-logstash/108733/2 "2017-11-22T20:35:34Z")

</div>

Could you provide more logs from Filebeat? Is there any concrete error message?

---

<div class="post-metadata">

### Author: ![Tal\_Druckmann](https://avatars.discourse-cdn.com/v4/letter/t/b9e5f3/32.png) [@Tal\_Druckmann](https://discuss.elastic.co/u/Tal_Druckmann)
#### Post date: [November 22, 2017, 9:07pm UTC](https://discuss.elastic.co/t/problem-with-filebeat-sending-data-to-logstash/108733/3 "2017-11-22T21:07:22Z")

</div>

I don't have much logs. Now I started using the filebeat as process not as a service. It worked for a while but now it doesn't. I run it with debug log and it sends details of the beat it is sending, yet nothing gets to elastic. It says from time to time that it has connection time out on connection. Although I don't see any error on logstash log file

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [November 23, 2017, 11:13am UTC](https://discuss.elastic.co/t/problem-with-filebeat-sending-data-to-logstash/108733/4 "2017-11-23T11:13:55Z")

</div>

Could you show the output of `filebeat -e -d "*"`?

---

<div class="post-metadata">

### Author: ![Tal\_Druckmann](https://avatars.discourse-cdn.com/v4/letter/t/b9e5f3/32.png) [@Tal\_Druckmann](https://discuss.elastic.co/u/Tal_Druckmann)
#### Post date: [November 23, 2017, 12:41pm UTC](https://discuss.elastic.co/t/problem-with-filebeat-sending-data-to-logstash/108733/5 "2017-11-23T12:41:02Z")

</div>

Thanks for your help. Eventually, it appears all my trouble was that the file it is following wasn't change. Now I am having trouble replacing the default timestamp filed.

---

<div class="post-metadata">

### Author: ![lara](https://avatars.discourse-cdn.com/v4/letter/l/e274bd/32.png) [@lara](https://discuss.elastic.co/u/lara)
#### Post date: [November 23, 2017, 1:45pm UTC](https://discuss.elastic.co/t/problem-with-filebeat-sending-data-to-logstash/108733/6 "2017-11-23T13:45:05Z")

</div>

Hi Tal,

Coming back to your original question. I thought you used logstash as an output for filebeat.  
Why then, in the logging output you posted, one sees such lines as

xxxxxxxxx INFO Elasticsearch xxxxxxx  
xxx INFO Connected to Elasticsearch xxxxxx

I have not tried Filebeat 6.0 but it seems that maybe in the filebeat.yml the elasticsearch output was left uncommented (as per default)?  
When I start my Filebeat 5.6. x (connecting to Logstash) it clearly states its output as Logstash and proceeds.  
You said you had run the connection with success, at least once.  
What had you changed since? Maybe you put back an original filebeat.yml with the elastic output uncommented.  
As I have not followed the evolution of your tests and as I do not use Filebeat 6.0, sorry in case suggestions are not pertinent to your problems.  
Cheers

Lara

---

<div class="post-metadata">

### Author: ![Tal\_Druckmann](https://avatars.discourse-cdn.com/v4/letter/t/b9e5f3/32.png) [@Tal\_Druckmann](https://discuss.elastic.co/u/Tal_Druckmann)
#### Post date: [November 26, 2017, 8:30am UTC](https://discuss.elastic.co/t/problem-with-filebeat-sending-data-to-logstash/108733/7 "2017-11-26T08:30:23Z")

</div>

Thank you so much for your help. My problem was that the log file that was attached to filebeat wasn't changed. Thus, it wouldn't upload anything. It took filebeat a while until it logged that line that said that the file wasn't changed.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 24, 2017, 8:30am UTC](https://discuss.elastic.co/t/problem-with-filebeat-sending-data-to-logstash/108733/8 "2017-12-24T08:30:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
