# Filbeat 6.3.1 not sending logs to stack 6.3.1
**URL:** https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188
**Category:** Beats
**Tags:** filebeat
**Created:** [July 16, 2018, 4:02pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188 "2018-07-16T16:02:08Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@Diggy](https://discuss.elastic.co/u/Diggy)
#### Post date: [July 16, 2018, 4:02pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/1 "2018-07-16T16:02:09Z")
I recently updated out Elastic stack to version 6.3.1. Last night, I updated filebeat on most hosts. From the time of update to now, log data are no longer being received. However, from a small number of hosts still running filebeat 6.2.4, the logs are being received. The config file for both 6.3.1 and 6.2.4 is identical. Help would be appreciated fixing this.
---
### Author:  [@Diggy](https://discuss.elastic.co/u/Diggy)
#### Post date: [July 16, 2018, 5:26pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/2 "2018-07-16T17:26:20Z")
An update - as soons as I rolled filebeat back to 6.2.4, on one server, I began to receive logs. But, of course, that's not a solution.
---
### Author:  [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [July 16, 2018, 6:42pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/3 "2018-07-16T18:42:54Z")
Could you share your filebeat logs and config? And do you use Logstash?
---
### Author:  [@Diggy](https://discuss.elastic.co/u/Diggy)
#### Post date: [July 16, 2018, 7:15pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/4 "2018-07-16T19:15:08Z")
ruflin,
I do use logstash. Filebeat config is here: [https://pastebin.com/HZyLQePa](https://pastebin.com/HZyLQePa) . The log file is here: [https://drive.google.com/open?id=1yCsc7xR4g2dckJmQIBY-yRPIJrATddlh](https://drive.google.com/open?id=1yCsc7xR4g2dckJmQIBY-yRPIJrATddlh) .
---
### Author:  [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [July 17, 2018, 9:16am UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/5 "2018-07-17T09:16:55Z")
Can you also share your LS config? How do you load the Filebeat template?
In the log file the following two lines show up pretty frequently:
```auto
|2018-07-16T14:00:54.204-0400|ERROR|logstash/async.go:235|Failed to publish events caused by: write tcp 69.25.192.68:58748->10.0.101.101:5044: write: connection reset by peer|
|---|---|---|---|
|2018-07-16T14:00:55.204-0400|ERROR|pipeline/output.go:92|Failed to publish events: write tcp 69.25.192.68:58748->10.0.101.101:5044: write: connection reset by peer|
```
Do have something like a proxy between FB and LS?
---
### Author:  [@Diggy](https://discuss.elastic.co/u/Diggy)
#### Post date: [July 17, 2018, 4:56pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/6 "2018-07-17T16:56:35Z")
ruflin,
Thanks for working on this with me.
As things would have it, when I looked in Kibana this morning, the logs sent by filebeat 6.3.1 servers are now being received. Huh? OK, great. But whether it's due to something I did (doubtful), or just not being patient (probable), I don't know.
However, that you found something wonky in the log file is something I'd like to pursue, if you don't mind. I put the LS configs (I've split them out into input, output, and filters) here: [https://pastebin.com/JijR5Sjt](https://pastebin.com/JijR5Sjt) .
Admittedly, despite my efforts, I've never been good at configuring the stack. As to filebeat, I believe I generated filebeat.template.json on one of the filebeat servers, then applied it by running this (quotes added for clarity): "curl -XPUT -H 'Content-Type: application/json' [http://10.0.101.101:9200/\_template/filebeat-6.2.4](http://10.0.101.101:9200/_template/filebeat-6.2.4) -d@filebeat.template.json". That's probably completely wrong, but seems to work, in part anyway. However, I've never been able to get geoip to work, despite considerable hacking, for instance.
Anyway, I would appreciate it if you could take a look, and avise.
Oh, and there is no proxy between FB and LS.
Diggy
---
### Author:  [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [July 18, 2018, 8:58am UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/7 "2018-07-18T08:58:18Z")
The error above indicates that sometimes the connection to LS is flaky. It could be the network but also LS being unresponsive. How many LS instance you have? Could it be that they are overloaded?
---
### Author:  [@Diggy](https://discuss.elastic.co/u/Diggy)
#### Post date: [July 18, 2018, 1:12pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/8 "2018-07-18T13:12:43Z")
I have just one LS instance. I monitor it, and it doesn't seem overloaded. But, I'll take another look.
Do my configs seem right, and did I load the Filebeat template properly? It puzzles me as to why Filebeat-based visualizations don't work (e.g. geoip). I note that there are just a small handful of "available fields" for Filebeat in Kbana. I've attached a picture of those fields (I think).
---
### Author:  [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [July 19, 2018, 12:07pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/9 "2018-07-19T12:07:51Z")
I would also expect a few more field. How did you load the template and index pattern?
---
### Author:  [@Diggy](https://discuss.elastic.co/u/Diggy)
#### Post date: [July 19, 2018, 12:53pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/10 "2018-07-19T12:53:48Z")
Well, therein lies the problem. I'm not exactly sure how to load them. As I mentioned above, I created "filebeat.template.json" on a filebeat box, then applied it by running this "curl -XPUT -H 'Content-Type: application/json' [http://10.0.101.101:9200/\_template/filebeat-6.2.4](http://10.0.101.101:9200/_template/filebeat-6.2.4) -d@filebeat.template.json". I did that once. It's embarrassing to ask, but does that take care of loading the template? As to loading the index pattern, I'm not sure how to do that either. I know there's plenty of documentation out there, but I've just never gotten the hang of how to do this. I know that asking for a tutorial of sorts from you is probably a pita, but I would certainly appreciate it. That way, I'll know how to make filebeat work as intended.
With continued thanks.
---
### Author:  [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [July 20, 2018, 8:09am UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/11 "2018-07-20T08:09:52Z")
Any chance to try the setup without Logstash? Because in this case you could just run `filebeat` and it would automatically setup the template and indices for you.
I wanted to check your LS config again to see what you do with LS, unfortunately the page is removed.
---
### Author:  [@Diggy](https://discuss.elastic.co/u/Diggy)
#### Post date: [July 20, 2018, 1:43pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/12 "2018-07-20T13:43:45Z")
ruflin,
I've reposted the LS config(s) here: [https://pastebin.com/SMKcchDr](https://pastebin.com/SMKcchDr)
Thanks for taking a look.
---
### Author:  [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [July 23, 2018, 9:02am UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/13 "2018-07-23T09:02:53Z")
Thanks for sharing it again. Is see your LS config is rather complex so LS is here to stay 🙂
There are 2 parts: index template for elasticsearch and index pattern for Kibana. Both can be loaded by the beat directly if pointed to ES / KB directly. The screenshot you posted above is from the index pattern in Kibana.
One thing I just realised now is that the command you post above is for loading the template for 6.2.4. Each version of Beats has it's own template and indices. So if you want to template for 6.3.1 you must also load it (before indices exist).
---
### Author:  [@Diggy](https://discuss.elastic.co/u/Diggy)
#### Post date: [July 23, 2018, 3:06pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/14 "2018-07-23T15:06:00Z")
Thank you.
Here's what I did before replying:
1. generated new template - filebeat export template \> /home/dyioulos/filebeat.template.json
2. installed new template - curl -XPUT -H 'Content-Type: application/json' [http://10.0.101.101:9200/\_template/filebeat-6.3.1](http://10.0.101.101:9200/_template/filebeat-6.3.1) -d@filebeat.template.json
3. cleared out old indices - curl -XDELETE '[http://10.0.101.101:9200/filebeat-\*](http://10.0.101.101:9200/filebeat-*)'
4. restarted the stack
5. refreshed filebeat-\* field list through Kibana Management page.
I still must be missing something, because I see the same filebeat fields in Kibana:
---
### Author:  [@Diggy](https://discuss.elastic.co/u/Diggy)
#### Post date: [July 24, 2018, 2:40pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/15 "2018-07-24T14:40:00Z")
Can anyone spot anything I'm doing wrong?
---
### Author:  [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [July 25, 2018, 6:41am UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/16 "2018-07-25T06:41:54Z")
If you connect Filebeat directly to Elasticsearch you can let filebeat load the index pattern with `filebeat setup`.
What do the documents look like that you have in the 6.3.1 indices? This thread started with not being able to send data, do you see data now?
---
### Author:  [@Diggy](https://discuss.elastic.co/u/Diggy)
#### Post date: [July 25, 2018, 1:07pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/17 "2018-07-25T13:07:39Z")
I do see the data now, and thank you for your help there. Apologies if I should have started a new thread regarding the index pattern. I hope it's OK if I just finish this out.
What is the full "filebeat setup" command that I should run? And, I run that from a host on which filebeat is installed, correct?
Again, many thanks.
---
### Author:  [@lenn4rd](https://discuss.elastic.co/u/lenn4rd)
#### Post date: [July 25, 2018, 5:51pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/18 "2018-07-25T17:51:12Z")
I recently had some issues after upgrading some Filebeat agents from 6.3.0 to 6.3.1. After reading through this thread and seeing your pointer to re-running the `setup` action I did just that and it worked.
I was assuming changes to the micro version (i.e. jumping from .0 to .1) won't break anything. Is that correct or should I re-import both the index template and index pattern every time the version number increases?
---
### Author:  [@lenn4rd](https://discuss.elastic.co/u/lenn4rd)
#### Post date: [July 25, 2018, 5:57pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/19 "2018-07-25T17:57:10Z")
Here's what `filebeat setup` does (see yourself with `filebeat help setup`):
```
This command does initial setup of the environment:
* Index mapping template in Elasticsearch to ensure fields are mapped.
* Kibana dashboards (where available).
* ML jobs (where available).
* Ingest pipelines (where available).
[...]
```
There are a few flags you can set (again, check the `help` command) to e.g. set up the index template only.
You can run this command either from a host already configured or from your local machine, e.g. using a Docker image. The agent doesn't need the full configuration because you can override the configuration at runtime. Have a look at the docs on [how to load the template manually](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html#load-template-manually).
---
### Author:  [@Diggy](https://discuss.elastic.co/u/Diggy)
#### Post date: [July 25, 2018, 8:39pm UTC](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188/20 "2018-07-25T20:39:08Z")
When I run "curl '10.0.101.101:9200/\_cat/templates?v&s=name&pretty'", this is what I get:
name index\_patterns order version
.ml-anomalies- [.ml-anomalies-_] 0 6030199
.ml-meta [.ml-meta] 0 6030199
.ml-notifications [.ml-notifications] 0 6030199
.ml-state [.ml-state] 0 6030199
.monitoring-alerts [.monitoring-alerts-6] 0 6020099
.monitoring-beats [.monitoring-beats-6-_] 0 6020099
.monitoring-es [.monitoring-es-6-_] 0 6020099
.monitoring-kibana [.monitoring-kibana-6-_] 0 6020099
.monitoring-logstash [.monitoring-logstash-6-_] 0 6020099
.triggered\_watches [.triggered\_watches_] 2147483647
.watch-history-7 [.watcher-history-7\*] 2147483647
.watches [.watches\*] 2147483647
**filebeat-6.2.2 [filebeat-6.2.0-\*] 1**
**filebeat-6.2.4 [filebeat-6.2.0-\*] 1**
**filebeat-6.3.1 [filebeat-6.3.1-\*] 1**
kibana\_index\_template:.kibana [.kibana] 0
logstash [logstash-_] 0 60001
logstash-index-template [.logstash] 0
security-index-template [.security-_] 1000
security\_audit\_log [.security\_audit\_log\*] 1000
Too many filebeat templates?
[Next page](https://discuss.elastic.co/t/filbeat-6-3-1-not-sending-logs-to-stack-6-3-1/140188.md?page=2)