# Issue with filebeat 6.1.2 and elasticsearch

**URL:** <https://discuss.elastic.co/t/issue-with-filebeat-6-1-2-and-elasticsearch/127102>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 6, 2018, 1:35pm UTC](https://discuss.elastic.co/t/issue-with-filebeat-6-1-2-and-elasticsearch/127102 "2018-04-06T13:35:53Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![marcandre](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@marcandre](https://discuss.elastic.co/u/marcandre)\
**Post date:** [April 6, 2018, 1:35pm UTC](https://discuss.elastic.co/t/issue-with-filebeat-6-1-2-and-elasticsearch/127102/1 "2018-04-06T13:35:53Z")

</div>

Hello,

I updated elasticsearch to version 6.1.2 I also have two version of filebeat in my infrastructure 5.6.2 and 6.1.2 I see this message in the filebeat log and the log never reach elasticsearc

```
2018-04-06T12:54:26.688Z WARN elasticsearch/client.go:502 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbea9f960688b4b59, ext:55801757983034, loc:(*time.Location)(0x200c1a0)}, Meta:common.MapStr(nil), Fields:common.MapStr{"message":"Apr 6 12:54:23 mslog puppet-agent[20048]: Applied catalog in 3.66 seconds", "prospector":common.MapStr{"type":"log"}, "beat":common.MapStr{"hostname":"mslog.oscaddie.net", "version":"6.2.3", "name":"mslog.oscaddie.net"}, "source":"/var/log/messages", "offset":1555985}, Private:file.State{Id:"", Finished:false, Fileinfo:(*os.fileStat)(0xc4205d9ee0), Source:"/var/log/messages", Offset:1555985, Timestamp:time.Time{wall:0xbea9c2e2426b28c7, ext:1118342988, loc:(*time.Location)(0x200c1a0)}, TTL:-1, Type:"log", FileStateOS:file.StateOS{Inode:0x100f8a9, Device:0xca01}}}, Flags:0x1} (status=400): {"type":"mapper_parsing_exception","reason":"Failed to parse mapping [doc]: Mapping definition for [error] has unsupported parameters: [properties : {code={type=long}, type={ignore_above=1024, type=keyword}, message={norms=false, type=text}}]","caused_by":{"type":"mapper_parsing_exception","reason":"Mapping definition for [error] has unsupported parameters: [properties : {code={type=long}, type={ignore_above=1024, type=keyword}, message={norms=false, type=text}}]"}}

```

For filebeat deployment, I use a puppet module: [GitHub - pcfens/puppet-filebeat](https://github.com/pcfens/puppet-filebeat)

> ### Filebeat configuration managed by Puppet
> 
> * * *
> 
> shutdown\_timeout: 0  
> name: [mslog.oscaddie.net](http://mslog.oscaddie.net)  
> tags:   
> fields: {}  
> fields\_under\_root: false  
> filebeat:  
> registry\_file: "/var/lib/filebeat/registry"  
> config\_dir: "/etc/filebeat/conf.d"  
> shutdown\_timeout: 0  
> output:  
> elasticsearch:  
> hosts:  
> - [mslog-int.oscaddie.net:9200](http://mslog-int.oscaddie.net:9200)  
> - [mslog.oscaddie.net:9200](http://mslog.oscaddie.net:9200)  
> protocol: https  
> username: username  
> password: Password  
> registry\_file: "/var/lib/filebeat/registry"  
> ssl.certificate\_authorities: "/path to ssl"  
> shipper: {}  
> logging: {}  
> runoptions: {}  
> processors: {}
> 
> ```
> ---
> 
> ```
> 
> filebeat:  
> prospectors:  
> - type: log  
> paths:  
> - /var/log/_.log  
> - /var/log/_/\*  
> - /var/log/\*  
> encoding: plain  
> exclude\_files:  
> - .dat$  
> - .gz$  
> - filebeat  
> - mslogprod.log  
> fields\_under\_root: false  
> document\_type: syslog-beat  
> scan\_frequency: 10s  
> harvester\_buffer\_size: 16384  
> max\_bytes: 10485760
> 
> ```
> tail_files: false
> 
> # Experimental: If symlinks is enabled, symlinks are opened and harvested. The harvester is openening the
> # original for harvesting but will report the symlink name as source.
> #symlinks: false
> 
> backoff: 1s
> max_backoff: 10s
> backoff_factor: 2
> 
> # Experimental: Max number of harvesters that are started in parallel.
> # Default is 0 which means unlimited
> #harvester_limit: 0
> 
> ### Harvester closing options
> 
> # Close inactive closes the file handler after the predefined period.
> # The period starts when the last line of the file was, not the file ModTime.
> # Time strings like 2h (2 hours), 5m (5 minutes) can be used.
> close_inactive: 5m
> 
> # Close renamed closes a file handler when the file is renamed or rotated.
> # Note: Potential data loss. Make sure to read and understand the docs for this option.
> close_renamed: false
> 
> # When enabling this option, a file handler is closed immediately in case a file can't be found
> # any more. In case the file shows up again later, harvesting will continue at the last known position
> # after scan_frequency.
> close_removed: true
> 
> # Closes the file handler as soon as the harvesters reaches the end of the file.
> # By default this option is disabled.
> # Note: Potential data loss. Make sure to read and understand the docs for this option.
> close_eof: false
> 
> ### State options
> 
> # Files for the modification data is older then clean_inactive the state from the registry is removed
> # By default this is disabled.
> clean_inactive: 0
> 
> # Removes the state for file which cannot be found on disk anymore immediately
> clean_removed: true
> 
> # Close timeout closes the harvester after the predefined time.
> # This is independent if the harvester did finish reading the file or not.
> # By default this option is disabled.
> # Note: Potential data loss. Make sure to read and understand the docs for this option.
> close_timeout: 0
> 
> ```

> GET /\_template/filebeat-\*  
> {  
> "filebeat-6.1.2": {  
> "order": 1,  
> "index\_patterns": [  
> "filebeat-6.1.2-_"  
> ],  
> But no trace of filebeat-6.1.2-_ in kibana dashboard When I try to define a index

GET /filebeat-\*/\_mapping I can't file any mapping with version 6.1.2

Centos 7.4  
filebeat test config: Config OK

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [April 9, 2018, 10:48am UTC](https://discuss.elastic.co/t/issue-with-filebeat-6-1-2-and-elasticsearch/127102/2 "2018-04-09T10:48:20Z")

</div>

Could you please share the debug logs of Filebeat? Also, please format everything using `</>`.

---

<div class="post-metadata">

**Author:** ![marcandre](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@marcandre](https://discuss.elastic.co/u/marcandre)\
**Post date:** [April 10, 2018, 2:40pm UTC](https://discuss.elastic.co/t/issue-with-filebeat-6-1-2-and-elasticsearch/127102/5 "2018-04-10T14:40:56Z")

</div>

Hello,

I realize that none of my filebeat 6.x client was able to send log to elasticsearch.

Thank you

```
filebeat -e -d "*"
2018-04-10T14:53:14.186Z INFO instance/beat.go:468 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2018-04-10T14:53:14.187Z DEBUG [beat] instance/beat.go:495 Beat metadata path: /var/lib/filebeat/meta.json
2018-04-10T14:53:14.187Z INFO instance/beat.go:475 Beat UUID: d5b83379-18aa-4e66-8efa-dc6fbab5e984
2018-04-10T14:53:14.187Z INFO instance/beat.go:213 Setup Beat: filebeat; Version: 6.2.3
2018-04-10T14:53:14.187Z DEBUG [beat] instance/beat.go:230 Initializing output plugins
2018-04-10T14:53:14.187Z DEBUG [processors] processors/processor.go:49 Processors:
2018-04-10T14:53:14.187Z INFO elasticsearch/client.go:145 Elasticsearch url: https://mslog-int.oscaddie.net:9200
2018-04-10T14:53:14.187Z INFO elasticsearch/client.go:145 Elasticsearch url: https://mslog.oscaddie.net:9200
2018-04-10T14:53:14.188Z INFO pipeline/module.go:76 Beat name: mslog.oscaddie.net
2018-04-10T14:53:14.188Z WARN [cfgwarn] config/config.go:100 DEPRECATED: config_dir is deprecated. Use `filebeat.config.prospectors` instead. Will be removed in version: 7.0.0
2018-04-10T14:53:14.188Z INFO config/config.go:106 Additional config files are fetched from: /etc/filebeat/conf.d
2018-04-10T14:53:14.188Z INFO config/config.go:75 Additional configs loaded from: /etc/filebeat/conf.d/syslogs.yml
2018-04-10T14:53:14.188Z INFO instance/beat.go:301 filebeat start running.
2018-04-10T14:53:14.188Z INFO [monitoring] log/log.go:97 Starting metrics logging every 30s
2018-04-10T14:53:14.189Z DEBUG [registrar] registrar/registrar.go:88 Registry file set to: /var/lib/filebeat/registry
2018-04-10T14:53:14.189Z INFO registrar/registrar.go:108 Loading registrar data from /var/lib/filebeat/registry
2018-04-10T14:53:14.193Z INFO registrar/registrar.go:119 States Loaded from registrar: 357
2018-04-10T14:53:14.193Z INFO crawler/crawler.go:48 Loading Prospectors: 1
2018-04-10T14:53:14.193Z DEBUG [processors] processors/processor.go:49 Processors:
2018-04-10T14:53:14.193Z DEBUG [registrar] registrar/registrar.go:150 Starting Registrar
2018-04-10T14:53:14.194Z DEBUG [prospector] log/config.go:178 recursive glob enabled
2018-04-10T14:53:14.194Z DEBUG [prospector] log/prospector.go:120 exclude_files: [\.dat(?-m:$) \.gz(?-m:$) <substring 'filebeat'> mslogprod(?-s:.)log]. Number of stats: 357
2018-04-10T14:53:14.194Z DEBUG [prospector] file/state.go:82 New state added for /var/log/boot.log
2018-04-10T14:53:14.194Z DEBUG [prospector] file/state.go:82 New state added for /var/log/cloud-init-output.log
2018-04-10T14:53:14.194Z DEBUG [prospector] file/state.go:82 New state added for /var/log/cloud-init.log
2018-04-10T14:53:14.194Z DEBUG [registrar] registrar/registrar.go:200 Processing 1 events
2018-04-10T14:53:14.194Z DEBUG [registrar] registrar/registrar.go:193 Registrar states cleaned up. Before: 357, After: 357
2018-04-10T14:53:14.194Z DEBUG [registrar] registrar/registrar.go:228 Write registry file: /var/lib/filebeat/registry
2018-04-10T14:53:14.194Z DEBUG [prospector] file/state.go:82 New state added for /var/log/yum.log-20180101
2018-04-10T14:53:14.194Z DEBUG [prospector] file/state.go:82 New state added for /var/log/java_install.log
2018-04-10T14:53:14.198Z DEBUG [registrar] registrar/registrar.go:253 Registry file updated. 357 states written.
2018-04-10T14:53:14.198Z DEBUG [registrar] registrar/registrar.go:200 Processing 1 events
2018-04-10T14:53:14.198Z DEBUG [registrar] registrar/registrar.go:193 Registrar states cleaned up. Before: 357, After: 357
2018-04-10T14:53:14.198Z DEBUG [registrar] registrar/registrar.go:228 Write registry file: /var/lib/filebeat/registry
2018-04-10T14:53:14.198Z DEBUG [prospector] file/state.go:82 New state added for /var/log/grubby
2018-04-10T14:53:14.202Z DEBUG [registrar] registrar/registrar.go:253 Registry file updated. 357 states written.
2018-04-10T14:53:14.202Z DEBUG [registrar] registrar/registrar.go:200 Processing 1 events
2018-04-10T14:53:14.202Z DEBUG [registrar] registrar/registrar.go:193 Registrar states cleaned up. Before: 357, After: 357
2018-04-10T14:53:14.202Z DEBUG [registrar] registrar/registrar.go:228 Write registry file: /var/lib/filebeat/registry
```

---

<div class="post-metadata">

**Author:** ![marcandre](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@marcandre](https://discuss.elastic.co/u/marcandre)\
**Post date:** [April 12, 2018, 6:30pm UTC](https://discuss.elastic.co/t/issue-with-filebeat-6-1-2-and-elasticsearch/127102/6 "2018-04-12T18:30:17Z")

</div>

Hello,

Anyone have an idea why my filebeat client version 6.x are not able to send log to elasticsearch ?

Thank you

---

<div class="post-metadata">

**Author:** ![marcandre](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@marcandre](https://discuss.elastic.co/u/marcandre)\
**Post date:** [April 16, 2018, 8:14pm UTC](https://discuss.elastic.co/t/issue-with-filebeat-6-1-2-and-elasticsearch/127102/7 "2018-04-16T20:14:44Z")

</div>

I found the solution. I had to update the index a second time for 6.2.3

Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 14, 2018, 10:14pm UTC](https://discuss.elastic.co/t/issue-with-filebeat-6-1-2-and-elasticsearch/127102/8 "2018-05-14T22:14:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.