# Missing data and a lot of warning and errors when logging to ES **URL:** \ **Category:** Beats\ **Tags:** metricbeat\ **Created:** [May 24, 2018, 3:02pm UTC](https://discuss.elastic.co/t/missing-data-and-a-lot-of-warning-and-errors-when-logging-to-es/133181 "2018-05-24T15:02:31Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![ilijamt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ilijamt/32/31573_2.png) [@ilijamt](https://discuss.elastic.co/u/ilijamt)\ **Post date:** [May 24, 2018, 3:02pm UTC](https://discuss.elastic.co/t/missing-data-and-a-lot-of-warning-and-errors-when-logging-to-es/133181/1 "2018-05-24T15:02:32Z")
I have metricbeat 6.2.4 with ES 6.2.4 and my logs are full of Metricbeat: [https://pastebin.com/VyXVYRk5](https://pastebin.com/VyXVYRk5) ES: [https://pastebin.com/8514Lcz8](https://pastebin.com/8514Lcz8) And the data is missing and not showing information regarding some of the statistics. On average everytime it pushesh to ES metricbeat generate ~8m of log file. Configuration for metricbeat > metricbeat.config.modules: > path: ${path.config}/modules.d/\*.yml > > fields: > env: "production" > > # Auto load the template > > setup.template.overwrite: False > > # Configure elasticsearch cluser for beats > > output.elasticsearch: > hosts: ["10.0.0.200:9200", "10.0.0.92:9200", "10.0.0.93:9200", "10.0.0.94:9200", "10.0.0.199:9200"] /etc/metricbeat/modules.d/system.yml > - metricsets: [cpu, load, memory, network, process, process\_summary, core, diskio, > socket] > module: system > period: 30s > process.include\_cpu\_ticks: true > processes: [.\*] > - filesystem.ignore\_types: [nfs, smbfs, autofs] > metricsets: [filesystem, fsstat] > module: system > period: 5m > processors: > - drop\_event.when.regexp: {system.filesystem.mount\_point: ^/(sys|cgroup|proc|dev|etc|host|lib)($|/)} > > - metricsets: [uptime, raid] > module: system > period: 15m ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [May 25, 2018, 1:17pm UTC](https://discuss.elastic.co/t/missing-data-and-a-lot-of-warning-and-errors-when-logging-to-es/133181/2 "2018-05-25T13:17:40Z")
What metricbeat template are you using? Did you disable template loading for Beats in general. There seems to be mismatch between the docs you try to ingest and the template. ---
**Author:** ![ilijamt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ilijamt/32/31573_2.png) [@ilijamt](https://discuss.elastic.co/u/ilijamt)\ **Post date:** [May 25, 2018, 5:38pm UTC](https://discuss.elastic.co/t/missing-data-and-a-lot-of-warning-and-errors-when-logging-to-es/133181/3 "2018-05-25T17:38:52Z")
It was on a clean ES installation. I loaded the templates by using metricbeat setup The issue seems to resolved on it's own, now I have a problem with filebeat parsing a time.Time from Golang the logs is full of it. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [May 28, 2018, 10:54am UTC](https://discuss.elastic.co/t/missing-data-and-a-lot-of-warning-and-errors-when-logging-to-es/133181/4 "2018-05-28T10:54:02Z")
Strange that it resolved itself. Could be that this happened when a new index was created? For the parsing error: Best open a new discuss issue to not mix different issues in the same topic. ---
**Author:** ![ilijamt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ilijamt/32/31573_2.png) [@ilijamt](https://discuss.elastic.co/u/ilijamt)\ **Post date:** [May 28, 2018, 3:49pm UTC](https://discuss.elastic.co/t/missing-data-and-a-lot-of-warning-and-errors-when-logging-to-es/133181/5 "2018-05-28T15:49:59Z")
Nothing was pushing to ES at the time, I first created the indexes using the setup commands and then started metricbeat. So yeah it was very strange, about what happened. But there is no errors in the log at the moment so we will see how it goes. Regarding the fliebeat yeah I will open a separate issue ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 25, 2018, 3:50pm UTC](https://discuss.elastic.co/t/missing-data-and-a-lot-of-warning-and-errors-when-logging-to-es/133181/6 "2018-06-25T15:50:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.