# Data loss when sending logs to Kibana through Filebeat

**URL:** https://discuss.elastic.co/t/data-loss-when-sending-logs-to-kibana-through-filebeat/315926
**Category:** Beats
**Tags:** filebeat
**Created:** [October 6, 2022, 1:42am UTC](https://discuss.elastic.co/t/data-loss-when-sending-logs-to-kibana-through-filebeat/315926 "2022-10-06T01:42:50Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![xyu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xyu/32/111725_2.png) [@xyu](https://discuss.elastic.co/u/xyu)
#### Post date: [October 6, 2022, 1:42am UTC](https://discuss.elastic.co/t/data-loss-when-sending-logs-to-kibana-through-filebeat/315926/1 "2022-10-06T01:42:50Z")

</div>

Hi, I am new to elastic stalk and currently I am facing some difficulties to read all the logs in JSON format.

> {"code":"28000","file":"auth.c","length":164,"level":"error","line":"496","message":"no entry for host","name":"error","timestamp":"2022-10-05 08:40:14.308"}

All the lines started with "code":"28000" is not able to send to Kibana. Does anyone know what is the issue?

> {"code":"SELF\_SIGNED\_CERT\_IN\_CHAIN","level":"error","message":"self signed certificate","timestamp":"2022-10-05 08:40:39.908"}

However, the above line is able to be sent to Kibana.

Thank you for helping

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 6, 2022, 2:40am UTC](https://discuss.elastic.co/t/data-loss-when-sending-logs-to-kibana-through-filebeat/315926/2 "2022-10-06T02:40:05Z")

</div>

Welcome to our community! 😃

Are those what you are trying to send to Elasticsearch?  
Is there an error when you are sending these you can share?

---

<div class="post-metadata">

### Author: ![xyu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xyu/32/111725_2.png) [@xyu](https://discuss.elastic.co/u/xyu)
#### Post date: [October 6, 2022, 2:49am UTC](https://discuss.elastic.co/t/data-loss-when-sending-logs-to-kibana-through-filebeat/315926/3 "2022-10-06T02:49:58Z")

</div>

No, there is no error message but when I check the number of hits in data view, it does not vary with the number of lines I have in a log

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 6, 2022, 3:02am UTC](https://discuss.elastic.co/t/data-loss-when-sending-logs-to-kibana-through-filebeat/315926/4 "2022-10-06T03:02:01Z")

</div>

You will need to share more information then please. Things like configs, logs, how are you comparing the number of logs to what is in Elasticsearch.

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [October 6, 2022, 11:23am UTC](https://discuss.elastic.co/t/data-loss-when-sending-logs-to-kibana-through-filebeat/315926/5 "2022-10-06T11:23:44Z")

</div>

> {"code":"SELF\_SIGNED\_CERT\_IN\_CHAIN","level":"error","message":"self signed certificate","timestamp":"2022-10-05 08:40:39.908"}

Most likely you have selfsigned certs. Try with this if you are sending directly to Elasticsearch:

`output.elasticsearch.verification_mode: "none"`

Default mode is full. Documentation is [here](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-ssl.html#server-verification-mode)

If is working then set certificate\_authorities on both side and set `full`

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 6, 2022, 12:31pm UTC](https://discuss.elastic.co/t/data-loss-when-sending-logs-to-kibana-through-filebeat/315926/6 "2022-10-06T12:31:24Z")

</div>

> [@Rios](#):
>
> Most likely you have selfsigned certs. Try with this if you are sending directly to Elasticsearch

This does not seem to be an error from filebeat, but the log message the OP is trying to send.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 6, 2022, 12:33pm UTC](https://discuss.elastic.co/t/data-loss-when-sending-logs-to-kibana-through-filebeat/315926/7 "2022-10-06T12:33:17Z")

</div>

Please share your `filebeat.yml` file and also check the mapping of your index, it may be a mapping issue.

In this message:

> {"code":"28000","file":"auth.c","length":164,"level":"error","line":"496","message":"no entry for host","name":"error","timestamp":"2022-10-05 08:40:14.308"}

You have some extra fields, maybe one of them have a different mapping.

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [October 6, 2022, 1:59pm UTC](https://discuss.elastic.co/t/data-loss-when-sending-logs-to-kibana-through-filebeat/315926/8 "2022-10-06T13:59:31Z")

</div>

> [@leandrojmp](#):
>
> This does not seem to be an error from filebeat, but the log message the OP is trying to send.

Might be. Well until we see filebeat.yml and logs, it's hard to know.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 3, 2022, 4:00pm UTC](https://discuss.elastic.co/t/data-loss-when-sending-logs-to-kibana-through-filebeat/315926/9 "2022-11-03T16:00:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
