# A New TiDB Module for Filebeat

**URL:** <https://discuss.elastic.co/t/a-new-tidb-module-for-filebeat/283439>\
**Category:** Beats\
**Tags:** beats-module, filebeat\
**Created:** [September 6, 2021, 11:12am UTC](https://discuss.elastic.co/t/a-new-tidb-module-for-filebeat/283439 "2021-09-06T11:12:10Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![yifan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yifan/32/94201_2.png) [@yifan](https://discuss.elastic.co/u/yifan)\
**Post date:** [September 6, 2021, 11:12am UTC](https://discuss.elastic.co/t/a-new-tidb-module-for-filebeat/283439/1 "2021-09-06T11:12:10Z")

</div>

Hello everyone,

I'm a developer from TiDB Community.

[TiDB](https://github.com/pingcap/tidb) is a popular open source HTAP database. TiDB users use tools such as TiUP and TiDB-operator to deploy TiDB clusters in a bare-metal or k8s environment.

Many TiDB users use ELK stack to collect TiDB cluster logs. Due to the complexity of TiDB cluster components and log configuration, it is also complicated for users to configure ELK.

To reduce the complexity of using ELK, I have developed a TiDB module.

[[Filebeat] A New TiDB Module for Filebeat by SabaPing · Pull Request #27663 · elastic/beats (github.com)](https://github.com/elastic/beats/pull/27663)

This module has 5 filesets. ["tidb", "pd", "tikv", "tiflash", "slowlog"].  
Four of them map to different components in a TiDB cluster:

- TiDB
- PD
- TiKV
- TiFlash

And the last one maps to slow logs generated by the TiDB component.

As to log parsing, it follows two specifications:

- Normal logs: [https://github.com/tikv/rfcs/blob/master/text/0018-unified-log-format.md](https://github.com/tikv/rfcs/blob/master/text/0018-unified-log-format.md)
- Slow logs: [https://docs.pingcap.com/tidb/stable/identify-slow-queries](https://docs.pingcap.com/tidb/stable/identify-slow-queries)

After parsing, users should be able to search log text and analyze slow logs in elasticsearch.

Pinging @filebeat-module-team. Could you help me take a look at this PR?

Thanks a lot! 🥳

---

<div class="post-metadata">

**Author:** ![yifan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yifan/32/94201_2.png) [@yifan](https://discuss.elastic.co/u/yifan)\
**Post date:** [September 16, 2021, 4:41am UTC](https://discuss.elastic.co/t/a-new-tidb-module-for-filebeat/283439/2 "2021-09-16T04:41:08Z")

</div>

Can someone please help me review my [PR](https://github.com/elastic/beats/pull/27663)?  
Out customers, who use Elasticsearch to search TiDB runtime logs, really want this feature 😍.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 14, 2021, 6:41am UTC](https://discuss.elastic.co/t/a-new-tidb-module-for-filebeat/283439/3 "2021-10-14T06:41:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.