# Logstash rds filtering?

**URL:** <https://discuss.elastic.co/t/logstash-rds-filtering/142655>\
**Category:** Logstash\
**Created:** [August 1, 2018, 10:14pm UTC](https://discuss.elastic.co/t/logstash-rds-filtering/142655 "2018-08-01T22:14:25Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jspriesterbach](https://avatars.discourse-cdn.com/v4/letter/j/db5fbb/32.png) [@jspriesterbach](https://discuss.elastic.co/u/jspriesterbach)\
**Post date:** [August 1, 2018, 10:14pm UTC](https://discuss.elastic.co/t/logstash-rds-filtering/142655/1 "2018-08-01T22:14:26Z")

</div>

Hello,

I currently have a script that will download all the error & slow logs from RDS for the day and put them in a folder. I am now trying to get logstash to read those files and put them into our ELK stack for use with kibana. Does anyone have an example filter that would work for RDS/mysql logs? at the moment I was just trying a simple:

filter {  
if [type] == "rds\_logs" {  
grok {  
match =\> { "message" =\> "%{RDS\_LOGS}"}  
}  
date {  
locale =\> "en"  
match =\> ["timestamp", "d/MMM/YYYY:HH:mm:ss Z"]  
}  
}  
}

But of course %RDS\_LOGS is not defined because there seems to be no grok filters built into logstash for rds. Heres an example log:

$ cat mysql-error-running.log.2018-07-29.01  
2018-07-29 00:45:38 2b7ebf3cf400 InnoDB: Warning: Setting innodb\_checksums to OFF is DEPRECATED. This option may be removed in future releases. You should set innodb\_checksum\_algorithm=NONE instead.  
2018-07-29 00:45:38 8513 [Note] InnoDB: Started in read only mode  
2018-07-29 00:45:38 8513 [Note] InnoDB: The InnoDB memory heap is disabled  
2018-07-29 00:45:38 8513 [Note] InnoDB: Mutexes and rw\_locks use GCC atomic builtins  
2018-07-29 00:45:38 8513 [Note] InnoDB: Compressed tables use zlib 1.2.3  
2018-07-29 00:45:38 8513 [Note] InnoDB: CPU does not support crc32 instructions

Thanks

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 1, 2018, 11:01pm UTC](https://discuss.elastic.co/t/logstash-rds-filtering/142655/2 "2018-08-01T23:01:17Z")

</div>

A quick web search returned [https://www.phase2technology.com/blog/adding-mysql-slow-query-logs-logstash](https://www.phase2technology.com/blog/adding-mysql-slow-query-logs-logstash)

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 1, 2018, 11:09pm UTC](https://discuss.elastic.co/t/logstash-rds-filtering/142655/3 "2018-08-01T23:09:06Z")

</div>

For the error logs something like this should get you started

```
    grok { 
        match => { 
            "message" => [ 
                 "%{DATE:date} %{TIME:time} %{NUMBER:randomNumber} \[%{WORD:loglevel}\] InnoDB: %{GREEDYDATA:logtext}",
                 "%{DATE:date} %{TIME:time} %{BASE16NUM:randomHexString} InnoDB: %{GREEDYDATA:logtext}"
            ]
        }
    }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 29, 2018, 11:09pm UTC](https://discuss.elastic.co/t/logstash-rds-filtering/142655/4 "2018-08-29T23:09:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.