.*?)"]
}
}
```
and output is this:
```
output {
if [tags] {
elasticsearch {
hosts => ["localhost"]
index => "%{tags}-%{+YYYY.MM.dd}"
}
}
}
```
Leak somewhere? Garbage collection time constantly increasing (jstat)
```
S0 S1 E O P YGC YGCT FGC FGCT GCT LGCC GCC
0.00 52.87 19.08 13.50 59.96 57919 2977.900 4916 206.461 3184.361 Allocation Failure No GC
0.00 52.87 84.52 13.50 59.96 57919 2977.900 4916 206.461 3184.361 Allocation Failure No GC
72.04 0.00 60.68 13.50 59.96 57920 2977.954 4916 206.461 3184.415 Allocation Failure No GC
0.00 65.02 64.85 13.51 59.96 57921 2978.009 4916 206.461 3184.471 Allocation Failure No GC
58.54 0.00 40.52 13.51 59.96 57922 2978.058 4916 206.461 3184.520 Allocation Failure No GC
58.54 0.00 98.01 13.51 59.96 57922 2978.058 4916 206.461 3184.520 Allocation Failure No GC
0.00 58.06 99.74 13.51 59.96 57924 2978.107 4916 206.461 3184.569 Allocation Failure
```
jstack for the process:
> **[Dropbox - Link not found](https://www.dropbox.com/s/u6tz0w4v0in8lev/logstash.jstack?dl=0)**
>
> Dropbox is a free service that lets you bring your photos, docs, and videos anywhere and share them easily. Never email yourself a file again!
---
### Author:  [@m0tek](https://discuss.elastic.co/u/m0tek)
#### Post date: [May 7, 2016, 9:44pm UTC](https://discuss.elastic.co/t/logstash-consuming-huge-amount-of-cpu-mem-leak-garbage-collector/49310/2 "2016-05-07T21:44:48Z")
Are you seeing any errors in the Logstash.log file ?
---
### Author:  [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)
#### Post date: [May 9, 2016, 6:52pm UTC](https://discuss.elastic.co/t/logstash-consuming-huge-amount-of-cpu-mem-leak-garbage-collector/49310/3 "2016-05-09T18:52:40Z")
Try using `kafka-consumer-offset-checker.sh` to see if the Kafka topics are being consumed while it is spinning.
You could also try an else output stdout to if anything isn't getting the tags field.
---
### Author:  [@Eduard\_Camaj](https://discuss.elastic.co/u/Eduard_Camaj)
#### Post date: [May 11, 2016, 11:45am UTC](https://discuss.elastic.co/t/logstash-consuming-huge-amount-of-cpu-mem-leak-garbage-collector/49310/4 "2016-05-11T11:45:02Z")
No, there is only errors connected with unable to parse some lines, like:
... must not contain the following characters [\, /, \*, ?, ", \<, \>, |, , ,] and similar, nothing "serious".
---
### Author:  [@Eduard\_Camaj](https://discuss.elastic.co/u/Eduard_Camaj)
#### Post date: [May 11, 2016, 11:46am UTC](https://discuss.elastic.co/t/logstash-consuming-huge-amount-of-cpu-mem-leak-garbage-collector/49310/5 "2016-05-11T11:46:53Z")
Yes, topics are being consumed and offset is set correctly. If I stop ELK and restart, it will continue where it left.
What do you mean about anything not getting tags field? What would be the case? And consequence?
---
### Author:  [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)
#### Post date: [May 11, 2016, 5:10pm UTC](https://discuss.elastic.co/t/logstash-consuming-huge-amount-of-cpu-mem-leak-garbage-collector/49310/6 "2016-05-11T17:10:37Z")
Well if the "if [tags]" is false then the message would effectively be dropped. Since your offsets are moving but nothing is in Elasticsearch I'd suspect that something is up with the logic. That would explain the spike in CPU as Kafka is probably slamming those messages through very quickly as it is basically outputting to /dev/null. Otherwise, perhaps it only manifests over time.
Are you able to provide your config is full? Only having pieces can sometimes mask an issue. Apologies if you already provided it all.
---
### Author:  [@Eduard\_Camaj](https://discuss.elastic.co/u/Eduard_Camaj)
#### Post date: [May 11, 2016, 6:50pm UTC](https://discuss.elastic.co/t/logstash-consuming-huge-amount-of-cpu-mem-leak-garbage-collector/49310/7 "2016-05-11T18:50:11Z")
Sorry if I wasn't completely clear.
The data is propagated to elasticsearch correctly. Logstash process is getting more and more CPU consumption until one point, and THEN is when log stash stops propagating data to elasticsearch.
That was the reason I included all that JVM stats.
I will provide full config later, not at home at the moment.
Thanks,
Eddie
---
### Author:  [@Eduard\_Camaj](https://discuss.elastic.co/u/Eduard_Camaj)
#### Post date: [May 17, 2016, 9:58am UTC](https://discuss.elastic.co/t/logstash-consuming-huge-amount-of-cpu-mem-leak-garbage-collector/49310/8 "2016-05-17T09:58:46Z")
Sadly, but it seems that regex is the one consuming too much CPU. When I removed regex and used CSV filter with columns option (which is basically what I wanted to do in the first place), CPU is now constantly around 150% (which also seems too high for 200 transactions/s). But hey, it is what it is currently.
What stays mistery is why it works for a while and then simply stops. Maybe to put more workers...
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:57am UTC](https://discuss.elastic.co/t/logstash-consuming-huge-amount-of-cpu-mem-leak-garbage-collector/49310/9 "2017-07-06T04:57:27Z")