# Logstash Servers are runs with high CPU consumption

**URL:** <https://discuss.elastic.co/t/logstash-servers-are-runs-with-high-cpu-consumption/198952>\
**Category:** Logstash\
**Created:** [September 10, 2019, 6:00pm UTC](https://discuss.elastic.co/t/logstash-servers-are-runs-with-high-cpu-consumption/198952 "2019-09-10T18:00:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![vsubrama](https://avatars.discourse-cdn.com/v4/letter/v/e9c0ed/32.png) [@vsubrama](https://discuss.elastic.co/u/vsubrama)\
**Post date:** [September 10, 2019, 6:00pm UTC](https://discuss.elastic.co/t/logstash-servers-are-runs-with-high-cpu-consumption/198952/1 "2019-09-10T18:00:52Z")

</div>

We are using 6.7.1 of Elastic stack. Our Logstash is running at high CPU (90+%) always. What do need need to look for to understand the high CPU consumption. We are using AWS machine M5.large for Logstash. Here is one of Logstash pipeline scripe we use:

input{  
s3 {  
bucket =\> "xxxx-diff-events-local"  
exclude\_pattern =\> "^((?!-diff-events-spark-streaming-).)\*$"  
prefix =\> "deployment/xxx/application/logs/"  
region =\> "us-west-2"  
codec =\> "json"  
tags =\> ["xxxx"]  
}  
}  
filter{  
if ! [message][action] {  
mutate {  
add\_field =\> { "[msg][description]" =\> "%{message}"}  
}  
mutate {  
rename =\> { "msg" =\> "message"}  
}  
}  
mutate {  
add\_field =\> {  
"filepath" =\> "%{[@metadata][s3][key]}"  
}  
}  
fingerprint {  
source =\> "message"  
target =\> "[@metadata][fingerprint]"  
method =\> "MURMUR3"  
}

```
mutate { add_field => { "[@metadata][milli]" => "%{[instant][nanoOfSecond]}" } }
truncate {
        fields => "[@metadata][milli]"
        length_bytes => 3
}
mutate {
        add_field => { "[@metadata][transactiontime]" => "%{[instant][epochSecond]}%{[@metadata][milli]}" }
}
date {
        match => ["[@metadata][transactiontime]", "UNIX_MS"]
        target => "@timestamp"
}
mutate { add_field => { "[@metadata][lkt_date]" => "%{year}.%{month}" } }
mutate { rename => { "message" => "context" } }
mutate {
    rename =>{
        "[context][action]" => "[context][workflow]"
        "[context][entity_guid]" => "[context][ent_guid]"
        "[context][partition]" => "[work][partition]"
        "[context][offset]" => "[work][seq_num]"
        "[contextMap][table_name]" => "[work][source]"
        "[context][app_query_response]" => "[target][status]"
        "[context][description]" => "[message]"
        "[envtag]" => "[service][envtag]"
        "[level]" => "[log][level]"
        "[loggerName]" => "[log][name]"
        "[threadId]" => "[log][thread][id]"
        "[thread]" => "[log][thread][name]"
    }
}
mutate{
    remove_field => ["instant", "month", "year", "monthday", "loggerFqcn", "threadPriority"]
    add_field =>{
        "[service][name]" => "diffevent-spark"
    }
}

```

}

output {  
file {  
path =\> "/var/lib/logstash/debug.out"  
}  
elasticsearch {  
hosts =\> ["[https://xxx:19200](https://xxx:19200)"]  
index =\> "diffevent-current-write"  
user =\> "xxx"  
password =\> "xxxxxxx"  
manage\_template =\> false  
}  
}

High CPU worries me. Is there anything wrong with the script above, Machine we are using etc. Any help with this will be greatly appreciated.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 10, 2019, 7:18pm UTC](https://discuss.elastic.co/t/logstash-servers-are-runs-with-high-cpu-consumption/198952/2 "2019-09-10T19:18:30Z")

</div>

I suggest you look at the [hot threads](https://www.elastic.co/guide/en/logstash/current/hot-threads-api.html) monitoring API.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 8, 2019, 7:18pm UTC](https://discuss.elastic.co/t/logstash-servers-are-runs-with-high-cpu-consumption/198952/3 "2019-10-08T19:18:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.