# How to determine bottleneck?

**URL:** <https://discuss.elastic.co/t/how-to-determine-bottleneck/148135>\
**Category:** Logstash\
**Created:** [September 11, 2018, 1:26pm UTC](https://discuss.elastic.co/t/how-to-determine-bottleneck/148135 "2018-09-11T13:26:37Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ancdaromidi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ancdaromidi/32/64824_2.png) [@ancdaromidi](https://discuss.elastic.co/u/ancdaromidi)\
**Post date:** [September 11, 2018, 1:26pm UTC](https://discuss.elastic.co/t/how-to-determine-bottleneck/148135/1 "2018-09-11T13:26:37Z")

</div>

I have current configuration for logstash

input {  
azureblob {  
storage\_account\_name =\> ""  
storage\_access\_key =\> ""  
container =\> ""  
registry\_path =\> ""  
codec =\> "json"  
interval =\> 20  
}  
}

filter {  
split { field =\> "[value]" }  
}

output {  
elasticsearch {  
hosts =\> [""]  
index =\> ""  
}  
}

I tried to run get rate of documents per 1m with the following configuration

input {  
azureblob {  
storage\_account\_name =\> ""  
storage\_access\_key =\> ""  
container =\> ""  
registry\_path =\> ""  
type =\> "generated"  
}  
}

```
filter {
  if [type] == "generated" {
    metrics {
      meter => "events"
      add_tag => "metric"
    }
  }
}

output {
  
  if "metric" in [tags] {
    stdout {
      codec => line {
        format => "rate: %{[events][rate_1m]}"
      }
    }
  }
}

```

This is the rate that I'm getting:  
rate: 0.04980578857764925  
rate: 0.04980578857764925  
rate: 0.045823537597075144  
rate: 0.042159689824745786  
rate: 0.038788787145762074  
rate: 0.038788787145762074  
rate: 0.03568740696370409  
rate: 0.03568740696370409  
rate: 0.030208737803509546  
rate: 0.030208737803509546  
rate: 0.025571144482683064  
rate: 0.02352658865697199  
rate: 0.02164550648912867

But with generator input plugin i get this:  
rate: 47137.93823682722  
rate: 48248.462291468255  
rate: 49477.902364928195  
rate: 50633.044504364756  
rate: 51357.98224743107  
rate: 52174.88988258032  
rate: 52953.85798199835  
rate: 53693.7463419169  
rate: 54395.856618437334  
rate: 54745.9935909323  
rate: 55282.01634769808  
rate: 55689.00496126213

With those results it seems to me that the issue is with the azure blob plugin. I also noticed that upon first usage of config file, it took around 30 minutes for the registry file to get created in the blob. Could this be an issue with connection of my logstash to azure? How can I check where the issue is coming from?

Some information of my logstash  
JVM STATS  
"jvm" : {  
"threads" : {  
"count" : 112,  
"peak\_count" : 113  
},  
"mem" : {  
"heap\_used\_percent" : 68,  
"heap\_committed\_in\_bytes" : 16979263488,  
"heap\_max\_in\_bytes" : 16979263488,  
"heap\_used\_in\_bytes" : 11617900784,  
"non\_heap\_used\_in\_bytes" : 161118392,  
"non\_heap\_committed\_in\_bytes" : 250851328,  
"pools" : {  
"survivor" : {  
"peak\_used\_in\_bytes" : 200605696,  
"used\_in\_bytes" : 200605696,  
"peak\_max\_in\_bytes" : 200605696,  
"max\_in\_bytes" : 200605696,  
"committed\_in\_bytes" : 200605696  
},  
"old" : {  
"peak\_used\_in\_bytes" : 11883056344,  
"used\_in\_bytes" : 10936568240,  
"peak\_max\_in\_bytes" : 15173353472,  
"max\_in\_bytes" : 15173353472,  
"committed\_in\_bytes" : 15173353472  
},  
"young" : {  
"peak\_used\_in\_bytes" : 1605304320,  
"used\_in\_bytes" : 480726848,  
"peak\_max\_in\_bytes" : 1605304320,  
"max\_in\_bytes" : 1605304320,  
"committed\_in\_bytes" : 1605304320  
}  
}  
},  
"gc" : {  
"collectors" : {  
"old" : {  
"collection\_time\_in\_millis" : 27531,  
"collection\_count" : 166  
},  
"young" : {  
"collection\_time\_in\_millis" : 3758202,  
"collection\_count" : 32540  
}  
}  
},  
"uptime\_in\_millis" : 170491217

PROCESS STATS  
"process" : {  
"open\_file\_descriptors" : 98,  
"peak\_open\_file\_descriptors" : 100,  
"max\_file\_descriptors" : 65536,  
"mem" : {  
"total\_virtual\_in\_bytes" : 31645831168  
},  
"cpu" : {  
"total\_in\_millis" : 174860860,  
"percent" : 3,  
"load\_average" : {  
"1m" : 0.52,  
"5m" : 0.63,  
"15m" : 0.67  
}

PIPELINE STATS  
"pipelines" : {  
"main" : {  
"workers" : 32,  
"batch\_size" : 125,  
"batch\_delay" : 50,  
"config\_reload\_automatic" : false,  
"config\_reload\_interval" : 3000000000,  
"dead\_letter\_queue\_enabled" : false  
}

Please let me know if you need more information.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 9, 2018, 1:26pm UTC](https://discuss.elastic.co/t/how-to-determine-bottleneck/148135/2 "2018-10-09T13:26:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.