Why i am not ablle to select my available fields in Percentiles Aggregation Field option in kibana visualization

[Pacha_Gopi]

Hi all,

I am using latest version of kibana and my task is visualize the API responce time (upstream_responce_time in nginx) in kibana dash board on the values of p50,p90,p99.when i select Percentiles Aggregation opition in the kibana visualization i am not getting the all avaialblle fields that i defined i am getting on two options loke mentioned below.

date
@timestamp

number
log.offset

My question is why i am not able to view different fields otherthan timestamp and logset. i have upstream_responce_time and request time and more,i am attaching both the screenshots of available fields and visualization can anyonne help me out of this isuue.

[flash1293 (Joe Reuter)]

Please check the mapping of your index and make sure upstram_response_time is indexed as a number field. I suspect it's currently mapped as string field (and you can't calculate percentiles on a string)

[Pacha_Gopi]

Hi thanks for responce i will check and update you

[Pacha_Gopi]

Hi i am sending logs from filebeat to my logstash and i am using kv filter pugin in filter section it is working fine but when i use mutate filter pugin to convert from string to integer my logstash is not working and i am getting the error below code is my filter configuration.

filter {

kv {

field_split => " "
value_split => "="
}
 mutate {
convert => { "upstream_response_time" => "integer" }

}

}

can you guide how can i use mutate plugin along with kv plugin.
Note: KV filter plugin is mandatory for my project

[Pacha_Gopi]

These are My logstash error logs

[2020-04-14T12:18:08,215][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.6.2"}
[2020-04-14T12:18:10,912][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \t\r\n], "#", "{" at line 11, column 5 (byte 65) after filter {I\n \n ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:41:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:in compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in block in compile_sources'", "org/jruby/RubyArray.java:2580:in map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:161:in initialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:27:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:326:in block in converge_state'"]}
[2020-04-14T12:18:11,560][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2020-04-14T12:18:16,335][INFO ][logstash.runner ] Logstash shut down.

[Pacha_Gopi]

Hi I converted the above upstream_response_time to number type from string but after this also i am facing same issue as above mentioned i am getting only logset in number section.

[flash1293 (Joe Reuter)]

Did you refresh your index pattern in the management index pattern section? Otherwise Kibana won't pick up the change. Before you do that make sure that all indices matching the pattern only have a number type for this field in their mapping, no string leftovers in old indices (you either have to delete the old indicies, re-index them with the new mapping or make sure your index pattern is not matching them)

[Pacha_Gopi]

Thank you so much problem got solved