Issue in displaying numeric data in kibana .
Client side Fluentd configuration
type tail tag underutilized.log path /var/log/utilization/monthly_summary.log format /^,HOST,(?[^,]+),DATE,(?[^,]+),STREAK,(?[^,]+),CPU_DAY_UTIL,(?[^,]+)$/ pos_file /var/spool/fluentd/underutilized.log.position.txt
<match underutilized.**>
type copy
type file path /var/log/fluentd/underutilized.log compress gzip type forward buffer_path /var/spool/fluentd/underutilized.*.buffer buffer_type file flush_interval 30s send_timeout 30s
heartbeat_type tcp
heartbeat_interval 10m
<server>
name dstsrvr01
host dstsrvr01.abc.com
port 24224
</server>
[user1@srvr05 ~]$ tail -3 /var/log/utilization/monthly_summary.log
,HOST,srvr05,DATE,2018-10-10,STREAK,123,CPU_DAY_UTIL,0%
,HOST,srvr05,DATE,2018-10-11,STREAK,124,CPU_DAY_UTIL,0%
,HOST,srvr05,DATE,2018-10-12,STREAK,125,CPU_DAY_UTIL,0%
Destination Server dstsrvr01 Kibana configuration
in /etc/td-agent/td-agent.conf
<match underutilized.**>
type elasticsearch
logstash_format true
flush_interval 5s
logstash_prefix underutilized
We are able to see the generated indexes if we run curl in dstsrvr01
curl 'localhost:9200/_cat/indices?v' | grep underutilized
yellow open underutilized-2018.09.02 S9PURAzIT2-SSo6QuIPH_Q 5 1 1554 0 390.9kb 390.9kb
yellow open underutilized-2018.08.29 itVOHT7_Qg6B-RNSAY40Bw 5 1 1563 0 404.7kb 404.7kb
yellow open underutilized-2018.06.10 LD0rp2rvQ8yW4j4kZjjR7g 5 1 62 0 138.5kb 138.5kb
yellow open underutilized-2018.07.09 7GljpUOQQByol0gv-C5ZSg 5 1 2043 0 463.8kb 463.8kb
We are displaying these data through Kibana .We are particular on the STREAK value ,which we want to be in integer format . But here it is in STRING format bydefault
and not able to apply numerical filter (eg STREAK > 40).Is ther any option to convert a particular field to integer .
Client version: fluentd-0.10.31-1.el5
Server version: td-agent-2.3.6-0.el7.x86_64
kibana-5.6.9-1.x86_64