Cannot list index created by fluentd

[mkain (Manish)]

Hi,

I am using fluentd HTTP adapter to receive json data and dump it in elasticsearch.
I can see packets are reaching elastic in n/w sniffer and there is no error in fluentd and elasticsearch log. But still I am unable to list the indices created by fluentd.

In tcpdump I can see following:

{"index":{"_index":"API-2019.09.03","_type":"fluentd"}}

{"url":"http://xx.xx.xx.xx:6800/","retCode":200,"resSize":8195,"resTime":0.031908,"@timestamp":"2019-09-03T16:29:34+05:30","tag":"API.nc.portal"}

{"index":{"_index":"API-2019.09.03","_type":"fluentd"}}

{"url":"http://xx.xx.xx.xx:6800/","retCode":200,"resSize":8195,"resTime":0.022068,"@timestamp":"2019-09-03T16:29:34+05:30","tag":"API.nc.portal"}

{"index":{"_index":"API-2019.09.03","_type":"fluentd"}}

Upon listing it does not shows any indices matching API-*

curl -X GET "localhost:9200/_cat/indices?v"

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

green open .kibana_1 FpjlQCtASg2B7rM9jMqd0w 1 0 24 1 88.4kb 88.4kb
yellow open logstash-2019.08.28 r9WOLRT3Q6qhj1EuXPiDVA 5 1 1807405 0 366.4mb 366.4mb
yellow open logstash-2019.09.02 U6DTOiPwQ_S89tfUt9GVHQ 5 1 1499866 0 319.2mb 319.2mb
yellow open logstash-2019.08.29 rLub-AfJQv-BVqGmbch4NA 5 1 1785888 0 371.6mb 371.6mb
yellow open apm-6.7.0-onboarding-2019.08.26 kP1aOy5_QPmcF4tP2Kv1Mw 1 1 3 0 17.4kb 17.4kb
yellow open logstash-2019.08.30 RleDf0dRSVmPb07imf_sHw 5 1 1492161 0 310.1mb 310.1mb
green open .kibana_task_manager odbrnDVfQWuIg_G6K3p6QQ 1 0 2 0 6.8kb 6.8kb
yellow open logstash-2019.09.01 wDXdMqiESeW_oXKsB77WCA 5 1 1482167 0 318.5mb 318.5mb
yellow open logstash-2019.09.03 qgPKKEpdQaC3WiEoQ3UrRg 5 1 522467 0 111.9mb 111.9mb
yellow open logstash-2019.08.31 5_K8cEQySw6NJPT5gtvmPg 5 1 1377857 0 291.4mb 291.4mb
yellow open logstash-2019.08.27 -HjSsnC7Q_WA6I51m4o-6Q 5 1 1696875 0 348mb 348mb

Any idea what am I missing ? I am using elasticsearch 6.7

Regards,
-Manish

[mkain (Manish)]

Fixed, I was using fluentd 3.0, which does not supported elasticsearch 6.x and 7.x.
After upgrading fluentd to 3.5, I can see fluentd created elasticsearch index.