Kibana Index Patterns not shown and cannot create a new one

[Hung_M_Le]

Hi

i am configuring in the filebeat.tml file the following setting. I can see that the new template was created and shown inside Kibana -> elasticsearch -> Index Management (I uploaded the picture for your reference) after running the following command :

% filebeat setup --index-management

The issue that I am facing is that I cannot create a new index pattern under Kibana -> index Patterns (create index pattern).

Please let me know what I am missing. I have collected the filebeat log. Please let me know where I can send this log to. Thanks

BR/

Hung Le
hungl@telenav.com

filebeat.inputs:

• type: log

  HUNG- Paths that should be crawled and fetched. Glob based paths.

  paths:

  • /Users/hungl/TEST_LOG/today_1.log
    #- /var/log/*.log
    #- c:\programdata\elasticsearch\logs*

#==================== Elasticsearch template setting ==========================

setup.template.enabled: true

setup.template.name: "hungle"

setup.template.pattern: "hungle-*"

setup.template.overwrite: true

setup.ilm.enabled: false
setup.ilm.enabled: auto
setup.ilm.rollover_alias: "hungle"
setup.ilm.pattern: "{now/d{MM.dd.yyyy|America/New_York}}-000001"
setup.ilm.policy_name: "HungLeLogPolicy"
setup.ilm.overwrite: true
setup.template.settings:
index.number_of_shards: 3

output.elasticsearch:

Array of hosts to connect to.

hosts: ["localhost:9200"]
index: "hungle-%{[agent.version]}-%{+yyyy.MM.dd}"

[Hung_M_Le]

Hi

I fixed the issue. The problem is related to the data. To trigger the data collection associated to the index in question, you need to append new data at the end of the file and not at the beginning. It seems that the filebeat will only monitor the file change/data change when data is appended at the end.

[Hung_M_Le]

On the same subject, I would like to know about creating the custom index, do you always have to run the following command at the command line:

filebeat setup --index-management

Is there another configure the filebeat to perform this task automatically?

Thank you in advance for you prompt response.