Create Index with filebeat

Elastic Stack Beats
1

Hello ,

I want to create a index and push data directly to elasticsearch with filebeat but i am not able to create a index its showing me the error can someone help me out.

Exiting: setup.template.name and setup.template.pattern have to be set if index name is modified

2

Can you write the code of "filbeat" that you are using?
Can you add the logs of filebeat with the erorr?
It can help us to identify the error.
thanks.

3

Hello @carmezsa my filebeat config is

filebeat.inputs:

  • type: log
    paths:
    • /home/MO*
      multiline.pattern: '^[0-9]{2}/[0-9]{2}/[0-9]{4}'
      multiline.negate: true
      multiline.match: after
      output.elasticsearch:
      hosts: ["localhost:9200"]
      index: "test1"

and my error is

Exiting: setup.template.name and setup.template.pattern have to be set if index name is modified

4

Hi @shrikantgulia

As written in the error message you need to set the same name for the setup.template.name and name, something like:

filebeat.inputs:

   - type: log
   paths:  /home/MO*
   ....
   index: "test1-%{+yyyy.MM.dd}"

setup.template.pattern: "test1"
setup.template.name: "test1-*"

This will fix the error as I had the same problem earlier but for me the index name is still filebeat-xxxx the index name don't change.

More information in the doc:

To use a different name, you set the index option in the Elasticsearch output. The value that you specify should include the root name of the index plus version and date information. You also need to configure the setup.template.name and setup.template.pattern options to match the new name.

Also better to use a daily index it cost nothing and prevent to forget when you will move to production. :grinning:

Tested on 7.5.1

1 Like
5

Hello @gabriel_tessier,

I dont want to use the default Index
So what should i do?
When i do setup.template.pattern: "test1"
setup.template.name: "test1-*"

it is showing error that no template with test1 present
because i didnt create template.

Q is it compulsory to create a template?

cant it be possible without creating template

6

@shrikantgulia

I didn't create any template and setting setup.template.pattern and name prevent the error.
But I'm still with the default index like you.

I didn't read that you need to create template in the documentation.

7

Hello @gabriel_tessier

my config file is Capture11

still i am getting my data in filebeat default index
Can someone help me please

8

Hi @shrikantgulia

You have an indentation problem you need to put setup.template at the beggining of the line not under output elastic.

please see my first post and the link to the documentation. As you poted a picture I can't edit.

Just remove the space before setup.template.pattern and setup.template.name

9

@gabriel_tessier

Can you Please post the config file here if possible
please

10

This is the new config file which i tried
still not getting the expected result

Capture12

11

Can someone Please help me with it

12

Can someone help me out with it

13

I am also facing the same issue (unable to change index name) and the doc doesn't help me enough clarifying the good use of setup.template.* options in the filebeat config file.
I'm finally wondering if what we try to do is the right thing.
Maybe some best practices for dealing with filebeat indexes would help us?

Sorry don't mean to hijack your topic, but I think it is related and I don't want to see it closed.

14

@shrikantgulia I guess you are experiencing the same issue as this one: https://github.com/elastic/beats/issues/11866

With ILM (Index Lifecycle Management) enabled, output.elasticsearch.index option will be ignored. And ILM is activated by default with clusters that support this feature.

1 Like
15

Not Woking
Can some one Please guide me over this.

16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.