# Unable to create custom index or Disable ILM **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 25, 2020, 9:25am UTC](https://discuss.elastic.co/t/unable-to-create-custom-index-or-disable-ilm/224964 "2020-03-25T09:25:39Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![rehannali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rehannali/32/57714_2.png) [@rehannali](https://discuss.elastic.co/u/rehannali)\ **Post date:** [March 25, 2020, 9:25am UTC](https://discuss.elastic.co/t/unable-to-create-custom-index-or-disable-ilm/224964/1 "2020-03-25T09:25:39Z")
Hi, I'm unable to create custom index from filebeat. I tried to disbale `ilm` and tried almost everything but i'm unable to disable it. > **configuration** > > ```conf > ###################### Filebeat Configuration Example ######################### > > # This file is an example configuration file highlighting only the most common > # options. The filebeat.reference.yml file from the same directory contains all the > # supported options with more comments. You can use it as a reference. > # > # You can find the full configuration reference here: > # https://www.elastic.co/guide/en/beats/filebeat/index.html > > # For more available modules and options, please see the filebeat.reference.yml sample > # configuration file. > > #=========================== Filebeat inputs ============================= > > filebeat.inputs: > > # Each - is an input. Most options can be set at the input level, so > # you can use different inputs for various configurations. > # Below are the input specific configurations. > > - type: log > enable: true > paths: > - /var/log/apache2/*.log > fields: > apache: true > log_type: apache2 > fields_under_root: true > > - type: log > > # Change to true to enable this input configuration. > enabled: true > > # Paths that should be crawled and fetched. Glob based paths. > paths: > - /var/log/oneload/web/*.log > #- c:\programdata\elasticsearch\logs\* > > # Exclude lines. A list of regular expressions to match. It drops the lines that are > # matching any regular expression from the list. > #exclude_lines: ['^DBG'] > > # Include lines. A list of regular expressions to match. It exports the lines that are > # matching any regular expression from the list. > #include_lines: ['^ERR', '^WARN'] > > # Exclude files. A list of regular expressions to match. Filebeat drops the files that > # are matching any regular expression from the list. By default, no files are dropped. > #exclude_files: ['.gz$'] > > # Optional additional fields. These fields can be freely picked > # to add additional information to the crawled log files for filtering > fields: > log_type: jboss > # level: debug > # review: 1 > > ### Multiline options > > # Multiline can be used for log messages spanning multiple lines. This is common > # for Java Stack Traces or C-Line Continuation > > # The regexp Pattern that has to be matched. The example pattern matches all lines starting with [ > #multiline.pattern: ^\[ > > # Defines if the pattern set under pattern should be negated or not. Default is false. > #multiline.negate: false > > # Match can be set to "after" or "before". It is used to define if lines should be append to a pattern > # that was (not) matched before or after or as long as a pattern is not matched based on negate. > # Note: After is the equivalent to previous and before is the equivalent to to next in Logstash > #multiline.match: after > > #============================= Filebeat modules =============================== > > filebeat.config.modules: > # Glob pattern for configuration loading > path: ${path.config}/modules.d/*.yml > > # Set to true to enable config reloading > reload.enabled: false > > # Period on which files under path should be checked for changes > #reload.period: 10s > > #==================== Elasticsearch template setting ========================== > setup.ilm.enable: false > > setup.template.enable: true > setup.template.name: "index-master40-%{[agent.version]}" > setup.template.pattern: "index-master40-%{[agent.version]}-*" > setup.template.settings: > index.number_of_shards: 1 > index.lifecycle.rollover_alias: "index-master40-%{[agent.version]}" > #index.codec: best_compression > #_source.enabled: false > > #================================ General ===================================== > > # The name of the shipper that publishes the network data. It can be used to group > # all the transactions sent by a single shipper in the web interface. > #name: > > # The tags of the shipper are included in their own field with each > # transaction published. > #tags: ["service-X", "web-tier"] > > # Optional fields that you can specify to add additional information to the > # output. > #fields: > # env: staging > > #================================ Outputs ===================================== > > # Configure what output to use when sending the data collected by the beat. > > #-------------------------- Elasticsearch output ------------------------------ > output.elasticsearch: > # Array of hosts to connect to. > hosts: ["10.100.101.41:9200"] > index: "index-master40-%{[agent.version]}-%{[fields.log_type]:other}-%{+yyyy-MM-dd}" > > # Protocol - either `http` (default) or `https`. > #protocol: "https" > > # Authentication credentials - either API key or username/password. > #api_key: "id:api_key" > username: "username" > password: "password" > > #================================ Processors ===================================== > > # Configure processors to enhance or manipulate events generated by the beat. > > processors: > - add_host_metadata: ~ > - add_cloud_metadata: ~ > - add_docker_metadata: ~ > - add_kubernetes_metadata: ~ > > ``` > These are logs output which says ILM enable and uses default pattern. Although it creates index template but never creates index. > **logs** > > ```log > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.960+0500 INFO [publisher] pipeline/retry.go:196 retryer: send unwait-signal to consumer > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.960+0500 INFO [publisher] pipeline/retry.go:198 done > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.960+0500 INFO [publisher] pipeline/retry.go:173 retryer: send wait signal to consumer > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.960+0500 INFO [publisher] pipeline/retry.go:175 done > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.961+0500 INFO elasticsearch/client.go:757 Attempting to connect to Elasticsearch version 7.4.2 > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.983+0500 INFO [license] licenser/es_callback.go:50 Elasticsearch license: Basic > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.983+0500 INFO [index-management] idxmgmt/std.go:258 Auto ILM enable success. > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.984+0500 INFO [index-management.ilm] ilm/std.go:139 do not generate ilm policy: exists=true, overwrite=false > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.984+0500 INFO [index-management] idxmgmt/std.go:271 ILM policy successfully loaded. > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.984+0500 INFO [index-management] idxmgmt/std.go:410 Set setup.template.name to '{filebeat-7.6.1 {now/d}-000001}' as ILM is enabled. > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.984+0500 INFO [index-management] idxmgmt/std.go:415 Set setup.template.pattern to 'filebeat-7.6.1-*' as ILM is enabled. > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.984+0500 INFO [index-management] idxmgmt/std.go:453 Set settings.index.lifecycle.name in template to {filebeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled. > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.985+0500 INFO template/load.go:89 Template filebeat-7.6.1 already exists and will not be overwritten. > Mar 25 14:17:01 oneload-app filebeat[72701]: 2020-03-25T14:17:01.985+0500 INFO [index-management] idxmgmt/std.go:295 Loaded index template. > > ``` > - I also want to create seperate monitoring index and for custom logs i want custom index which you can see in paths. > > - There is also i need to know if i use custom logs do i need to create manuall template for index or elasticsearch will automatically identify using key:value pair? I'm using filebeat 7.6.1. ---
**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\ **Post date:** [March 27, 2020, 1:00am UTC](https://discuss.elastic.co/t/unable-to-create-custom-index-or-disable-ilm/224964/2 "2020-03-27T01:00:04Z")
Hi, thanks for posting your Filebeat configuration. And extra thanks for formatting it neatly and encapsulating it inside`[details]` tags. Not all forum users go this extra mile and we appreciate it as it really helps readability of code/configuration snippets! ❤ Now, back to your question. Looking at the settings in your configuration file I see these lines: ```auto setup.ilm.enable: false setup.template.enable: true ``` These settings are actually named `setup.ilm.enabled` and `setup.template.enabled` (notice the extra `d` at the end of the setting names). Also, you don't really need to set `setup.template.enabled: true` as that is the default setting value anyway. Please try with the fixed setting names and let us know how it goes! Thanks, Shaunak ---
**Author:** ![rehannali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rehannali/32/57714_2.png) [@rehannali](https://discuss.elastic.co/u/rehannali)\ **Post date:** [March 27, 2020, 5:36am UTC](https://discuss.elastic.co/t/unable-to-create-custom-index-or-disable-ilm/224964/3 "2020-03-27T05:36:08Z")
@shaunak Hi, Thanks for your reponse as well as appreciating ❤. Yes i wrote it wrong and it worked after changing `setup.ilm.enable` to `setup.ilm.enabled`. > I have another question, Is it okay to send logs directly from filebeat or i need to process it though logstash? I quite didn't understand what to do about this query. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 24, 2020, 5:36am UTC](https://discuss.elastic.co/t/unable-to-create-custom-index-or-disable-ilm/224964/4 "2020-04-24T05:36:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.