# How to set ElasticSearch Apm Server indexing template as YYYY.mm?

**URL:** <https://discuss.elastic.co/t/how-to-set-elasticsearch-apm-server-indexing-template-as-yyyy-mm/177811>\
**Category:** APM\
**Created:** [April 22, 2019, 7:48am UTC](https://discuss.elastic.co/t/how-to-set-elasticsearch-apm-server-indexing-template-as-yyyy-mm/177811 "2019-04-22T07:48:42Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yusuf\_Karatoprak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yusuf_karatoprak/32/44539_2.png) [@Yusuf\_Karatoprak](https://discuss.elastic.co/u/Yusuf_Karatoprak)\
**Post date:** [April 22, 2019, 7:48am UTC](https://discuss.elastic.co/t/how-to-set-elasticsearch-apm-server-indexing-template-as-yyyy-mm/177811/1 "2019-04-22T07:48:43Z")

</div>

I just created Apm server it looks awesome but Indexing like that"Apm-6.6.2-2019.04.19" doesn't make sense because I will have at the end of the month 30 logs. I want to use change "apm-%{[beat.version]}-%{+yyyy.MM}" instead of "apm-%{[beat.version]}-%{+yyyy.MM.dd}" Below code is my purpose but it's not working:

Kibana,Elastic.Apm : 6.6.2

```auto
apm-server:
  host: mycompany.name.com:8200

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["http://mycompany.name.com:9200"]
  index: "apm-%{[beat.version]}-%{+yyyy.MM.dd}"
  indices:
    - index: "critical-%{[beat.version]}-%{+yyyy.MM.dd}"
      when.contains:
        message: "CRITICAL"
    - index: "error-%{[beat.version]}-%{+yyyy.MM.dd}"
      when.contains:
        message: "ERR"

  timeout: 180
  bulk_max_size: 2

setup.template.name: "apm"
setup.template.pattern: "apm-*"

```

---

<div class="post-metadata">

**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\
**Post date:** [April 22, 2019, 8:30am UTC](https://discuss.elastic.co/t/how-to-set-elasticsearch-apm-server-indexing-template-as-yyyy-mm/177811/2 "2019-04-22T08:30:10Z")

</div>

Your configuration looks for a field called `message`, which is irrelevant for the APM documents.

Try to uncomment and modify the existing `output.elasticsearch` configuration on your `apm-server.yml` file accordingly, which is based on the `processor.event` field. Please read all comments around these configurations to make sure you don't break anything.

---

<div class="post-metadata">

**Author:** ![Yusuf\_Karatoprak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yusuf_karatoprak/32/44539_2.png) [@Yusuf\_Karatoprak](https://discuss.elastic.co/u/Yusuf_Karatoprak)\
**Post date:** [April 23, 2019, 6:10am UTC](https://discuss.elastic.co/t/how-to-set-elasticsearch-apm-server-indexing-template-as-yyyy-mm/177811/3 "2019-04-23T06:10:29Z")

</div>

Hi @Eyal_Koren Can you give me an example? Actually I don't understand What you mean? Thank you

---

<div class="post-metadata">

**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\
**Post date:** [April 23, 2019, 6:32am UTC](https://discuss.elastic.co/t/how-to-set-elasticsearch-apm-server-indexing-template-as-yyyy-mm/177811/4 "2019-04-23T06:32:15Z")

</div>

This is from `apm-server.yml`:

```auto
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  # Scheme and port can be left out and will be set to the default (http and 9200)
  # In case you specify and additional path, the scheme is required: http://localhost:9200/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:9200
  hosts: ["localhost:9200"]

  # Boolean flag to enable or disable the output module.
  #enabled: true

  # Set gzip compression level.
  #compression_level: 0

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"

  # Dictionary of HTTP parameters to pass within the url with index operations.
  #parameters:
    #param1: value1
    #param2: value2

  # Number of workers per Elasticsearch host.
  #worker: 1

  # By using the configuration below, apm documents are stored to separate indices,
  # depending on their `processor.event`:
  # - error
  # - transaction
  # - span
  # - sourcemap
  #
  # The indices are all prefixed with `apm-%{[observer.version]}`.
  # To allow managing indices based on their age, all indices (except for sourcemaps)
  # end with the information of the day they got indexed.
  # e.g. "apm-6.3.0-transaction-2018.03.20"
  #
  # Be aware that you can only specify one Elasticsearch template.
  # In case you modify the index patterns you must also update those configurations accordingly,
  # as they need to be aligned:
  # * `setup.template.name`
  # * `setup.template.pattern`
  #index: "apm-%{[observer.version]}-%{+yyyy.MM.dd}"
  #indices:
  # - index: "apm-%{[observer.version]}-sourcemap"
  # when.contains:
  # processor.event: "sourcemap"
  #
  # - index: "apm-%{[observer.version]}-error-%{+yyyy.MM.dd}"
  # when.contains:
  # processor.event: "error"
  #
  # - index: "apm-%{[observer.version]}-transaction-%{+yyyy.MM.dd}"
  # when.contains:
  # processor.event: "transaction"
  #
  # - index: "apm-%{[observer.version]}-span-%{+yyyy.MM.dd}"
  # when.contains:
  # processor.event: "span"
  #
  # - index: "apm-%{[observer.version]}-metric-%{+yyyy.MM.dd}"
  # when.contains:
  # processor.event: "metric"
  #
  # - index: "apm-%{[observer.version]}-onboarding-%{+yyyy.MM.dd}"
  # when.contains:
  # processor.event: "onboarding"
...

```

Try basing on this.  
I hope this helps.  
Eyal.

---

<div class="post-metadata">

**Author:** ![Yusuf\_Karatoprak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yusuf_karatoprak/32/44539_2.png) [@Yusuf\_Karatoprak](https://discuss.elastic.co/u/Yusuf_Karatoprak)\
**Post date:** [April 23, 2019, 6:42am UTC](https://discuss.elastic.co/t/how-to-set-elasticsearch-apm-server-indexing-template-as-yyyy-mm/177811/5 "2019-04-23T06:42:24Z")

</div>

Hi @Eyal_Koren. Maybe You misunderstand me.Doesn't matter. You are really helpful and brilliant Thank you. But My index is YYYY.mm.dddd format it causes I will have apm.2019.04.23,apm.2019.04.24, apm.2019.04.25, apm.2019.04.26 .......

So it makes my indexes not useful for tracking anything down. Is it answering for my question? Where is your YYYY.mm format for that?

Thank you again for your great help!

---

<div class="post-metadata">

**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\
**Post date:** [April 23, 2019, 6:49am UTC](https://discuss.elastic.co/t/how-to-set-elasticsearch-apm-server-indexing-template-as-yyyy-mm/177811/6 "2019-04-23T06:49:32Z")

</div>

Yusuf,

Please see in the enclosed `output.elasticsearch` configuration. It has an `indices` section (currently commented out) that contains the indices pattern definition. You can see its `when.contains` is looking for the `processor.event` field, which means indices will be created per processor event type (transaction, span etc.). Try relying on this with the date-format change and see if it does what you want, otherwise try playing with it.

Cheers,  
Eyal.

---

<div class="post-metadata">

**Author:** ![simitt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simitt/32/106406_2.png) [@simitt](https://discuss.elastic.co/u/simitt)\
**Post date:** [April 23, 2019, 8:59am UTC](https://discuss.elastic.co/t/how-to-set-elasticsearch-apm-server-indexing-template-as-yyyy-mm/177811/7 "2019-04-23T08:59:18Z")

</div>

If you only want to get rid of the `dd` format, you can remove it from the config, as Eyal described above, eg.

```auto
  index: "apm-%{[observer.version]}-%{+yyyy.MM}"
  indices:
    - index: "apm-%{[observer.version]}-sourcemap"
      when.contains:
        processor.event: "sourcemap"

    - index: "apm-%{[observer.version]}-error-%{+yyyy.MM}"
      when.contains:
        processor.event: "error"

    - index: "apm-%{[observer.version]}-transaction-%{+yyyy.MM}"
      when.contains:
        processor.event: "transaction"

    - index: "apm-%{[observer.version]}-span-%{+yyyy.MM}"
      when.contains:
        processor.event: "span"

    - index: "apm-%{[observer.version]}-metric-%{+yyyy.MM}"
      when.contains:
        processor.event: "metric"

    - index: "apm-%{[observer.version]}-onboarding-%{+yyyy.MM}"
      when.contains:
        processor.event: "onboarding"

```

Please note that the whole `index` and `indices` configuration will be overwritten if you make changes to it in the config file. This means you will need to enable all indices you want to have setup in the configuration.

In case you additionally want to send documents to different indices depending on matching string values, you can also use the `when.regexp` condition, e.g.

```auto
    - index: "apm-%{[observer.version]}-XYZ-%{+yyyy.MM}"
      when.regexp:
        error.culprit: ".*XYZ.*"

```

Concerning your example above, please note that an `error` document can contain an `error.log.message` and an array of `error.exceptions`, where every exception again can have a `message`.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 14, 2019, 4:59am UTC](https://discuss.elastic.co/t/how-to-set-elasticsearch-apm-server-indexing-template-as-yyyy-mm/177811/8 "2019-05-14T04:59:18Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.