How to create index for each service?

[seven-yu]

I am using version 7.4.0

Apm creates a global index for all services by default：
apm-%{[beat.version]}-%{+yyyy.MM.dd}

Considering the management of ES storage capacity, I need to analyze the data growth trend of some major services, not the total amount of all services

I want to automatically create an index for each new service. Is there any experience I can refer to?

[axw (Andrew Wilkins)]

Hi @seven-yu, welcome to the forum!

You can change the index name by setting either output.elasticsearch.index or output.elasticsearch.indices. These configuration variables take an index name "format string", which can reference fields the events. For your use-case, you can include %{[service.name]} in the format string to create an index per service.

e.g.

output.elasticsearch:
  hosts: ["http://localhost:9200"]
  indices:
   - index: "apm-%{[observer.version]}-sourcemap"
      when.contains:
        processor.event: "sourcemap"

   - index: "apm-%{[observer.version]}-onboarding-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "onboarding"

   - index: "apm-%{[observer.version]}-%{[service.name]}-error-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "error"

   - index: "apm-%{[observer.version]}-%{[service.name]}-transaction-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "transaction"

   - index: "apm-%{[observer.version]}-%{[service.name]}-span-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "span"

   - index: "apm-%{[observer.version]}-%{[service.name]}-metric-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "metric"

Note that the "onboarding" and "sourcemap" indices are not service-specific, hence I did not include %{[service.name]} in their names in this example.