# Default APM Server docker did not create apm-7.5.1-transaction and apm-7.5.1-span with suffix number behind **URL:** \ **Category:** APM\ **Tags:** server\ **Created:** [January 21, 2020, 2:49am UTC](https://discuss.elastic.co/t/default-apm-server-docker-did-not-create-apm-7-5-1-transaction-and-apm-7-5-1-span-with-suffix-number-behind/215815 "2020-01-21T02:49:14Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Huy\_Ngo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/huy_ngo/32/61101_2.png) [@Huy\_Ngo](https://discuss.elastic.co/u/Huy_Ngo)\ **Post date:** [January 21, 2020, 2:49am UTC](https://discuss.elastic.co/t/default-apm-server-docker-did-not-create-apm-7-5-1-transaction-and-apm-7-5-1-span-with-suffix-number-behind/215815/1 "2020-01-21T02:49:14Z")
**Kibana version** : 7.5.1 **Elasticsearch version** : 7.5.1 **APM Server version** : 7.5.1 **APM Agent language and version** : go **Browser version** : Version 79.0.3945.88 (Official Build) (64-bit) **Original install method (e.g. download page, yum, deb, from source, etc.) and version**: Docker image apm-server:7.5.1 **Fresh install or upgraded from other version?** Fresh install **Is there anything special in your setup?** We're using fluent bit to ship log to Elasticsearch **Description of the problem including expected versus actual behavior. Please include screenshots (if relevant)**: After install apm-server and access to output Elasticsearch, checked the Indice I don't see the indices had been created with "apm-7.5.1-transaction-000001" and "apm-7.5.1-span-000001". That make the life cycle rotate did not work for two indices above. Error generated: illegal\_argument\_exception: index.lifecycle.rollover\_alias [apm-7.5.1-transaction] does not point to index [apm-7.5.1-transaction] ``` ⎈ |PRODUCTION_USE1:logging)]$ curl localhost:9200/_cat/indices?v | grep span % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Handling connection for 9200 100 4953 100 4953 0 0 3984 0 0:00:01 0:00:01 --:--:-- 3987 green open apm-7.5.1-span FnJJgBSUTOKpmRf3mux1yg 1 1 43516421 0 25.8gb 12.9gb ``` **Steps to reproduce** : 1. Install apm as K8s DaemonSet 2. Verify that apm can output to ES successfully 3. Check the indices make sure ES has Indices with suffix apm-7.5.1-transaction-000001 **Errors in browser console (if relevant)**: ![Screenshot 2020-01-21 at 9.46.17 AM](https://us1.discourse-cdn.com/elastic/original/3X/6/c/6cd9320032b129900733aec8bb52455559c5eef9.png) **Provide logs and/or server output (if relevant)**: > ``` > --- > apiVersion: apps/v1 > kind: DaemonSet > metadata: > name: euw1-apm > namespace: logging > labels: > app: euw1-apm > apm-infra: euw1-apm > spec: > updateStrategy: > type: RollingUpdate > rollingUpdate: > maxUnavailable: 100 > selector: > matchLabels: > app: euw1-apm > apm-infra: euw1-apm > template: > metadata: > labels: > app: euw1-apm > apm-infra: euw1-apm > spec: > hostNetwork: true # NOTE: Agent is configured to have same IP as the host/node > dnsPolicy: ClusterFirstWithHostNet > containers: > - name: euw1-apm > image: "docker.elastic.co/apm/apm-server:7.5.1" > imagePullPolicy: IfNotPresent > args: > - "--strict.perms=false" > - "-e" > - "-E" > - "output.elasticsearch.hosts=[\"http://replacing-me:9200\"]" > # - "output.elasticsearch.hosts=[\"http://euw1-es-http:9200\"]" > # - "-E" > # - "output.elasticsearch.index=\"apm-%{[observer.version]}-%{[processor.event]}-%{+yyyy.MM.dd}\"" > # - "-E" > # - "setup.template.name=\"apm-server\"" > # - "-E" > # - "setup.template.pattern=\"apm-%{[observer.version]}-%{[processor.event]}-*\"" > > ports: > - containerPort: 8200 > protocol: TCP > livenessProbe: > httpGet: > path: / > port: 8200 > initialDelaySeconds: 60 > readinessProbe: > httpGet: > path: / > port: 8200 > initialDelaySeconds: 60 > > ``` Added more log when starting APM: ``` 2020-01-21T04:05:49.232Z INFO [request] middleware/log_middleware.go:76 request ok {"request_id": "3f896fbd-aa69-4969-9e55-466722d3a023", "method": "GET", "URL": "/", "content_length": 0, "remote_address": "172.16.158.138", "user-agent": "kube-probe/1.14+", "response_code": 200} 2020-01-21T04:05:56.957Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://euw1-es-http.logging.svc.cluster.local:9200)): Connection marked as failed because the onConnect callback failed: resource 'apm-7.5.1-span' exists, but it is not an alias 2020-01-21T04:05:56.957Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://euw1-es-http.logging.svc.cluster.local:9200)) with 153 reconnect attempt(s) ``` Thanks, Huy ---
**Author:** ![simitt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simitt/32/106406_2.png) [@simitt](https://discuss.elastic.co/u/simitt)\ **Post date:** [January 23, 2020, 8:29am UTC](https://discuss.elastic.co/t/default-apm-server-docker-did-not-create-apm-7-5-1-transaction-and-apm-7-5-1-span-with-suffix-number-behind/215815/2 "2020-01-23T08:29:19Z")
Hi @Huy_Ngo, looking at your config file I wonder if you had run the APM Server with a custom `output.elasticsearch.index` configuration before, which would set up indices without ILM. In `7.5.` indices are by default setup using ILM, the according setting is `apm-server.ilm.enabled: auto`. If however a custom `output.elasticsearch.index` is configured, ILM is disabled by default. The config file you show suggests that you might have had APM Server running with some custom indices. In such a case you would need to set `apm-server.ilm.setup.overwrite: true` to allow APM Server to take care of setting up indices with ILM. However, since in your case indices already exist that have the same name as the expected aliases, APM Server will not be able to overwrite those. Is it feasible for you to delete the troubling indices and restart APM Server with the default configuration for indices and ILM? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 13, 2020, 4:29am UTC](https://discuss.elastic.co/t/default-apm-server-docker-did-not-create-apm-7-5-1-transaction-and-apm-7-5-1-span-with-suffix-number-behind/215815/3 "2020-02-13T04:29:23Z")
This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.