# How to specify the index to send the logs of specific microservice **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [November 24, 2021, 10:03am UTC](https://discuss.elastic.co/t/how-to-specify-the-index-to-send-the-logs-of-specific-microservice/290038 "2021-11-24T10:03:48Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![David\_Oceans](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david_oceans/32/51452_2.png) [@David\_Oceans](https://discuss.elastic.co/u/David_Oceans)\ **Post date:** [November 24, 2021, 10:03am UTC](https://discuss.elastic.co/t/how-to-specify-the-index-to-send-the-logs-of-specific-microservice/290038/1 "2021-11-24T10:03:49Z")
Hi, I'm working with GKE and Elastic cloud. I have configured filebeat in my k8s cluster and is working fine, but is sending all the logs to one index, and I would like for some specific services send the logs to another index, thats can be possible? I didn't found any annotations in filebeat to set up this, and I don't know if its possible or not. This is my actual configmap ```auto apiVersion: v1 kind: ConfigMap metadata: name: filebeat-config namespace: logging labels: k8s-app: filebeat data: filebeat.yml: |- logging.level: warning filebeat.autodiscover: providers: - type: kubernetes node: ${NODE_NAME} hints.enabled: true hints.default_config: type: container paths: - /var/log/containers/*${data.kubernetes.container.id}.log exclude_lines: '^[[:space:]]*$' multiline.pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:' multiline.negate: false multiline.match: after processors: - add_cloud_metadata: - add_host_metadata: - decode_json_fields: fields: ["message"] target: "company" overwrite_keys: true - add_fields: target: '' fields: gkeclustername: company-apps-stage environment: stage #setup.dashboards.beat: filebeat #setup.dashboards.enabled: true cloud.id: ${ELASTIC_CLOUD_ID} cloud.auth: ${ELASTIC_CLOUD_AUTH} setup.ilm.rollover_alias: "${INDEX_NAME_CLUSTER}" output.elasticsearch: hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}'] username: ${ELASTICSEARCH_USERNAME} password: ${ELASTICSEARCH_PASSWORD} --- ``` In daemonset block I see the env var where the logs are sending but all logs... ```auto kind: DaemonSet .... .... env: - name: ELASTICSEARCH_HOST value: xxxxxxwest1.gcp.cloud.es.io - name: ELASTICSEARCH_PORT value: "9243" - name: ELASTICSEARCH_USERNAME value: xxxx - name: ELASTICSEARCH_PASSWORD value: xxxxx - name: ELASTIC_CLOUD_ID value: xxxxxxx - name: ELASTIC_CLOUD_AUTH value: xxxx:xxxx - name: INDEX_NAME_CLUSTER value: gke_stage - name: NODE_NAME valueFrom: fieldRef: fieldPath: spec.nodeName ``` Can you help me with any clue? Thank you very much ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [November 24, 2021, 9:08pm UTC](https://discuss.elastic.co/t/how-to-specify-the-index-to-send-the-logs-of-specific-microservice/290038/2 "2021-11-24T21:08:03Z")
That would normally be handled in the `output.elasticsearch` section, but you're using ILM which, unfortunately, complicates things as Filebeat cannot handle variables in the ILM output section. I don't know of another solution here off the top of my head, hopefully someone else can chime in. ---
**Author:** ![Edoardo\_Tenani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/edoardo_tenani/32/94302_2.png) [@Edoardo\_Tenani](https://discuss.elastic.co/u/Edoardo_Tenani)\ **Post date:** [December 2, 2021, 3:30pm UTC](https://discuss.elastic.co/t/how-to-specify-the-index-to-send-the-logs-of-specific-microservice/290038/3 "2021-12-02T15:30:02Z")
Hi @David_Oceans, if I understood correctly you would like to be able to send log lines to different indexes based on their content? So you have one single log "stream" where multiple services send logs and you want to be able to specify destination indexes based on the log message content or attributes. When you say "service" are your referring to specific pods? i.e. you have a nginx pod and a mysql pods and would like their log to be sent to different indexes. Have I understood this right? Thanks ---
**Author:** ![David\_Oceans](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david_oceans/32/51452_2.png) [@David\_Oceans](https://discuss.elastic.co/u/David_Oceans)\ **Post date:** [December 3, 2021, 11:10am UTC](https://discuss.elastic.co/t/how-to-specify-the-index-to-send-the-logs-of-specific-microservice/290038/4 "2021-12-03T11:10:57Z")
Yes, I mean with "Services" a differents pods, exactly what you said. Do you know the best way? I can you a simple annotations in thats pods/deployments of the services to sent to specific index for that service? Thank you very much ---
**Author:** ![Edoardo\_Tenani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/edoardo_tenani/32/94302_2.png) [@Edoardo\_Tenani](https://discuss.elastic.co/u/Edoardo_Tenani)\ **Post date:** [December 17, 2021, 4:12pm UTC](https://discuss.elastic.co/t/how-to-specify-the-index-to-send-the-logs-of-specific-microservice/290038/5 "2021-12-17T16:12:30Z")
Hello @David_Oceans , sorry for the delay on this, it slipped through my notifications. We had a discussion about this internally, and there should be a way to tackle this. First I suggest you to have a look at this other discussion, as it seems related: [Send k8s logs with Filebeat to separate indices - #2 by jbury](https://discuss.elastic.co/t/send-k8s-logs-with-filebeat-to-separate-indices/250632/2) Otherwise this is possible using logstash, between filebeat and Elasticsearch. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 14, 2022, 6:13pm UTC](https://discuss.elastic.co/t/how-to-specify-the-index-to-send-the-logs-of-specific-microservice/290038/6 "2022-01-14T18:13:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.