# Elasticsearch Ingest Node or Filebeats processors or Logstash to add fields to my logs

**URL:** https://discuss.elastic.co/t/elasticsearch-ingest-node-or-filebeats-processors-or-logstash-to-add-fields-to-my-logs/257709
**Category:** Elasticsearch
**Tags:** docker
**Created:** [December 4, 2020, 11:54pm UTC](https://discuss.elastic.co/t/elasticsearch-ingest-node-or-filebeats-processors-or-logstash-to-add-fields-to-my-logs/257709 "2020-12-04T23:54:30Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![ghettosamson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ghettosamson/32/80305_2.png) [@ghettosamson](https://discuss.elastic.co/u/ghettosamson)
#### Post date: [December 4, 2020, 11:54pm UTC](https://discuss.elastic.co/t/elasticsearch-ingest-node-or-filebeats-processors-or-logstash-to-add-fields-to-my-logs/257709/1 "2020-12-04T23:54:30Z")

</div>

I am running Dockerized Elastic Stack on a single AWS EC2 instance, with Packetbeats, Filebeats, Metricbeats, Kibana and Elasticsearch. I've been reading the documentation over this past week and I need help determining if I need to move to using Logstash or Ingest Node to add fields to my container logs, or if there is a way to format my logs within my Docker apps to have Filebeats ship them with the necessary fields. My web API is running in Docker, and I want some information pulled out of my logs and made into fields. For example  
User tswift has logged in  
User pebbles accessed the endpoint createHospital  
A new hospital MyNewHospital was created  
For line 1, I would like user to be pulled out as a field  
For line 2, I would like user and endpoint to be pulled out as a field  
For line 3, I would like hospital to be pulled out as a filed  
I know this is possible with Logstash and grok filters, but is there a way to do this with Filebeats or Ingest Node. Or is there a way to format my logs so it doesn't need extra processing?

---

<div class="post-metadata">

### Author: ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)
#### Post date: [December 5, 2020, 3:17am UTC](https://discuss.elastic.co/t/elasticsearch-ingest-node-or-filebeats-processors-or-logstash-to-add-fields-to-my-logs/257709/2 "2020-12-05T03:17:04Z")

</div>

With the default config, sending beats to elastic, you are probably using ingest now, if your are using the beats modules. You can verify with a GET/\_nodes/stats/ingest call. If so, you might be able to add code to the supplied ingest pipeline.

---

<div class="post-metadata">

### Author: ![ghettosamson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ghettosamson/32/80305_2.png) [@ghettosamson](https://discuss.elastic.co/u/ghettosamson)
#### Post date: [December 7, 2020, 8:12pm UTC](https://discuss.elastic.co/t/elasticsearch-ingest-node-or-filebeats-processors-or-logstash-to-add-fields-to-my-logs/257709/3 "2020-12-07T20:12:45Z")

</div>

Here is what I get back from the curl.

```auto
{
  "_nodes": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "cluster_name": "docker-cluster",
  "nodes": {
    "_M3xRqdIRRS4AsK4NZbSHg": {
      "timestamp": 1607371764725,
      "name": "f4d0e0bc40ea",
      "transport_address": "XXXX:9300",
      "host": "XXXX",
      "ip": "XXXX:9300",
      "roles": [
        "data",
        "data_cold",
        "data_content",
        "data_hot",
        "data_warm",
        "ingest",
        "master",
        "ml",
        "remote_cluster_client",
        "transform"
      ],
      "attributes": {
        "ml.machine_memory": "16624467968",
        "xpack.installed": "true",
        "transform.node": "true",
        "ml.max_open_jobs": "20"
      },
      "ingest": {
        "total": {
          "count": 0,
          "time_in_millis": 0,
          "current": 0,
          "failed": 0
        },
        "pipelines": {
          "xpack_monitoring_6": {
            "count": 0,
            "time_in_millis": 0,
            "current": 0,
            "failed": 0,
            "processors": [
              {
                "script": {
                  "type": "script",
                  "stats": {
                    "count": 0,
                    "time_in_millis": 0,
                    "current": 0,
                    "failed": 0
                  }
                }
              },
              {
                "gsub": {
                  "type": "gsub",
                  "stats": {
                    "count": 0,
                    "time_in_millis": 0,
                    "current": 0,
                    "failed": 0
                  }
                }
              }
            ]
          },
          "xpack_monitoring_7": {
            "count": 0,
            "time_in_millis": 0,
            "current": 0,
            "failed": 0,
            "processors": []
          }
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 4, 2021, 8:12pm UTC](https://discuss.elastic.co/t/elasticsearch-ingest-node-or-filebeats-processors-or-logstash-to-add-fields-to-my-logs/257709/4 "2021-01-04T20:12:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
