Parsing a logstash file

Elastic Observability Logs
1

I want to parse my logstash file to be able to delete some unnecessary fields that appear in kibana

This is the input file from filebeat

{
  "source": "10.200.226.62:57500",
  "subscription-name": "default-1764331045",
  "timestamp": 1764328788493751000,
  "time": "2025-11-28T12:19:48.493751+01:00",
  "updates": [
    {
      "Path": "interfaces/interface[name=GigabitEthernet0/0]/state/counters/out-
octets",
      "values": {
        "interfaces/interface/state/counters/out-octets": "334577330"
      }
    }
  ]
}
{
  "source": "10.200.10.10:57500",
  "subscription-name": "default-1764331045",
  "timestamp": 1764328788497842000,
  "time": "2025-11-28T12:19:48.497842+01:00"
}
{
  "sync-response": true
}
{
  "source": "10.200.10.10:57500",
  "subscription-name": "default-1764331045",
  "timestamp": 1764328798498615000,
  "time": "2025-11-28T12:19:58.498615+01:00",
  "updates": [
    {
      "Path": "interfaces/interface[name=GigabitEthernet0/0]/state/counters/out-octets",
      "values": {
        "interfaces/interface/state/counters/out-octets": "334579570"
      }
    }
  ]

This is an example of the logstash file

input {
  beats {
    port => 5044
  }
}

filter {

}

output {
  stdout { codec => rubydebug }
    elasticsearch {
    hosts => ["https://localhost:9200"]
    index => "telegraf-logstash"
    user => "elastic"
    password => "xxxxxxxxx"
    ssl_enabled => true
    ssl_certificate_authorities => ["/etc/elasticsearch/certs/http_ca.crt"]
  }
}

this is the output

image
image1768×892 247 KB

I dont want fields like host architecture etc

Thanks

I dont want all this

2

Look in your beats config for

#The following example enriches each event with host
#metadata.

processors:
add_host_metadata: ~

If it’s there, comment it out.

3

Also you can drop fields in filebeat.yml:

processors:
  - add_host_metadata: ...
  
  - drop_fields:
      fields: ["agent.version","agent.type","agent.id","agent.ephemeral_id","host","mac","ip","log.file.vol","log.file.idxhi","log.file.idxlo","log.offset"]
      ignore_missing: true

Just adopt list.

Of couse, you can delete in LS:

mutate {
       remove_field => [ "agent", "host" ]
     }