How to write only specific fields to elasticsearch from logstash

Elastic Stack Logstash
1

I want to output an event to elasticsearch, but I really only want to have 4 - 6 fields in the index, but the event is coming in with 130 fields. I know that I can use the remove_field array in a mutate or other filter, but that means that I would have to list out 120+ field names in the logstash config.

Is there an easy way to specify only specific fields to write to ES? I was thinking about the clone filter, but it doesn't seem to let you clone only certain events of a field.

1 Like
2

What sort of file is it? CSV, other?

3

Not a file. It's a log being sent to logstash on a TCP port.

4

The prune filter allows you to remove all fields except the list of 4–6 fields that you want to keep.

3 Likes
5

awesome! This looks like the filter I was looking for. I'll test it. Thanks for the heads up!