# Add filter to send specific fields to elasticsearch
**URL:** https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096
**Category:** Logstash
**Created:** [February 12, 2019, 6:32pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096 "2019-02-12T18:32:27Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@adityak248](https://discuss.elastic.co/u/adityak248)
#### Post date: [February 12, 2019, 6:32pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/1 "2019-02-12T18:32:27Z")
Hello Team,
Greetings
I am looking to add only specific fields to send via filebeat or
I am looking to receive only specific fields in elasticsearch or
I am looking for Logstash to filter the incoming fields and store only those which I wanted in the db.
My ultimate goal is to have only few fields which I wanted to come in logs upon looking at Kibana.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 12, 2019, 6:43pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/2 "2019-02-12T18:43:34Z")
You could try a logstash [prune](https://www.elastic.co/guide/en/logstash/current/plugins-filters-prune.html) filter with the whitelist\_names option.
---
### Author:  [@adityak248](https://discuss.elastic.co/u/adityak248)
#### Post date: [February 12, 2019, 7:34pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/3 "2019-02-12T19:34:39Z")
@Badger
Am quite new to use this rule,
Could you please explain me with example?
---
### Author:  [@adityak248](https://discuss.elastic.co/u/adityak248)
#### Post date: [February 12, 2019, 7:39pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/4 "2019-02-12T19:39:42Z")
So I have been using this filter which is the default one I found in the website.
filter {
if [type] == "syslog" {
grok {
match =\> { "message" =\> "%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}" }
}
```
date {
```
match =\> ["timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
}
}
}
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 12, 2019, 7:43pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/5 "2019-02-12T19:43:09Z")
OK, I think I misinterpreted your question. What does your input look like, and what do you want to see in elasticsearch?
---
### Author:  [@adityak248](https://discuss.elastic.co/u/adityak248)
#### Post date: [February 12, 2019, 7:54pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/6 "2019-02-12T19:54:44Z")
Hey @Badger
Here it is, am pasting entire config.
input {
beats {
port =\> 5044
ssl =\> false
}
filter {
if [type] == "syslog" {
grok {
match =\> { "message" =\> "%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}" }
}
```
date {
```
match =\> ["timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
}
}
}
output {
elasticsearch {
hosts =\> localhost
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
}
stdout {
codec =\> rubydebug
}
}
and now my logs are coming as
{
"\_index": "filebeat-2019.02.12",
"\_type": "doc",
"\_id": "skuhlhdunA-Ssjuo",
"\_version": 1,
"\_score": null,
"\_source": {
"beat": {
"name": "ABC",
"hostname": "ABC",
"version": "6.X.X"
},
"host": {
"name": "ABC",
"id": "acjkgdljhvjhfgfjhlebkjhdg",
"architecture": "",
"containerized": true,
"os": {
"platform": "aaa",
"codename": "Core",
"version": "7 ",
"family": "zaaaa"
}
},
"message": "The log with the affected area",
"offset": 6743534,
"input": {
"type": "log"
},
"prospector": {
"type": "log"
},
"meta": {
"cloud": {
"machine\_type": "qqqqqq",
"region": "",
"availability\_zone": "sdfsfd",
"instance\_id": "fsdfsdfsfsfdf",
"provider": "sfds"
}
},
"tags": [
"beats\_input\_codec\_plain\_applied"
],
"@timestamp": "2019-02-12T17:20:46.163Z",
"source": "error.log",
"@version": "1"
},
"fields": {
"@timestamp": [
"2019-02-12T17:20:46.163Z"
]s
},
"sort": [
1549992046163
]
}
The requirement is, I don't want any of those except for few like hostname , message, source,
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 12, 2019, 8:09pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/7 "2019-02-12T20:09:28Z")
OK, so maybe a prune filter is what you want. It is not installed by default, the documentation that I linked to before explains how to install it.
```
filter {
prune {
whitelist_names => ["^hostname$", "^message$", "^source"]
}
}
```
---
### Author:  [@adityak248](https://discuss.elastic.co/u/adityak248)
#### Post date: [February 12, 2019, 8:19pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/8 "2019-02-12T20:19:17Z")
Hey @Badger
To confirm this is how am working on the config
input {
beats {
port =\> 5044
ssl =\> false
}
}
filter {
prune {
whitelist\_names =\> ["^hostname$", "^message$", "^source"]
}
}
output {
elasticsearch {
hosts =\> localhost
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
}
stdout {
codec =\> rubydebug
}
}
Please confirm if it looks good, is this what is expected.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 12, 2019, 8:43pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/9 "2019-02-12T20:43:32Z")
That's what I am suggesting, yes.
---
### Author:  [@adityak248](https://discuss.elastic.co/u/adityak248)
#### Post date: [February 12, 2019, 8:45pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/10 "2019-02-12T20:45:32Z")
That didn't help me @Badger
It still giving me the same output.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 12, 2019, 9:09pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/11 "2019-02-12T21:09:54Z")
Did you ingest some new logs? Adding the filter will not change the documents already ingested.
---
### Author:  [@adityak248](https://discuss.elastic.co/u/adityak248)
#### Post date: [February 12, 2019, 9:22pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/12 "2019-02-12T21:22:33Z")
@Badger
Yes I did of course. 😄
I am getting the entire stack again which I didn't want.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 12, 2019, 9:26pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096/13 "2019-03-12T21:26:22Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.