# Add new field to index using Logstash getting null values
**URL:** https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370
**Category:** Logstash
**Created:** [February 6, 2019, 11:51pm UTC](https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370 "2019-02-06T23:51:03Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Miguel\_Plazas](https://discuss.elastic.co/u/Miguel_Plazas)
#### Post date: [February 6, 2019, 11:51pm UTC](https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370/1 "2019-02-06T23:51:03Z")
Hi, currently i would like to add a new field **gain** in a current index based on a field value as follows:
```
input {
elasticsearch {
hosts => "http://localhost:9200"
index => "my_index"
}
}
filter {
if [gain_factor] > "1.0" {
mutate {
add_field => ["gain", "exist"]
}
}
else {
mutate {
add_field => ["gain", "no-exist"]
}
}
}
output {
elasticsearch {
hosts => "http://localhost:9200"
index => "my_index"
}
stdout {}
}
```
When i make a query as follows:
POST \_xpack/sql?format=txt
{
"query" : "SELECT gain\_factor, gain FROM my\_index"
}
I'm getting that a lot documents in the field **gain** are **null**. my target is just add a new field in my index and set "exist" in gain field when gain\_factor \> 1.0 and when gain\_factor \< 1.0 set "no-exist" in **gain** field. How can i do that?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 7, 2019, 12:56am UTC](https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370/2 "2019-02-07T00:56:21Z")
Take a look at the [docinfo](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html#plugins-inputs-elasticsearch-docinfo) option on the elasticsearch input. You will need that to preserve the document\_id so that you can overwrite an existing document.
The handling of document\_type is probably going to confuse you. It is in the process of being retired, so the documentation says you need it (and you may) but if you use it then the software warns you not to use it.
Start off by setting 'docinfo =\> true' on the input and setting
```
document_id => "%{[@metadata][_id]}"
```
on the elasticsearch output.
---
### Author:  [@Miguel\_Plazas](https://discuss.elastic.co/u/Miguel_Plazas)
#### Post date: [February 7, 2019, 9:35pm UTC](https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370/3 "2019-02-07T21:35:47Z")
Thanks! It works perfect!!!!
---
### Author:  [@Miguel\_Plazas](https://discuss.elastic.co/u/Miguel_Plazas)
#### Post date: [February 7, 2019, 9:46pm UTC](https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370/4 "2019-02-07T21:46:07Z")
When the logstash finish the index processing stops and say:
`[INFO] 2019-02-07 16:25:47.671 [[main]-pipeline-manager] pipeline - Pipeline has terminated {:pipeline_id=>"main", :thread=>"#"}`
it is possible to keep the config file working in real time in such a way that when the index gets new data, it will be processed with logstash?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 7, 2019, 10:08pm UTC](https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370/5 "2019-02-07T22:08:57Z")
Run it using a [schedule](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html#_scheduling) and update your query so it only fetches documents that do not have a gain field.
---
### Author:  [@Miguel\_Plazas](https://discuss.elastic.co/u/Miguel_Plazas)
#### Post date: [February 7, 2019, 10:31pm UTC](https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370/6 "2019-02-07T22:31:41Z")
> [@Badger](#):
>
> update your query
how can i do that? and the shedule to make it in real time is this?
`schedule => "* * * * *"`
thanks for your help
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 7, 2019, 10:41pm UTC](https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370/7 "2019-02-07T22:41:19Z")
That schedule would run the query once a second, yes. I have never run anything that often, so I do not know if it creates any issues.
As to what the query should be, that's an elasticsearch question, but you pass a query using the query option on the input.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 7, 2019, 10:41pm UTC](https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370/8 "2019-03-07T22:41:22Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.