# Query ElasticSeach with Conf Logstash

**URL:** <https://discuss.elastic.co/t/query-elasticseach-with-conf-logstash/223101>\
**Category:** Logstash\
**Created:** [March 11, 2020, 10:52am UTC](https://discuss.elastic.co/t/query-elasticseach-with-conf-logstash/223101 "2020-03-11T10:52:21Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![brunojpsantos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brunojpsantos/32/51588_2.png) [@brunojpsantos](https://discuss.elastic.co/u/brunojpsantos)\
**Post date:** [March 11, 2020, 10:52am UTC](https://discuss.elastic.co/t/query-elasticseach-with-conf-logstash/223101/1 "2020-03-11T10:52:21Z")

</div>

Hi,  
I want to use a pipeline to do a query in elasticsearch and replace just some fields from csv input file.  
I have this fields in elasticsearch: ID, State  
And I want to replace the field State with CSV file  
Anyone can help me?

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [March 11, 2020, 11:01am UTC](https://discuss.elastic.co/t/query-elasticseach-with-conf-logstash/223101/2 "2020-03-11T11:01:02Z")

</div>

Hi there,

can you share here a snapshot of what you have in elasticsearch and what you have in your csv? That way I'll write the pipeline once.

Thanks

---

<div class="post-metadata">

**Author:** ![brunojpsantos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brunojpsantos/32/51588_2.png) [@brunojpsantos](https://discuss.elastic.co/u/brunojpsantos)\
**Post date:** [March 11, 2020, 11:05am UTC](https://discuss.elastic.co/t/query-elasticseach-with-conf-logstash/223101/3 "2020-03-11T11:05:38Z")

</div>

In the elasticsearch the index is: test\_update  
and the mapping is:  
"mapping": {  
"properties": {  
"ID": {  
"type": "keyword"  
},  
"State": {  
"type": "keyword"  
}  
}  
}  
I have this data in moment:  
ID : 1  
State : In progress

And in the csv to update data I have:  
ID,State  
1, Completed

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [March 11, 2020, 11:29am UTC](https://discuss.elastic.co/t/query-elasticseach-with-conf-logstash/223101/4 "2020-03-11T11:29:15Z")

</div>

Ok, for your next posts, please format your text in any editor (VSCode, Atom, Sublime ecc) properly spacing and indenting it, then paste it here, highlight it and click on the **`Preformatted text`** tool ( ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/a/5a50a65637c15c2b9cdc310625382882d1d99bb1.png) ), otherwise it'll be unreadable.

As for your question, do you want to upload all the lines contained in the csv and update those elasticsearch documents which have an ID already ingested, or do you want to simply update the already existing documents without add new ones?

This means, if in Elasticsearch you have something like this:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/4/f43180c5c3243875766721de4a0c68990fc97a6c.png)

And in your CSV something like this:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/0/608c86efa379f62f441a26c515b0bd994c39f98a.png)

Do you want only the document with `id_3` to be updated or also the lines with `id_4` and `id_5` added?

---

<div class="post-metadata">

**Author:** ![brunojpsantos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brunojpsantos/32/51588_2.png) [@brunojpsantos](https://discuss.elastic.co/u/brunojpsantos)\
**Post date:** [March 11, 2020, 11:32am UTC](https://discuss.elastic.co/t/query-elasticseach-with-conf-logstash/223101/5 "2020-03-11T11:32:14Z")

</div>

I want all be updated..  
In this case, the state of id\_3 will be "state\_3\_new"  
And the id\_4 and id\_5 will be add...

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [March 11, 2020, 11:42am UTC](https://discuss.elastic.co/t/query-elasticseach-with-conf-logstash/223101/6 "2020-03-11T11:42:12Z")

</div>

Ok so, admitting your documents have been ingested using the ID field as \_id of the document, as it should be and as following:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/8/685abfaf40e95a4bd4fbffd2da573daf4caffc9b.png)  
(you can see the `_id` of the document corresponds to what is written in the `ID` field)

This pipeline should do what you want:

```
input {
  file {
    path => "/absolute_path_to_your_csv_file"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

filter {
  csv {
    separator => ","
    columns => ["ID","State"]
  }

  if [ID] == "ID" {
    drop{}
  }
}

output {
  elasticsearch {
    hosts => "localhost:9200"
    index => "test_update"
    document_id => "%{ID}"
    action => "update"
    doc_as_upsert => true
  }
}

```

In fact, running it, this is the result:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/d/9d82e7a4c7f7e01c66eef1321c9d4d6e9987513c.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 8, 2020, 11:42am UTC](https://discuss.elastic.co/t/query-elasticseach-with-conf-logstash/223101/7 "2020-04-08T11:42:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.