# Reindex + enrich data

**URL:** https://discuss.elastic.co/t/reindex-enrich-data/161762
**Category:** Logstash
**Created:** [December 20, 2018, 10:43pm UTC](https://discuss.elastic.co/t/reindex-enrich-data/161762 "2018-12-20T22:43:35Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![andre22](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@andre22](https://discuss.elastic.co/u/andre22)
#### Post date: [December 20, 2018, 10:43pm UTC](https://discuss.elastic.co/t/reindex-enrich-data/161762/1 "2018-12-20T22:43:35Z")

</div>

Hi, I am trying to reindex sensor data in order to enrich it via lookup. My goal is to add multiple fields, as shown in the csv plugin config below.  
While the reindexing itself starts, I see no difference in the new index. Here is my configuration:

```
reindex.conf
    input {
      elasticsearch {
        hosts => ["localhost:9200"]
        query => '{ "query": { "query_string": { "query": "*" } } }'
        index => "knx-*"
        docinfo => true
      }
    }
    filter {
      translate {
        field => "edomi-GA_KO"
    # destination => "alldata"
        dictionary_path => "/etc/logstash/conf.d/lookup/knxlookup.csv"
    # exact => false
        fallback => "no match"
    # start_position => "beginning"
        }
      csv {
        source => "translation"
        columns => [ 
        "GA",
        "ID",
        "Name",
        "DPT",
        "Geschoss",
        "Raum",
        "Gewerk",
        "Funktion"
        ]
        separator => ";"
        }
    }
    output {
      elasticsearch {
        hosts => ["localhost:9200"]
        index => "knx2-%{+YYYY.MM}"
        document_id => "%{[@metadata][_id]}"
      }
    }

```

the lookup file is present and has this value:

```
1/1/0;133;Raum 1 Licht schalten;1;EG;Raum 1;Licht;ein / aus
1/1/1;134;Raum 2 Licht schalten;1;EG;Raum 2;Licht;ein / aus

```

The Field "edomi-GA\_KO" is present in the source index and has data matching the first column in the lookup file.  
Following the documentation the translate plugin would add a field named "translation" by default, which also isn't created. Logstash logs don't show errors but indicate that the reindex is starting which I can use afterwards.

Thanks in advance!  
André

EDIT:

Now after letting it sit for some minutes and the new index is created, has the same number of documents as the source index. after refreshing the index pattern I see the "translation" field and the first column "GA" showing up. But they aren't visible in Kibana Discover. I notice that still something is going on, and cat on the indices gives me this (docs.deleted is growing while I am writing this):

```
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open knx2-2017.05 QfwJbTbZSaqD2FprBcndKQ 1 0 6122337 2443710 2.2gb 2.2gb
green open knx-2017.05a iZ-NZ1ojRb2qXuDOp-z-zQ 1 0 6122337 0 956.8mb 956.8mb
```

---

<div class="post-metadata">

### Author: ![andre22](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@andre22](https://discuss.elastic.co/u/andre22)
#### Post date: [December 20, 2018, 11:10pm UTC](https://discuss.elastic.co/t/reindex-enrich-data/161762/2 "2018-12-20T23:10:13Z")

</div>

Okay after a while the translation field shows up, unfortunately it always shows "no match" so there's something wrong with my translate config.

---

<div class="post-metadata">

### Author: ![andre22](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@andre22](https://discuss.elastic.co/u/andre22)
#### Post date: [December 20, 2018, 11:35pm UTC](https://discuss.elastic.co/t/reindex-enrich-data/161762/3 "2018-12-20T23:35:20Z")

</div>

Solved it by myself. I had to recreate the lookupto searate the first column with a comma (because translate doesn't seen to accept the ";" and then all other column with it.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 17, 2019, 11:35pm UTC](https://discuss.elastic.co/t/reindex-enrich-data/161762/4 "2019-01-17T23:35:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.