# Logstash export not working correctly, only a part of data exported

**URL:** https://discuss.elastic.co/t/logstash-export-not-working-correctly-only-a-part-of-data-exported/346657
**Category:** Logstash
**Created:** [November 7, 2023, 9:10pm UTC](https://discuss.elastic.co/t/logstash-export-not-working-correctly-only-a-part-of-data-exported/346657 "2023-11-07T21:10:46Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![andre22](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@andre22](https://discuss.elastic.co/u/andre22)
#### Post date: [November 7, 2023, 9:10pm UTC](https://discuss.elastic.co/t/logstash-export-not-working-correctly-only-a-part-of-data-exported/346657/1 "2023-11-07T21:10:46Z")

</div>

Hi,

i want to export some data from old indexes and write them into a text file. When I restart logstash, it exports some data (a part of one day, the index has a complete month) and goes back to do nothing. I am using the latest Version 7 on this machine.

config:

```auto
input {
  elasticsearch {
    hosts => ["localhost:9200"]
    query => '{ "query": { "query_string": { "query": "*" } } }'
    index => "home-2023.09*"
    docinfo => true
  }
}

filter {
  translate {
    field => "edomi-KO-ID"
    destination => "translated_data"
    dictionary_path => "/etc/logstash/conf.d/lookup_edomi-KNXGA.csv"
    fallback => "NOT_FOUND"
    refresh_interval => 3600 
  }

  if [translated_data] == "NOT_FOUND" {
    drop {}
  }

  translate {
    field => "edomi-KO-ID"
    destination => "edomi-GA_KO"
    dictionary_path => "/etc/logstash/conf.d/lookup_edomi-KNXGA.csv"
    refresh_interval => 3600
    fallback => "null"
  }
}

output {
  csv {
    fields => ["@timestamp", "Funktion", "Geschoss", "Gewerk", "KNX-Name", "KNX-Wert", "KNX-Wert-Float", "Name", "Raum", "edomi-GA_KO", "edomi-KO-ID", "edomi-PA", "edomi-Typ"]
    path => "/opt/export/knx2-%{+YYYY-MM-dd}.txt"
  }
}

```

Edit:  
What I want to achieve:

- retrieve the index from the input section
- check if the value in edomi-KO-ID exists in the first line of the lookup file
- if not, drop the message
- if it exists, use the lookup file to add a value into the field edomi-GA\_KO
- write to a file that contains the date in the filename.

the lookup table has roughly 70 lines. the exported file has 602 lines

Am I missing something? Could this be a caching issue or something?

Thanks for your help!

---

<div class="post-metadata">

### Author: ![andre22](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@andre22](https://discuss.elastic.co/u/andre22)
#### Post date: [November 7, 2023, 10:27pm UTC](https://discuss.elastic.co/t/logstash-export-not-working-correctly-only-a-part-of-data-exported/346657/2 "2023-11-07T22:27:33Z")

</div>

I didn't realize that the disk space watermark was surpassed during my tests which led elasticsearch to either stop or not start again when I restarted it during my tests. the export is working now, causing another question for which I will create a new thread later.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 5, 2023, 10:28pm UTC](https://discuss.elastic.co/t/logstash-export-not-working-correctly-only-a-part-of-data-exported/346657/3 "2023-12-05T22:28:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.