# Logstash configuration not working on read csv file

**URL:** https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515
**Category:** Logstash
**Created:** [June 7, 2022, 2:28am UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515 "2022-06-07T02:28:06Z")
**Posts on this page:** 13
**Page:** 1

<div class="post-metadata">

### Author: ![selflabs](https://avatars.discourse-cdn.com/v4/letter/s/97f17d/32.png) [@selflabs](https://discuss.elastic.co/u/selflabs)
#### Post date: [June 7, 2022, 2:28am UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/1 "2022-06-07T02:28:06Z")

</div>

Hello Everyone, Logstash not reading csv file, i have tried all possible case but unable to pick data from the csv file. my ultimate goal is read data from REST API and mapping with CSV file. Please suggest, Configuration as below -

```auto
input{
  http{
      host => "127.0.0.1"
      port => " *****"
      ecs_compatibility => disabled
      type => "input"
     }

}
filter {
json{ source => "message"
ecs_compatibility => disabled }
split { field => "result" }
mutate { add_field => {"ids" => "%{[result][server]}"} }
translate {
        dictionary_path => "/etc/logstash/conf.d/myfilename.csv"
        source => "[ids]"
        target => "[OS]"
        fallback => "I'm a teapot"
    }
mutate {remove_field => ["server","http","url","agent"] }
}
output {
  stdout { codec => rubydebug }
}

```

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [June 7, 2022, 3:00am UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/2 "2022-06-07T03:00:40Z")

</div>

You need to provide more context.

What is not working? What does your input message looks like? What is your output? What is the expected output?

You do not have a file plugin, so by _reading csv_ file you mean the csv file used in the translate filter? Share this as well.

---

<div class="post-metadata">

### Author: ![selflabs](https://avatars.discourse-cdn.com/v4/letter/s/97f17d/32.png) [@selflabs](https://discuss.elastic.co/u/selflabs)
#### Post date: [June 7, 2022, 5:26am UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/3 "2022-06-07T05:26:58Z")

</div>

Hi, I'm fetching hostname from one of the event and search in csv file, on the match it should populate value field on display.  
yes i have a plugin, but here in configuration am using translate.

Data present in csv file as below -  
Hostname,Value  
server\_1,Windows  
server\_2,Linux  
server\_3,Unix  
server\_4,Windows  
server\_5,Linux  
server\_6,Unix

Getting below error while executing this configuration -  
`[ERROR] 2022-06-07 10:52:12.251 [[main]-pipeline-manager] javapipeline - Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::Filters::Dictionary::DictionaryFileError: Translate: Missing or stray quote in line 1 when loading dictionary file at /etc/logstash/conf.d/filename.csv>, :backtrace=>["/usr/share/logstash/vendor/jruby/lib/ruby/stdlib/csv.rb:1899:in `block in shift'"`

`[ERROR] 2022-06-07 10:52:12.274 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}`

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [June 7, 2022, 12:51pm UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/4 "2022-06-07T12:51:10Z")

</div>

> [@selflabs](#):
>
> \<LogStash::Filters::Dictionary::DictionaryFileError: Translate: Missing or stray quote in line 1 when loading dictionary file at /etc/logstash/conf.d/filename.csv\>

Can you check the file? This is saying that you have some issue with quotes.

---

<div class="post-metadata">

### Author: ![selflabs](https://avatars.discourse-cdn.com/v4/letter/s/97f17d/32.png) [@selflabs](https://discuss.elastic.co/u/selflabs)
#### Post date: [June 8, 2022, 2:18pm UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/5 "2022-06-08T14:18:35Z")

</div>

Hi, I have tried all possible formats/quotes but error remain persists.  
Please suggest if there is any desired format for save.  
below which i have tried.

## "Hostname","Value" "server\_1","Windows" "server\_2","Linux"

## "Hostname","Value" "server\_1","Windows", "server\_2","Linux"

## "Hostname", "Value", "server\_1", "Windows", "server\_2","Linux",

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [June 8, 2022, 3:00pm UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/6 "2022-06-08T15:00:15Z")

</div>

Your dictionary should look something like this:

```auto
server_1,Windows
server_2,Linux
server_3,Unix
server_4,Windows
server_5,Linux
server_6,Unix

```

According to the [documentation](https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html#plugins-filters-translate-dictionary_path) you only need quotes when using integers as keys, so you doesn't need to use it when your keys are strings.

Is your dictionary looking as the one I shared and you are still getting the exactly same error? Can you enable debug log to get more information?

---

<div class="post-metadata">

### Author: ![selflabs](https://avatars.discourse-cdn.com/v4/letter/s/97f17d/32.png) [@selflabs](https://discuss.elastic.co/u/selflabs)
#### Post date: [June 14, 2022, 6:51am UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/7 "2022-06-14T06:51:24Z")

</div>

Hi Leandro, I have tried all possible ways but can't do a lookup of a field in a csv file.  
can you please suggest the translate plugin logic, herewith am pasting the error logs with debug mode.

```auto
input{
  http{
      host => "127.0.0.1"
      port => " *****"
      ecs_compatibility => disabled
      type => "input"
     }
}
filter {
	json{ source => "message"
    	ecs_compatibility => disabled }
	    split { field => "result" }
	    mutate { add_field => {"ids" => "%{[result][server]}"} }
    csv {
        separator => ","
        skip_header => "true"
        columns => ["server", "os"]
        ecs_compatibility => disabled
    }
    translate {
        dictionary_path => "/etc/logstash/conf.d/myfilename.csv"
        source => "[ids]"
        target => "[os]"
        fallback => "I'm a teapot"
    }
mutate {remove_field => ["server","http","url","agent"] }
}
output {
  stdout { codec => rubydebug }
}
![translate-plugin|690x339](upload://xG0nRedmlJ2wMtNZLL0EHiOANxK.png)

```

 ![translate-plugin](https://us1.discourse-cdn.com/elastic/original/3X/f/5/f5d97b333660b5b969cda160f15043a1049d9319.png)

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [June 14, 2022, 3:15pm UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/8 "2022-06-14T15:15:57Z")

</div>

It is still giving the same error, that it is something wrong in your `csv` file, in the first line.

Do you have anything else in your file besides the key value pairs? A header, maybe? You can't have anything in the file that is not in the format:

`key,value`

Can you open your file in a terminal text editor and share a print of it?

---

<div class="post-metadata">

### Author: ![selflabs](https://avatars.discourse-cdn.com/v4/letter/s/97f17d/32.png) [@selflabs](https://discuss.elastic.co/u/selflabs)
#### Post date: [June 15, 2022, 5:34am UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/9 "2022-06-15T05:34:32Z")

</div>

Hi Leandro, PFA print of that csv file. i'm not using any header in the file.  
 ![translate1](https://us1.discourse-cdn.com/elastic/original/3X/5/e/5ef121cd7788a40c69bec2e20e0c0060a96f59fd.png)

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [June 15, 2022, 11:56am UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/10 "2022-06-15T11:56:55Z")

</div>

It looks like a normal csv, I have no hints why Logstash would give you such error.

I would try to manually create a new file in the yml format.

```auto
"server_1": "Windows"
"server_2": "Linux"
"server_3": "Unix"
"server_4": "Windows"
"server_5": "Linux"
"server_6": "Unix"

```

And then use this file in your translate filter.

---

<div class="post-metadata">

### Author: ![selflabs](https://avatars.discourse-cdn.com/v4/letter/s/97f17d/32.png) [@selflabs](https://discuss.elastic.co/u/selflabs)
#### Post date: [June 15, 2022, 12:19pm UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/11 "2022-06-15T12:19:08Z")

</div>

Hi Leandro, We have tested with yaml and json format and it works perfectly fine. the problem occurs only on csv format. so can we conclude that csv format is not supported by translate plugin/within logstash.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [June 15, 2022, 12:24pm UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/12 "2022-06-15T12:24:49Z")

</div>

This is not entirely right, the csv format is supported, but for some reason your csv wasn't working, maybe there was something related on how it is created in your system or things like that.

But to know why it wasn't working you would need to open a bug report in the Logstash Github.

Particularly I prefer to use `yml` files in my dictionaries, so if you changed to `yml` and it worked, you should keep using `yml`.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 13, 2022, 12:25pm UTC](https://discuss.elastic.co/t/logstash-configuration-not-working-on-read-csv-file/306515/13 "2022-07-13T12:25:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
