# Logstash add field from another input **URL:** \ **Category:** Logstash\ **Created:** [February 6, 2019, 1:33pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292 "2019-02-06T13:33:54Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![fadihaddad](https://avatars.discourse-cdn.com/v4/letter/f/4bbf92/32.png) [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Post date:** [February 6, 2019, 1:33pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/1 "2019-02-06T13:33:54Z")
hello i have a csv file with this field "type" : "255~15674532~15724727~50195|256~41089839104~41221424128~131585024" and another csv file with 3 fields value , name , function first i used this ruby filter code to split this type field ruby { code =\> " y = event.get('type').split('|').collect { |t| c = t.split '~' { 'type\_ID' =\> c[0].to\_i, type\_Before' =\> c[1].to\_i,- 'type\_After' =\> c[2].to\_i, 'type\_Change' =\> c[3].to\_i, ``` } } event.set('Type', y) ``` " } and i got nested fields as a result okay so the type\_ID is the same as value in the other csv file. so what i want is to return the 2 fields name and function of the type\_ID so like i receive random type\_ID and i want to index them with their name and function from the other csv file idk if there is a filter that can help or the ruby filter or translation ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 6, 2019, 2:42pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/2 "2019-02-06T14:42:03Z")
A [translate](https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html) filter would work provided you always have two types. ``` translate { field => "[Type][0][type_ID]" destination => "[Type][0][type_Name]" dictionary_path => "/home/user/foo.csv" } translate { field => "[Type][1][type_ID]" destination => "[Type][1][type_Name]" dictionary_path => "/home/user/foo.csv" } ``` If it is a variable number then I would extract some of the code from the translate filter and implement it in ruby. ---
**Author:** ![fadihaddad](https://avatars.discourse-cdn.com/v4/letter/f/4bbf92/32.png) [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Post date:** [February 6, 2019, 3:07pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/3 "2019-02-06T15:07:52Z")
> [@Badger](#): > > and implement it in ruby. Yeah it is a variable number , i have a lot of Types so can you show me how can i implement it in a ruby filter like showing the type\_name and type\_function for each type ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 6, 2019, 4:24pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/4 "2019-02-06T16:24:11Z")
``` ruby { init => ' @dict = Hash.new CSV.foreach("/home/user/foo.csv", { encoding: "UTF-8", headers: true, header_converters: :symbol, converters: :all}) do |row| @dict[row[0]] = row[1] end ' code => ' t = event.get("Type") t.each_index { |i| t[i]["type_Name"] = @dict[t[i]["type_ID"]] } event.set("Type", t) ' } ``` will give you ``` "Type" => [ [0] { "type_Before" => 15674532, "type_Change" => 50195, "type_ID" => 255, "type_After" => 15724727, "type_Name" => "Foo" }, [1] { "type_Before" => 41089839104, "type_Change" => 131585024, "type_ID" => 256, "type_After" => 41221424128, "type_Name" => "Bar" } ], ``` if the csv contains ```auto "Type_Id","Type_Name" 255,Foo 256,Bar ``` Error handling is left as an exercise for the reader. ---
**Author:** ![fadihaddad](https://avatars.discourse-cdn.com/v4/letter/f/4bbf92/32.png) [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Post date:** [February 6, 2019, 9:38pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/5 "2019-02-06T21:38:51Z")
Thank you so much for your help and i wanted to ask you if we could add 2 @dict to take more than 1 row for example ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 6, 2019, 9:58pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/6 "2019-02-06T21:58:47Z")
I do not understand the question. The init loads every row of the CSV (except the first) into the hash. ---
**Author:** ![fadihaddad](https://avatars.discourse-cdn.com/v4/letter/f/4bbf92/32.png) [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Post date:** [February 7, 2019, 5:01am UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/7 "2019-02-07T05:01:06Z")
Oh so if i want to load for example a second row i add @dict[row[0]] = row[2] ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 7, 2019, 3:00pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/8 "2019-02-07T15:00:17Z")
> [@fadihaddad](#): > > Oh so if i want to load for example a second row i add > > @dict[row[0]] = row[2] No. CSV.foreach iterates over every row of the file. Each row of the csv is passed to the block as an array so ``` @dict[row[0]] = row[1] ``` Adds an entry to the has called @dict which has the key equal to column 1 and the value equal to column 2. As I said, the entire file except for the first row gets added to the hash. ---
**Author:** ![fadihaddad](https://avatars.discourse-cdn.com/v4/letter/f/4bbf92/32.png) [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Post date:** [February 12, 2019, 8:08am UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/9 "2019-02-12T08:08:45Z")
oh sorry my mistake I didn't mean row I meant column if I wanted to make the first column also key to the third column ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 12, 2019, 12:28pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/10 "2019-02-12T12:28:05Z")
No, you cannot. If you have a CSV that looks like this ```auto "Type_Id","Type_Name","Other_Name" 255,Foo,Hello 256,Bar,World ``` then if you run something like ``` CSV.foreach("/home/user/foo.csv", { encoding: "UTF-8", headers: true, header_converters: :symbol, converters: :all}) do |row| @dict[row[0]] = row[1] @dict[row[0]] = row[2] end ``` The second entry overwrites the first, and you end up with a hash just containing ``` {255=>"Hello", 256=>"World"} ``` ---
**Author:** ![fadihaddad](https://avatars.discourse-cdn.com/v4/letter/f/4bbf92/32.png) [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Post date:** [February 12, 2019, 12:56pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/11 "2019-02-12T12:56:58Z")
okay thank you and 1 last question. can I make 2keys for 1 value? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 12, 2019, 1:01pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/12 "2019-02-12T13:01:56Z")
Yes, if you wanted the lookup to be the other ways around, from name to id, then you could do something like ``` @dict[row[1]] = row[0] @dict[row[2]] = row[0] ``` ---
**Author:** ![fadihaddad](https://avatars.discourse-cdn.com/v4/letter/f/4bbf92/32.png) [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Post date:** [February 12, 2019, 1:03pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/13 "2019-02-12T13:03:17Z")
okay thank you ---
**Author:** ![fadihaddad](https://avatars.discourse-cdn.com/v4/letter/f/4bbf92/32.png) [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Post date:** [February 12, 2019, 1:29pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/14 "2019-02-12T13:29:49Z")
if I want to check if this key exists in a column what do I use so it doesn't give me an exception ---
**Author:** ![fadihaddad](https://avatars.discourse-cdn.com/v4/letter/f/4bbf92/32.png) [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Post date:** [February 12, 2019, 2:08pm UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/15 "2019-02-12T14:08:24Z")
well let me tell you my case so i can simplify everything so I have 2 csv files reference\_1.csv and reference\_2.csv plus the one I am indexing I have 2 fields test1 and test2 test1 has { "test1\_ID": "123" etc } test2 is test2{ "test2\_name": "name" etc } the reference\_2.csv has test1\_id test2\_name Description code\_name 12 name1 xxxx xxxx 12 name2 xxxx xxxx 13 name1 xxxx xxxx 13 name2 xxxx xxxx I want to check if test1\_id is available in reference\_2 than I reference to it else I reference to reference\_1 then I want to use test1\_id and test2\_name as keys to take code\_name so I get test1{ "test1\_ID": "123" "code\_name": "codename" etc } ---
**Author:** ![fadihaddad](https://avatars.discourse-cdn.com/v4/letter/f/4bbf92/32.png) [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Post date:** [February 13, 2019, 7:22am UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/16 "2019-02-13T07:22:48Z")
better open a new topic ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 13, 2019, 7:22am UTC](https://discuss.elastic.co/t/logstash-add-field-from-another-input/167292/17 "2019-03-13T07:22:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.