# Problem with parsing the CSV file correctly **URL:** \ **Category:** Logstash\ **Created:** [October 12, 2019, 2:55pm UTC](https://discuss.elastic.co/t/problem-with-parsing-the-csv-file-correctly/203320 "2019-10-12T14:55:37Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Rysiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rysiu/32/61919_2.png) [@Rysiu](https://discuss.elastic.co/u/Rysiu)\ **Post date:** [October 12, 2019, 2:55pm UTC](https://discuss.elastic.co/t/problem-with-parsing-the-csv-file-correctly/203320/1 "2019-10-12T14:55:38Z")
Hello, I have a problem with parsing the CSV file in Logstash. The structure of my template in index looks like this: ``` { "mappings": { "doc": { "properties": { "ip_src": { "type": "ip" }, "ip_dst": { "type": "ip" }, "hops_no": { "type": "byte" }, "v_route": { "type": "nested", "properties": { "vip": { "type": "ip" }, "vdelay": { "type": "float" } } }, "m_time": { "type": "date", "format": "yyyy-MM-dd HH:mm:ss||epoch_second" } } } }, "version": 1 } ``` An example of one line from an input file is: ``` 89.73.142.84;89.73.142.84;"2002-12-12 12:12:12";3;89.73.142.84@0.1#89.73.142.85@0.3#89.73.142.86@0.5 ``` The contents of the Logstash configuration file is: ``` input { file { path => "/home/user/data/data_pre/*_0" start_position => "beginning" sincedb_path => "/dev/null" file_completed_action => "delete" file_completed_log_path => "/home/user/data/data_post/archive" } } filter { csv { separator => ";" columns => ["ip_src", "ip_dst", "m_time", "hops_no", "vhops"] } ruby { code => ' pb = event.get("vhops"); b = pb.split("#"); ary = Array.new; for c in b; keyvar = c.split("@")[0]; valuevar = c.split("@")[1]; d = "{vip : " << keyvar << ", vdelay : " << valuevar << "}"; ary.push(d); end; event.set("v_route", ary); ' } mutate { remove_field => ["message", "vhops"] } } output { elasticsearch { hosts => ["http://192.168.X.Y:9200"] index => "index00" } stdout {} } ``` There is a problem with the v\_route field. In Logstash logs I have: ``` ... [2019-10-12T14:22:10,380][INFO][logstash.pipeline] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#"} [2019-10-12T14:22:10,429][INFO][filewatch.observingtail] START, creating Discoverer, Watch with file and sincedb collections [2019-10-12T14:22:10,429][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]} [2019-10-12T14:22:10,699][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600} [2019-10-12T14:22:11,319][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"index00", :_type=>"doc", :routing=>nil}, #], :response=>{"index"=>{"_index"=>"routes_ipv4", "_type"=>"doc", "_id"=>"i3pZwG0BJFV1L_wCJ0wp", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [v_route] tried to parse field [null] as object, but found a concrete value"}}}} ``` I'm building the v\_route field wrong? Where is the problem? My Logstash & ES has version 6.8. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 12, 2019, 3:26pm UTC](https://discuss.elastic.co/t/problem-with-parsing-the-csv-file-correctly/203320/2 "2019-10-12T15:26:45Z")
vroute is an array of strings, which looks wrong to me. ``` "v_route" => [ [0] "{vip : 89.73.142.84, vdelay : 0.1}", [1] "{vip : 89.73.142.85, vdelay : 0.3}", [2] "{vip : 89.73.142.86, vdelay : 0.5}" ], ``` What did you intend it to be? ---
**Author:** ![Rysiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rysiu/32/61919_2.png) [@Rysiu](https://discuss.elastic.co/u/Rysiu)\ **Post date:** [October 12, 2019, 4:48pm UTC](https://discuss.elastic.co/t/problem-with-parsing-the-csv-file-correctly/203320/3 "2019-10-12T16:48:11Z")
Oh you're right. I wanted to keep the data about this field in the [nested](https://www.elastic.co/guide/en/elasticsearch/reference/6.8/nested.html) datatype - array of objects with the values vip (ip type) and vdelay (float type). Could you tell me how to modify this code? My field in the template: ``` "v_route": { "type": "nested", "properties": { "vip": { "type": "ip" }, "vdelay": { "type": "float" } } }, ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 12, 2019, 4:59pm UTC](https://discuss.elastic.co/t/problem-with-parsing-the-csv-file-correctly/203320/4 "2019-10-12T16:59:34Z")
If you change the loop to be ``` for c in b; keyvar = c.split("@")[0]; valuevar = c.split("@")[1]; h = Hash.new h["vip"] = keyvar h["vdelay"]= valuevar ary << h end; ``` You will get an array of hashes ``` "v_route" => [ [0] { "vip" => "89.73.142.84", "vdelay" => "0.1" }, [1] { "vip" => "89.73.142.85", "vdelay" => "0.3" }, [2] { "vip" => "89.73.142.86", "vdelay" => "0.5" } ], ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 9, 2019, 4:59pm UTC](https://discuss.elastic.co/t/problem-with-parsing-the-csv-file-correctly/203320/5 "2019-11-09T16:59:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.