Append to a property if exists

Hello,

Input csv looks like :

 "","Code","Date","Time","Open","High","Low","Close","Volume"
 "1","3IINFOTECH",20150703,"09:16:00",5.55,4.55,4.55,4.55,835
 "2","3IINFOTECH",20150703,"09:17:00",5.55,4.55,4.55,4.55,390 

Logstash config is :

input {
  file {
    path => "/Users/sbezgoan/Documents/Elastic/StockData/nse-company-stocks/three.csv"
    start_position => "beginning"
    sincedb_path => "/Users/sbezgoan/Documents/Tools/logstash-7.5.1/temp.log"
  }
}
filter {
  csv {
      separator => ","
     columns => ["SNo","Code", "Date", "Time", "Open","High", "Low", "Close","Volume"]
     skip_header => true
  }
  mutate {
     add_field => {"DateTime" =>  "%{Date} %{Time}"}
}
 mutate {
     remove_field => ["%{Time}", "%{Date}"]
}
date{
  match => ["DateTime" , "YYYYMMdd HH:mm:ss"]
}
}
output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "nse-stocks2"
     action => "update"
     doc_as_upsert => "true"
     document_id => "%{[SNo]}"
     manage_template => "true"
     script => 'if (ctx._source.Open == null) { ctx._source.Open = new ArrayList();} if(ctx._source.Open != null) {ctx._source.Open.add("%{[Open]}");}'
  }
stdout {}
}

The aim is if a document with the same ID already exists, then append to the Open field new incoming values creating an array of Open values.

The script doesn't seem to be doing the intended, appreciate any help on how I can achieve Open field mapping as a multi-valued array.

Also, I don't like the fact that I am allowing logstash to manage templates (elastic mappings).

I think the issue is the way I am retrieving the existing _source, but not sure how to fix.

I have followed the discussion at

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.