# Logstash and CSV - append to field while upsert
**URL:** https://discuss.elastic.co/t/logstash-and-csv-append-to-field-while-upsert/247071
**Category:** Logstash
**Created:** [September 1, 2020, 9:50am UTC](https://discuss.elastic.co/t/logstash-and-csv-append-to-field-while-upsert/247071 "2020-09-01T09:50:06Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@AnthonyA](https://discuss.elastic.co/u/AnthonyA)
#### Post date: [September 1, 2020, 9:50am UTC](https://discuss.elastic.co/t/logstash-and-csv-append-to-field-while-upsert/247071/1 "2020-09-01T09:50:06Z")
Hi there 😀
I'm using logstash. My input is a CSV file. I'm using one of the columns as the document\_id. There are lines in the CSV file that have the same document\_id.
In my output, I'm using upsert. When Logstash performs an update, I want it to append one of the fields instead of replacing the existing one.
This is my Logstash file:
```auto
input {
file {
path => "/home/anthony/10.txt"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => "|"
skip_header => "true"
columns => ["type_identifiant_pp","identifiant_pp","identification_nationale_pp","code_civilite_exercice","libelle_civilite_exercice","libelle_commune"]
}
}
output {
stdout {}
amazon_es {
hosts => ["XXXXXX.eu-west-3.es.amazonaws.com"]
region => "eu-west-3"
aws_access_key_id => 'XXXXX'
aws_secret_access_key => 'XXXXX'
index => "index-002"
document_id => "%{identification_nationale_pp}"
doc_as_upsert => "true"
}
}
```
I tried to use a script like this one:
`script => "ctx._source.libelle_commune += params.event.get('libelle_commune')"`
And also like the ones described in these topics:
> [@Logstash Elasticsearch ouput: Append array while upserting](https://discuss.elastic.co/t/logstash-elasticsearch-ouput-append-array-while-upserting/70010/):
>
> I am upserting documents: new fields should be added and existing fields should be replaced by the new documents field value. With one exception: the value of text field 'shodan.protocols' should always be appended instead of replaced. This field should contain multiple unique string values. My logstash output looks like this: elasticsearch { index =\> "test" document\_id =\> "%{ip}" doc\_as\_upsert =\> true …
> [@Logstash create doc if not exist else append array](https://discuss.elastic.co/t/logstash-create-doc-if-not-exist-else-append-array/153262):
>
> Elasticsearch: 6.4.2 Logstash: 6.4.2 Goal: Index the data if it hasn't been indexed before, otherwise, if the document does exist, then append an array. Input data round 1: { "item": ["1"] } Input data round 2: { "item": ["2"] } Desired data in Elasticsearch: { "item": ["1","2"] } The below will index the first time the document is seen or append "item" if it has not. However, it will always append even if the value is in the array. e.g. if the below is ran three times in a row, the out…
> [@Append to a property if exists](https://discuss.elastic.co/t/append-to-a-property-if-exists/214889):
>
> Hello, Input csv looks like : "","Code","Date","Time","Open","High","Low","Close","Volume" "1","3IINFOTECH",20150703,"09:16:00",5.55,4.55,4.55,4.55,835 "2","3IINFOTECH",20150703,"09:17:00",5.55,4.55,4.55,4.55,390 Logstash config is : input { file { path =\> "/Users/sbezgoan/Documents/Elastic/StockData/nse-company-stocks/three.csv" start\_position =\> "beginning" sincedb\_path =\> "/Users/sbezgoan/Documents/Tools/logstash-7.5.1/temp.log" } } filter { csv { separator =\> …
> [@Add an array using Mutate's add\_field](https://discuss.elastic.co/t/add-an-array-using-mutates-add-field/200946):
>
> Hi Guys, I have a logstash pipeline where I am receiving a JSON file as HTTP input and forwarding it to output plugin. I want to introduce below structure to input JSON : "parentField": { "field0": "value0", "arrayName": [{ "field1": "value1", "field2": "value2" }] } To achieve that I am trying to use below filter :: filter { mutate { add\_field =\> { "[parentField][field0]" =\> "value0" } add\_field =\> { "[parentField][arrayName][0][field1]" =\> "value1" } add\_field =\> { "[parentF…
I've spent a lot of time on this. I read a lot of documentation, but I must admit I'm stuck here.
Could someone give me a hint? 🙇♂️ Thanks!
Anthony
---
### Author:  [@AnthonyA](https://discuss.elastic.co/u/AnthonyA)
#### Post date: [September 2, 2020, 9:23am UTC](https://discuss.elastic.co/t/logstash-and-csv-append-to-field-while-upsert/247071/2 "2020-09-02T09:23:14Z")
Hi,
I also tried a script like this one:
```auto
script => 'if (ctx._source.libelle_commune == null) { ctx._source.libelle_commune = new ArrayList();} if(ctx._source.libelle_commune != null) {ctx._source.libelle_commune.add("%{[libelle_commune]}");}'
```
To no avail 😒
Has anyone an idea or something I could try?
---
### Author:  [@AnthonyA](https://discuss.elastic.co/u/AnthonyA)
#### Post date: [September 3, 2020, 3:26pm UTC](https://discuss.elastic.co/t/logstash-and-csv-append-to-field-while-upsert/247071/3 "2020-09-03T15:26:40Z")
It looks like my syntax is wrong and should be like that but this does not work:
```auto
script_lang => "painless"
script_type => "inline"
script => 'if (!ctx._source.containsKey("libelle_commune")) ctx._source.libelle_commune = new ArrayList(); ctx._source.libelle_commune.add("test");'
```
---
### Author:  [@AnthonyA](https://discuss.elastic.co/u/AnthonyA)
#### Post date: [September 4, 2020, 7:47am UTC](https://discuss.elastic.co/t/logstash-and-csv-append-to-field-while-upsert/247071/4 "2020-09-04T07:47:10Z")
I can upsert and use a script with PHP. This is some working code:
```auto
$params['body'] = [
"script" => [
'lang' => 'painless',
'source' => 'if (!ctx._source.containsKey("libelle_commune")) ctx._source.libelle_commune = new ArrayList(); ctx._source.libelle_commune.add(params.libelle_commune);',
'params' => [
'libelle_commune' => $l
]
],
"upsert" => [
'nom_exercice' => $row['nom_exercice'],
'prenom_exercice' => $row['prenom_exercice'],
'libelle_commune' => [$row['libelle_commune']]
]
];
```
But I can't get it working using logstash. Can anyone help me translate the query written in PHP into an output for logstash? 🖖🏻
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 2, 2020, 7:47am UTC](https://discuss.elastic.co/t/logstash-and-csv-append-to-field-while-upsert/247071/5 "2020-10-02T07:47:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.