# ADD FIELD OVERWRITES DATA

**URL:** <https://discuss.elastic.co/t/add-field-overwrites-data/244008>\
**Category:** Logstash\
**Created:** [August 6, 2020, 11:03am UTC](https://discuss.elastic.co/t/add-field-overwrites-data/244008 "2020-08-06T11:03:52Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Daniel\_Lopez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_lopez/32/50639_2.png) [@Daniel\_Lopez](https://discuss.elastic.co/u/Daniel_Lopez)\
**Post date:** [August 6, 2020, 11:03am UTC](https://discuss.elastic.co/t/add-field-overwrites-data/244008/1 "2020-08-06T11:03:52Z")

</div>

Hi guys,

I'm trying to add data in a field like an array, but my grok or mutate overwrites it, could someone take a look?

This is my code:

`filter {  
if [message] =~ "TargetID:" {  
drop{}  
}  
grok {  
patterns\_dir =\> ["/usr/share/logstash/patterns"]  
break\_on\_match =\> false  
match =\> {

```
		"message" =>[
			"%{DATESTAMP:timing}",
			"%{FIRSTLINE:firstline}",
			"%{EVENTPD:eventpd}"
	 
		]
	}
	tag_on_failure => []
 }

    
 if [firstline]{
	grok {
	
		break_on_match => true
		match => ["firstline","%{WORD:test}\s%{WORD:test}\s%{WORD:test}\s%{WORD:test}\s%{WORD:test}\s\[%{USERNAME:sessionId}\]\s%{WORD:test}\s%{WORD:test}\s%{WORD:test}\s\[\[%{WORD:eventCodetmp}"]
			
		
	}
mutate{
	
		#remove_field => ["firstline"]
		remove_field => ["message"]
		remove_field => ["@version"]
		remove_field => ["timing"]
        remove_field => ["test"]
        add_field => ["eventCode" , "%{eventCodetmp}"]
        #remove_field => ["eventCodetmp"]
		
		
		
		rename => {"@timestamp" => "timestamp"}
        rename => {"timing" => "startTime"}
		}
	

 	}

```

date  
{  
locale =\> "es"  
match =\> ["startTime","dd/MM/YY HH:mm:ss.SSS", "ISO8601"]  
timezone =\> "Europe/Vienna"  
target =\> "startTime"  
}

```
 if [eventpd]{
    
	grok {
    break_on_match => true
	match => ["eventpd","SessionID\:\s%{USERNAME:sessionId}\]\s\#\#\#\sEvent\s\[%{WORD:eventCodetmp1}"]
 		
	

	}
		mutate{
		#remove_field => ["eventpd"]
		remove_field => ["message"]
		remove_field => ["@version"]
		remove_field => ["timing"]
		add_field => ["eventCode" , "%{eventCodetmp1}"]
        #remove_field => ["eventCodetmp"]
		rename => {"@timestamp" => "timestamp"}
        rename => {"timing" => "startTime"}
		}
	}
	
 }	`

```

The tmp variables are being fill with correct data, but always, when eventpd happends it overwrite the value of eventCode instead of add the value as array

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 6, 2020, 1:54pm UTC](https://discuss.elastic.co/t/add-field-overwrites-data/244008/2 "2020-08-06T13:54:36Z")

</div>

Use mutate+merge rather than mutate+add\_field

BTW, remove\_field take an array, so you can do

```
		remove_field => ["message", "@version", "timing"]

```

Similarly for

```
rename => {
    "@timestamp" => "timestamp"
    "timing" => "startTime"
}

```

Generally I recommend against supplying an option to a filter more than once, since logstash will sometimes do very unexpected things when merging the contents of the two options.

---

<div class="post-metadata">

**Author:** ![Daniel\_Lopez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_lopez/32/50639_2.png) [@Daniel\_Lopez](https://discuss.elastic.co/u/Daniel_Lopez)\
**Post date:** [August 7, 2020, 9:12am UTC](https://discuss.elastic.co/t/add-field-overwrites-data/244008/3 "2020-08-07T09:12:12Z")

</div>

I fotgot to say that I used update option to elastic, I use a personal id (sessionId) to update the documents, that is the reason because it is always updating the eventCode value, not adding info, overwriting it. I tried merge , and it is always updating with only one of the results, for example eventCode : %{eventCodetmp}, VALUE or VALUE, %{eventCodetmp1}

any idea for a workaroung? Maybe with ruby but i never used it ☹

---

<div class="post-metadata">

**Author:** ![Daniel\_Lopez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_lopez/32/50639_2.png) [@Daniel\_Lopez](https://discuss.elastic.co/u/Daniel_Lopez)\
**Post date:** [August 19, 2020, 9:57am UTC](https://discuss.elastic.co/t/add-field-overwrites-data/244008/4 "2020-08-19T09:57:46Z")

</div>

I could manage with script in output config

if (ctx.\_source.eventCode == null) {ctx.\_source.eventCode= new ArrayList(); ctx.\_source.eventCode.add("%{[eventCode]}")} else {ctx.\_source.eventCode.add("%{[eventCode]}")}

but i sometimes a get this error in log

dynamic method [java.lang.String, add/1 not found

Any help?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 16, 2020, 9:57am UTC](https://discuss.elastic.co/t/add-field-overwrites-data/244008/5 "2020-09-16T09:57:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.