# Mutate add\_field attribute accidently become duplicate when using multiple logstash

**URL:** <https://discuss.elastic.co/t/mutate-add-field-attribute-accidently-become-duplicate-when-using-multiple-logstash/309536>\
**Category:** Logstash\
**Created:** [July 13, 2022, 1:52pm UTC](https://discuss.elastic.co/t/mutate-add-field-attribute-accidently-become-duplicate-when-using-multiple-logstash/309536 "2022-07-13T13:52:31Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mahendrapratitos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mahendrapratitos/32/99690_2.png) [@mahendrapratitos](https://discuss.elastic.co/u/mahendrapratitos)\
**Post date:** [July 13, 2022, 1:52pm UTC](https://discuss.elastic.co/t/mutate-add-field-attribute-accidently-become-duplicate-when-using-multiple-logstash/309536/1 "2022-07-13T13:52:31Z")

</div>

I tried to execute 8 logstash that accept different port and different filebeat.  
I run this logstash on Linux environment.  
when I run the logstash, turns out some attribute that I create by using script mutate add\_field duplicates 8 times.  
the values that I created manually are

- LogLevel
- logtype
- table\_alias  
here's the output  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/4/f48df9c6d05cb8b54231d0ca4fbca1db03c32270.png)

```auto
log.file.path
D:\Temenos\env\t24ibsm\bnk.run\XMLdriver.log
	
log.flags
multiline
	
log.offset
15,018,814
	
LogLevel
INFO, INFO, INFO, INFO, INFO, INFO, INFO, INFO
	
logtype
Temenos_XMLdriver, Temenos_XMLdriver, Temenos_XMLdriver, Temenos_XMLdriver, Temenos_XMLdriver, Temenos_XMLdriver, Temenos_XMLdriver, Temenos_XMLdriver
	
nama_produk_dan_versi
R10.97389 - 7576 - (jmainfunction.b,0)
	
rawMessage
ErrorRecord: HResult: 0x80004005Description: New transaction is not allowed because there are other threads running in the session.SQLErrorInfo: 42000Source: Microsoft SQL Server Native Client 10.0File: dbProcess.cpp, Line: 3267

```

I used 8 different input port  
here's my input script

```auto
input {
  beats {
    port => 5047
  }
}

```

here're my mutate add\_field script

```auto
if "ERROR" in [rawMessage]
			{
				mutate {
					add_field => { "LogLevel" => "ERROR" }
					
				}
			}
			else if "deadlock" in [rawMessage] or "WARNING" in [rawMessage]
			{
				mutate {
					add_field => { "LogLevel" => "WARN" }				
				}
			}
			else 
			{
				mutate {
					add_field => { "LogLevel" => "INFO"}				
					}
			}
			mutate 	
			{
				remove_field => ["category3"]
				remove_field => ["dashgetter"]
				remove_field => ["day","month","monthday","time","HOUR","MINUTE","SECOND"]
				add_field => { "logtype" => "menos_XMLdriver" }
				#add_field => {"category4_category5" => %[category4," ",category5]}
				add_field => {"table_alias" => "%{table} - %{alias}"}
				#add_field => {"hostname" => "menos01"}
			}
		
			if ![alias] and ![table]
			{
				mutate{
					remove_field => ["table_alias"]
					}
			}
			else if ![table]
			{
				mutate {
					gsub => ["table_alias", "%{table} -", ""]
					}
			} 
			else if ![alias]
			{ 
				mutate{
					gsub => ["table_alias", " - %{alias}", ""]
					}
			}

```

if logstash only running 1 input it's working perfectly  
but when running 8 inputs the manual field that I created using mutate add\_field becomes duplicate in the output.

Please Help

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [July 13, 2022, 2:47pm UTC](https://discuss.elastic.co/t/mutate-add-field-attribute-accidently-become-duplicate-when-using-multiple-logstash/309536/2 "2022-07-13T14:47:02Z")

</div>

Please share a sample of your original message and your entire pipeline that is giving you this error, with just part of the pipeline is not possible to try to replicate.

---

<div class="post-metadata">

**Author:** ![mahendrapratitos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mahendrapratitos/32/99690_2.png) [@mahendrapratitos](https://discuss.elastic.co/u/mahendrapratitos)\
**Post date:** [July 22, 2022, 11:23am UTC](https://discuss.elastic.co/t/mutate-add-field-attribute-accidently-become-duplicate-when-using-multiple-logstash/309536/3 "2022-07-22T11:23:48Z")

</div>

Solved, I have to configure pipeline for each input (Ip)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 19, 2022, 11:24am UTC](https://discuss.elastic.co/t/mutate-add-field-attribute-accidently-become-duplicate-when-using-multiple-logstash/309536/4 "2022-08-19T11:24:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.