# Keep getting this error indexing into Elastic

**URL:** <https://discuss.elastic.co/t/keep-getting-this-error-indexing-into-elastic/182603>\
**Category:** Logstash\
**Created:** [May 24, 2019, 9:37am UTC](https://discuss.elastic.co/t/keep-getting-this-error-indexing-into-elastic/182603 "2019-05-24T09:37:50Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![bigster](https://avatars.discourse-cdn.com/v4/letter/b/8e7dd6/32.png) [@bigster](https://discuss.elastic.co/u/bigster)\
**Post date:** [May 24, 2019, 9:37am UTC](https://discuss.elastic.co/t/keep-getting-this-error-indexing-into-elastic/182603/1 "2019-05-24T09:37:50Z")

</div>

Hi all,

I've got this logstash conf of consuming a JMS channel, do some transformations, and indexing into an Elastic Cluster.

But i keep getting this error:

[2019-05-24T10:29:43,516][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>"4b6d4e72-cd20-44ba-b3e8-ad78bae28cd6", :\_index=\>"eai\_prod\_log\_v2-2019.05.24", :\_type=\>"doc", :routing=\>nil}, #LogStash::Event:0xfec8b3e], :response=\>{"index"=\>{"\_index"=\>"index-2019.05.24", "\_type"=\>"doc", "\_id"=\>"4b6d4e72-cd20-44ba-b3e8-ad78bae28cd6", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse field [client\_id] of type [long] in document with id '4b6d4e72-cd20-44ba-b3e8-ad78bae28cd6'", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"For input string: "%{[logEnrich][0][client\_id]}""}}}}}

```
input {
	jms {
		destination => "QUEUE"
		yaml_file => "D:\mq.yml"
		yaml_section => "section2"
	}	
}
filter {
	xml{
        source => "message"
        store_xml => false
		remove_namespaces => true
		force_array => false
        xpath => [
            "/LogMessage/TransactionID/text()", "transaction_id",
			"/LogMessage/ClientID/text()", "client_id",
		"/LogMessage/ServiceName/text()", "service_name",
			"/LogMessage/ServiceKey/text()", "service_key"
        ]
    }

	############################ Enrich the log with DB Data ##############################
	if [service_key] {
	
		mutate { 
			gsub => [
				"service_key", "&amp;", "&",
				"service_key", "&lt;", "<",
				"service_key", "&gt;", ">"
			]
		}
		
		jdbc_streaming {
			jdbc_driver_library => "..\db2jcc.jar"
			jdbc_driver_class => "com.ibm.db2.jcc.DB2Driver"
			jdbc_connection_string => "Database"
			jdbc_user => "user"
			jdbc_password => "pwd"
			cache_expiration => 10.0
			parameters => { "serviceKey" => "service_key" }
			parameters => { "serviceName" => "service_name" }
			statement => "
				SELECT CLIENT_ID, APPLICATION_ID, CLIENT_NAME
				FROM TABLE
				WHERE SECRET_KEY = :serviceKey AND APPLICATION_NAME = :serviceName"
			
			target => "logEnrich"
		}
	} else {
		jdbc_streaming {
			jdbc_driver_library => "..\db2jcc.jar"
			jdbc_driver_class => "com.ibm.db2.jcc.DB2Driver"
			jdbc_connection_string => "Database"
			jdbc_user => "user"
			jdbc_password => "pwd"
			parameters => { "serviceName" => "service_name" }
			statement => "
				SELECT 0 AS CLIENT_ID, APPLICATION_ID, '' AS CLIENT_NAME
				FROM TABLE
				WHERE APPLICATION_NAME = :serviceName "
				
			target => "logEnrich"
		}
	}
	
	mutate {
		add_field => {
			"client_id" => "%{[logEnrich][0][client_id]}"
			"application_id" => "%{[logEnrich][0][application_id]}"
			"client_name" => "%{[logEnrich][0][client_name]}"
		}
	}
	
	if [client_id] == "%{[logEnrich][0][client_id]}}" {
		mutate { replace => ["client_id", "0"] }
	}
	
	if [application_id] == "%{[logEnrich][0][application_id]}" {
		mutate { replace => ["application_id", "0"] }
	}
	
	if [client_name] == "%{[logEnrich][0][client_name]}" {
		mutate { replace => ["client_name", ""] }
	}	
	########################################################################################
	
	############################ Creating the Unique ID for doc ############################
	uuid {
		target => "uuid"
		overwrite => true
	}
	########################################################################################

	############################ CLEANING ##############################
	mutate { remove_field => "logEnrich" }
	mutate { remove_field => "service_key" }
	####################################################################
}
output {
    elasticsearch {
        hosts => ["elasticp01:9200", "elasticp02:9200"]
		index => "index-%{+YYYY.MM.dd}"
		document_id =>"%{uuid}"
    }

	if [client_id] == "%{[logEnrich][0][client_id]}}" {
		file {
				path => "D:\debug-PROD-%{+YYYY-MM-dd}.log"
				codec => rubydebug
		}
	}
}

```

The error tell that the client\_id field has invalid values, but how can it be based in the conf above?  
Anyone can see where is the mistake?

Cheers,

---

<div class="post-metadata">

**Author:** ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)\
**Post date:** [May 24, 2019, 10:02am UTC](https://discuss.elastic.co/t/keep-getting-this-error-indexing-into-elastic/182603/2 "2019-05-24T10:02:49Z")

</div>

Do you have a float and integer on the same field client\_id maybe?

---

<div class="post-metadata">

**Author:** ![bigster](https://avatars.discourse-cdn.com/v4/letter/b/8e7dd6/32.png) [@bigster](https://discuss.elastic.co/u/bigster)\
**Post date:** [May 24, 2019, 10:51am UTC](https://discuss.elastic.co/t/keep-getting-this-error-indexing-into-elastic/182603/3 "2019-05-24T10:51:53Z")

</div>

I believe not.  
The client\_id is obtained from the database.  
In some cases i can't obtained the client\_id, so i set the client\_id with the value 0.  
That's it.

Based on the error it seems that the set of the value is not done and it keeps the value %{[logEnrich][0][client\_id]}

Cheers,

---

<div class="post-metadata">

**Author:** ![logger](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@logger](https://discuss.elastic.co/u/logger)\
**Post date:** [May 24, 2019, 10:57am UTC](https://discuss.elastic.co/t/keep-getting-this-error-indexing-into-elastic/182603/4 "2019-05-24T10:57:51Z")

</div>

Hi there,

Maybe you have a „}“ too much after client\_id:  
if [client\_id] == "%{[logEnrich][0][client\_id]}}" {  
mutate { replace =\> ["client\_id", "0"] }  
}

---

<div class="post-metadata">

**Author:** ![bigster](https://avatars.discourse-cdn.com/v4/letter/b/8e7dd6/32.png) [@bigster](https://discuss.elastic.co/u/bigster)\
**Post date:** [May 24, 2019, 1:48pm UTC](https://discuss.elastic.co/t/keep-getting-this-error-indexing-into-elastic/182603/5 "2019-05-24T13:48:03Z")

</div>

You are correct, but i think its not the problem, because i remove the extra "}" and i still getting the error:

[2019-05-24T13:11:03,453][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch.  
{  
: status=\>400,  
: action=\>[  
"index",  
{  
: \_id=\>"897d3edd-b307-4fb8-86a5-40ede5f2103e",  
: \_index=\>"eai\_prod\_log\_v2-2019.05.24",  
: \_type=\>"doc",  
: routing=\>nil  
},  
#\<LogStash: : Event: 0x2684b01f\>  
],  
: response=\>{  
"index"=\>{  
"\_index"=\>"eai\_prod\_log\_v2-2019.05.24",  
"\_type"=\>"doc",  
"\_id"=\>"897d3edd-b307-4fb8-86a5-40ede5f2103e",  
"status"=\>400,  
"error"=\>{  
"type"=\>"mapper\_parsing\_exception",  
"reason"=\>"failed to parse field [client\_id] of type [long] in document with id '897d3edd-b307-4fb8-86a5-40ede5f2103e'",  
"caused\_by"=\>{  
"type"=\>"illegal\_argument\_exception",  
"reason"=\>"For input string: "%{[logEnrich][0][client\_id]}""  
}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![logger](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@logger](https://discuss.elastic.co/u/logger)\
**Post date:** [May 24, 2019, 2:29pm UTC](https://discuss.elastic.co/t/keep-getting-this-error-indexing-into-elastic/182603/6 "2019-05-24T14:29:32Z")

</div>

hmm,

maybe try this one:

```
if [client_id] =~ /logEnrich][0][client_id {
	mutate { replace => ["client_id", "0"] }
}

```

this will check if the substring is inside [client\_id] and if true it will replace it with 0

---

<div class="post-metadata">

**Author:** ![bigster](https://avatars.discourse-cdn.com/v4/letter/b/8e7dd6/32.png) [@bigster](https://discuss.elastic.co/u/bigster)\
**Post date:** [May 28, 2019, 2:31pm UTC](https://discuss.elastic.co/t/keep-getting-this-error-indexing-into-elastic/182603/7 "2019-05-28T14:31:05Z")

</div>

Hi there,

I found the problem, indeed the problem was the type of data that client\_id field has.  
Initially the client\_id was filled with the value "1", default value present in the XML (that i don't know about).  
When i add the field client\_id with the value obtained from the DB the field was with the value [1, 100] and this is not valid for the type of Elastic, that is a long not an array.

Removed the initial default value solves the problem.

Cheers,

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 25, 2019, 2:31pm UTC](https://discuss.elastic.co/t/keep-getting-this-error-indexing-into-elastic/182603/8 "2019-06-25T14:31:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.