# How to map and index the specific json object into elastic search **URL:** \ **Category:** Logstash\ **Created:** [July 18, 2019, 1:14pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204 "2019-07-18T13:14:23Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![manjudr](https://avatars.discourse-cdn.com/v4/letter/m/aeb1de/32.png) [@manjudr](https://discuss.elastic.co/u/manjudr)\ **Post date:** [July 18, 2019, 1:14pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/1 "2019-07-18T13:14:24Z")
Let's I'm reading data from the kafka topic and indexing specific data into specific elastic search index. Example : Logstash reading message from the kafka topic i has two messages(1. LOG, 2.ERROR) Message1. //{ "eid": "LOG", "ets": 1528400654594, "ver": "3.0", "errorInfo":{} } { "eid": "ERROR", "ets": 1528400654594, "ver": "3.0", "errorInfo":{} } Now I need to index these two message into elasticsearch using logstash message1 should go **"LOG-Index"** and message2 should go to **"ERROR-Index"** //input { kafka { bootstrap\_servers =\> "localhost:9092" topics =\> ["local.telemetry.log"] type =\> "local.telemetry.log" codec =\> "json" } } output { elasticsearch { codec =\> "json" hosts =\> ["[http://localhost:9200](http://localhost:9200)"] index =\> "%{type}-indexer" user =\> "elastic" password =\> "changeme" } } ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 18, 2019, 1:17pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/2 "2019-07-18T13:17:02Z")
> [@manjudr](#): > > index =\> "%{type}-indexer" Shouldn't that be "%{eid}-index"? ---
**Author:** ![manjudr](https://avatars.discourse-cdn.com/v4/letter/m/aeb1de/32.png) [@manjudr](https://discuss.elastic.co/u/manjudr)\ **Post date:** [July 18, 2019, 1:25pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/3 "2019-07-18T13:25:33Z")
How will we get eid? in this '"%{eid}-index"' ---
**Author:** ![manjudr](https://avatars.discourse-cdn.com/v4/letter/m/aeb1de/32.png) [@manjudr](https://discuss.elastic.co/u/manjudr)\ **Post date:** [July 18, 2019, 1:26pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/4 "2019-07-18T13:26:05Z")
Should we need to use any filter ? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 18, 2019, 1:32pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/5 "2019-07-18T13:32:43Z")
> [@manjudr](#): > > "eid": "LOG", Your kafka messages are JSON, and you are using a json codec on the input so if your JSON contains ``` "eid": "LOG", ``` I would expect your events to have an eid field. ---
**Author:** ![manjudr](https://avatars.discourse-cdn.com/v4/letter/m/aeb1de/32.png) [@manjudr](https://discuss.elastic.co/u/manjudr)\ **Post date:** [July 18, 2019, 1:37pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/6 "2019-07-18T13:37:47Z")
Yes 'eid ' will be available So you mean when i use `codec` then we should able to parse the message right ? And One more clarification since **eid** value is in uppercase can we able create an index name with lowercase without modifying the message ? For example : Something like this possible ? // '"%{eid}.toLowercase()-index"' ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 18, 2019, 1:50pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/7 "2019-07-18T13:50:28Z")
If you are OK with changing eid then use ``` mutate { lowercase => ["eid"] } ``` If not, make a copy and change that. ``` mutate { copy => { "eid" => "[@metadata][eid]" } } mutate { lowercase => ["[@metadata][eid]" ] } ``` Then you can use ``` index => "%{[@metadata][eid]}-index" ``` ---
**Author:** ![manjudr](https://avatars.discourse-cdn.com/v4/letter/m/aeb1de/32.png) [@manjudr](https://discuss.elastic.co/u/manjudr)\ **Post date:** [July 18, 2019, 1:56pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/8 "2019-07-18T13:56:32Z")
Thank you ...@Badger ---
**Author:** ![manjudr](https://avatars.discourse-cdn.com/v4/letter/m/aeb1de/32.png) [@manjudr](https://discuss.elastic.co/u/manjudr)\ **Post date:** [July 22, 2019, 9:00am UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/9 "2019-07-22T09:00:46Z")
Hi, In the logstash output, Is it possible to mention index name with mapping object file path ? ``` output { elasticsearch { codec => "json" hosts => ["http://localhost:9200"] index => "%{[@metadata][eid]}-index" user => "elastic" password => "changeme" document_type => "_doc" template => "../config/mappingObj.json" template_name => "%{[@metadata][eid]}-*" }} ``` ---
**Author:** ![manjudr](https://avatars.discourse-cdn.com/v4/letter/m/aeb1de/32.png) [@manjudr](https://discuss.elastic.co/u/manjudr)\ **Post date:** [July 22, 2019, 10:14am UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/10 "2019-07-22T10:14:01Z")
Logstash 2.3 Mutate don't have `copy` function ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 22, 2019, 12:24pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/11 "2019-07-22T12:24:19Z")
Does it have add\_field? If so, use mutate { add\_field =\> { "[@metadata][eid]" =\> "%{eid}" } } ---
**Author:** ![manjudr](https://avatars.discourse-cdn.com/v4/letter/m/aeb1de/32.png) [@manjudr](https://discuss.elastic.co/u/manjudr)\ **Post date:** [July 22, 2019, 12:26pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/12 "2019-07-22T12:26:03Z")
Thanks, I have resolved with the if and else condition ---
**Author:** ![manjudr](https://avatars.discourse-cdn.com/v4/letter/m/aeb1de/32.png) [@manjudr](https://discuss.elastic.co/u/manjudr)\ **Post date:** [July 22, 2019, 12:28pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/13 "2019-07-22T12:28:18Z")
And in elastic search mapping is it possible to handle the `BigDecimal`? Becoz we are facing this error `"error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [time_field]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"1.56379850611E+12\" is malformed at \".56379850611E+12\""}}}}` **mapping object** "time\_field": { "format": "strict\_date\_optional\_time||epoch\_millis", "type": "date" }, Any idea? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 22, 2019, 1:11pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/14 "2019-07-22T13:11:02Z")
The mapper will not support BigDecimal because the underlying [Java class](https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html) does not. Use ruby to convert it ``` ruby { code => 'event.set("date", BigDecimal.new(event.get("someField")).to_i)' } date { match => ["date", UNIX_MS] } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 19, 2019, 1:11pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204/15 "2019-08-19T13:11:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.