# Could not index event to Elasticsearch in ES 6.0.0

**URL:** <https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325>\
**Category:** Elasticsearch\
**Created:** [December 12, 2017, 11:56am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325 "2017-12-12T11:56:58Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [December 12, 2017, 11:56am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/1 "2017-12-12T11:56:59Z")

</div>

I'm using ES 6.0 and beat 6.0.

I check in logstash-plain.log, i got many error like this

> [2017-12-12T18:30:25,655][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"fb-fim-dns-6.0.1-2017.50", :\_type=\>"doc", :\_routing=\>nil}, #LogStash::Event:0x113962d4], :response=\>{"index"=\>{"\_index"=\>"fb-fim-dns-6.0.1-2017.50", "\_type"=\>"doc", "\_id"=\>"DMF9SmAB\_Dli1qCHfj8b", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"object mapping for [geoip.location] tried to parse field [null] as object, but found a concrete value"}}}}

> [2017-12-13T01:14:27,072][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"fb-fim-mail-6.0.1-2017.50", :\_type=\>"doc", :\_routing=\>nil}, #LogStash::Event:0x69b5aa0d], :response=\>{"index"=\>{"\_index"=\>"fb-fim-mail-6.0.1-2017.50", "\_type"=\>"doc", "\_id"=\>"rh\_vS2ABGJRfL5NeixeC", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse [geoip.location]", "caused\_by"=\>{"type"=\>"i\_o\_exception", "reason"=\>"Current token (START\_OBJECT) not numeric, can not use numeric value accessors\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@58d69e2; line: 1, column: 905]"}}}}}

> [2017-12-13T00:51:04,074][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"wb-fim-terminal-5.6.0-2017.50", :\_type=\>"wineventlog", :\_routing=\>nil}, #LogStash::Event:0x5002cce5], :response=\>{"index"=\>{"\_index"=\>"wb-fim-terminal-5.6.0-2017.50", "\_type"=\>"wineventlog", "\_id"=\>"CBTaS2ABGJRfL5NeE75m", "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Rejecting mapping update to [wb-fim-terminal-5.6.0-2017.50] as the final mapping would have more than 1 type: [doc, wineventlog]"}}}}

What should i do?. How to slove my problem? Thanks!

I try to add mapping for `geoip` but not work

Here is my index template [Filebeat teamplate](https://pastebin.com/ymEdTgPF)

---

<div class="post-metadata">

**Author:** ![ericrichter](https://avatars.discourse-cdn.com/v4/letter/e/898d66/32.png) [@ericrichter](https://discuss.elastic.co/u/ericrichter)\
**Post date:** [December 12, 2017, 8:09pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/2 "2017-12-12T20:09:45Z")

</div>

logstash 6.0.1 should fix this issue

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [December 13, 2017, 2:17am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/3 "2017-12-13T02:17:32Z")

</div>

Thank @ericrichter, i will try to update V6.0.1 and report againt.

---

<div class="post-metadata">

**Author:** ![mujtabahussain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mujtabahussain/32/17514_2.png) [@mujtabahussain](https://discuss.elastic.co/u/mujtabahussain)\
**Post date:** [December 13, 2017, 3:09am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/4 "2017-12-13T03:09:07Z")

</div>

> [@tatdat](#):
>
> "failed to parse [geoip.location]"

Have you had a look at the data you are sending through?

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [December 13, 2017, 3:22am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/5 "2017-12-13T03:22:35Z")

</div>

Sure,  
My index template worked in ES 5.6.x but when i update to 6.0.0, i got this issue.  
I used geoip filter plugin, translate filter plugin. Some data of geoip will be pushed to geoip field

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [December 13, 2017, 7:31am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/6 "2017-12-13T07:31:57Z")

</div>

Can someone help me.

I updated ES, LS to 6.0.1, still got error

> [2017-12-13T14:26:49,427][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"fb-fim-mail-6.0.1-2017.50", :\_type=\>"doc", :\_routing=\>nil}, #LogStash::Event:0x8e3cde7], :response=\>{"index"=\>{"\_index"=\>"fb-fim-mail-6.0.1-2017.50", "\_type"=\>"doc", "\_id"=\>"dPbFTmAB8SyTVy-3c\_PI", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse [geoip.location]", "caused\_by"=\>{"type"=\>"i\_o\_exception", "reason"=\>"Current token (START\_OBJECT) not numeric, can not use numeric value accessors\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@24cd9e54; line: 1, column: 833]"}}}}}  
> [2017-12-13T14:26:49,427][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"fb-fim-mail-6.0.1-2017.50", :\_type=\>"doc", :\_routing=\>nil}, #LogStash::Event:0x22890712], :response=\>{"index"=\>{"\_index"=\>"fb-fim-mail-6.0.1-2017.50", "\_type"=\>"doc", "\_id"=\>"dfbFTmAB8SyTVy-3c\_PI", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse [geoip.location]", "caused\_by"=\>{"type"=\>"i\_o\_exception", "reason"=\>"Current token (START\_OBJECT) not numeric, can not use numeric value accessors\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@38b1c1ca; line: 1, column: 799]"}}}}}

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 13, 2017, 7:44am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/7 "2017-12-13T07:44:34Z")

</div>

Did you follow @mujtabahussain advice?

> [@mujtabahussain](#):
>
> Have you had a look at the data you are sending through?

Could you print with a stdout output and json or rubydebug codec what is supposed to be indexed by elasticsearch?

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [December 13, 2017, 7:59am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/8 "2017-12-13T07:59:02Z")

</div>

> [@dadoonet](#):
>
> Could you print with a stdout output and json or rubydebug codec what is supposed to be indexed by elasticsearch?

Some output here

```
{
           "source" => "C:\\inetpub\\logs\\LogFiles\\W3SVC1\\u_ex171205.log",
             "type" => "iis-access",
       "http.value" => "exchangecookie=7e1608b9803a40c1ac80e497a023dad4;+dtCookie=D3845DF09C18E5F80F291479A07643AB|X2RlZmF1bHR8MQ",
         "hostname" => "HCAS01-SRV",
         "beattype" => "doc",
         "clientip" => "10.3.50.215",
         "@version" => "1",
             "beat" => {
            "name" => "HCAS01-srv",
        "hostname" => "HCAS01-srv",
         "version" => "6.0.1"
    },
             "host" => "HCAS01-srv",
    "http.response" => "200",
      "http.method" => "POST",
       "user_agent" => {
              "os" => "Mac OS X",
           "major" => "5",
           "minor" => "0",
           "build" => "",
        "os_minor" => "10",
        "os_major" => "10",
            "name" => "ExchangeWebServices",
         "os_name" => "Mac OS X",
          "device" => "Other"
    },
        "timestamp" => "2017-12-05 06:38:25",
     "http.request" => "/autodiscover/autodiscover.xml",
            "geoip" => {},
           "offset" => 1181493910,
        "http.port" => "443",
         "beatname" => "fb-fim-mail",
        "http.site" => "W3SVC1",
       "prospector" => {
        "type" => "log"
    },
          "message" => "2017-12-05 06:38:25 W3SVC1 HCAS01-SRV 10.4.11.62 POST /autodiscover/autodiscover.xml - 443 company.com\\tuanna20 10.3.50.215 HTTP/1.1 Mac+OS+X/10.10.5+(14F2511);+ExchangeWebServices/5.0+(213);+Mail/8.2+(2104) exchangecookie=7e1608b9803a40c1ac80e497a023dad4;+dtCookie=D3845DF09C18E5F80F291479A07643AB|X2RlZmF1bHR8MQ 200",
             "tags" => [],
       "@timestamp" => 2017-12-13T02:22:38.513Z,
           "locaIp" => "10.4.11.62",
         "username" => "company.com\\tuanna20",
       "http.agent" => "Mac+OS+X/10.10.5+(14F2511);+ExchangeWebServices/5.0+(213);+Mail/8.2+(2104)"
}
{
           "source" => "C:\\inetpub\\logs\\LogFiles\\W3SVC1\\u_ex171205.log",
             "type" => "iis-access",
       "http.value" => "-",
         "hostname" => "HCAS01-SRV",
         "beattype" => "doc",
         "clientip" => "59.153.244.39",
         "@version" => "1",
             "beat" => {
            "name" => "HCAS01-srv",
        "hostname" => "HCAS01-srv",
         "version" => "6.0.1"
    },
             "host" => "HCAS01-srv",
    "http.response" => "200",
      "http.method" => "POST",
       "user_agent" => {
           "name" => "Mobile Safari UI/WKWebView",
        "os_name" => "Other",
             "os" => "Other",
         "device" => "Other",
          "build" => ""
    },
        "timestamp" => "2017-12-05 06:38:26",
     "http.request" => "/Microsoft-Server-ActiveSync/default.eas",
            "geoip" => {
             "city_name" => "Hanoi",
              "timezone" => "Asia/Ho_Chi_Minh",
                    "ip" => "59.153.244.39",
              "latitude" => 21.0333,
          "country_name" => "Vietnam",
         "country_code2" => "VN",
        "continent_code" => "AS",
         "country_code3" => "VN",
           "region_name" => "Thanh Pho Ha Noi",
              "location" => {
            "lon" => 105.85,
            "lat" => 21.0333
        },
           "region_code" => "HN",
             "longitude" => 105.85
    },
           "offset" => 1181500150,
        "http.port" => "443",
         "beatname" => "fb-fim-mail",
        "http.site" => "W3SVC1",
       "prospector" => {
        "type" => "log"
    },
          "message" => "2017-12-05 06:38:26 W3SVC1 HCAS01-SRV 10.4.11.62 POST /Microsoft-Server-ActiveSync/default.eas User=lucqv@fpt.com.vn&DeviceId=341IARRU2961T2Q5G4H7GMCH5O&DeviceType=iPhone&Cmd=Sync&Log=V141_Fc1_Fid:4_Ty:Em_Filt5_St:S_Sk:2047955523_Sst5_SsCmt5_Cli:0a0c0d1f0e_BR1_BPR0_LdapC1_LdapL15_RpcC20_RpcL59_Pk820662046_S1_As:AllowedG_Mbx:MB-02-SRV.HO.FPT.VN_Throttle0_Budget:(A)Conn%3a0%2cHangingConn%3a0%2cAD%3a%24null%2f%24null%2f0%25%2cCAS%3a%24null%2f%24null%2f1%25%2cAB%3a%24null%2f%24null%2f0%25%2cRPC%3a%24null%2f%24null%2f1%25%2cFC%3a1000%2f0%2cPolicy%3aDefaultThrottlingPolicy%5F56c53095-e0c2-4ad3-a65a-c54c057050bf%2cNorm_ 443 lucqv@company.com 59.153.244.39 HTTP/1.1 Apple-iPhone7C2/1502.202 - 200",
             "tags" => [],
       "@timestamp" => 2017-12-13T02:22:38.515Z,
           "locaIp" => "10.4.11.62",
         "username" => "lucqv@company.com",
       "http.agent" => "Apple-iPhone7C2/1502.202"
}

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 13, 2017, 10:30am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/9 "2017-12-13T10:30:06Z")

</div>

Could you do the same with a json codec so we can try to index it manually and see why this is failing?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 13, 2017, 10:37am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/10 "2017-12-13T10:37:30Z")

</div>

> [@tatdat](#):
>
> "geoip" =\> {},

It looks like `geoip` is an empty object, which is causing problems. What does your full configuration look like?

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [December 13, 2017, 2:53pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/11 "2017-12-13T14:53:15Z")

</div>

> [@tatdat](#):
>
> source

Hi @dadoonet, some log with a json codec

[https://pastebin.com/HqsG99ja](https://pastebin.com/HqsG99ja)

> [@Christian\_Dahlqvist](#):
>
> It looks like geoip is an empty object, which is causing problems. What does your full configuration look like?

For IIS access (type=iis-access)

```
filter {
if [type] == "iis-access" {
    grok {
      match => { "message" => "%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:site} %{WORD:method} %{URIPATH:page} %{NOTSPACE:querystring} %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clientip} %{NOTSPACE:agent} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:scstatus} %{NUMBER:bytes}" }
    }
    grok {
      match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{WORD:http.site} %{NOTSPACE:hostname} %{IPORHOST:locaIp} %{WORD:http.method} %{URIPATH:http.request} %{NOTSPACE:http.msg} %{INT:http.port} %{DATA:username} %{IPORHOST:clientip} %{DATA} %{DATA:http.agent} %{DATA:http.value} %{NUMBER:http.response}" }
    }
    grok {
      match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{WORD:http.method} %{URIPATH:http.request} %{NOTSPACE:http.msg} %{DATA:username} %{IPORHOST:clientip} %{DATA:http.agent} %{DATA:http.value} %{NUMBER:http.response}" }
    }
    geoip {
      source => "clientip"
      target => "geoip"
      database => "/etc/logstash/GeoLite2-City.mmdb"
    }
    useragent {
      source=> "agent"
      target=> "user_agent"
    }
    useragent {
      source=> "http.agent"
      target=> "user_agent"
    }
    translate {
      regex => true
      dictionary_path => "/etc/logstash/translates/internal-ip.yaml"
      field => "clientip"    
    }
    json {
      source => "translation"
      remove_field => ["translation"]
    }
    date {
        match => ["timestamp" , "dd/MMM/YYYY:HH:mm:ss Z"]
    }
    mutate {
        remove_tag => ["beats_input_codec_plain_applied","_grokparsefailure","_geoip_lookup_failure","_dateparsefailure"]
	remove_field => ["log_timestamp","http.msg"]
    }
  }
}

```

With public IP, im using GeoIP filter, with internal IP, im using transale with content look like: **internal-ip.yaml**

```
'10.1.11.164': '{"geoip": {"unit": "FESFB01", "unit_desc": "FESFB01", "city_name": "Hanoi", "country_name": "Vietnam", "latitude": 21.033, "longitude": 105.85, "location": [105.85, 21.033]}}'
'10.1.11.165': '{"geoip": {"unit": "FESFB02", "unit_desc": "FESFB02", "city_name": "Hanoi", "country_name": "Vietnam", "latitude": 21.033, "longitude": 105.85, "location": [105.85, 21.033]}}'

```

I think there is a problem with my transate file? but in version 5.x, it worked without problem.

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [December 14, 2017, 6:21am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/12 "2017-12-14T06:21:51Z")

</div>

Yeah, im still waiting for you.  
Because , i can't get log with log problem. ☹

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 14, 2017, 6:54am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/13 "2017-12-14T06:54:59Z")

</div>

I would recommend configuring a stdout output with a rubydebug codec, so we can clearly see the structure of the event as it leaves Logstash.

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [December 14, 2017, 9:46am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/14 "2017-12-14T09:46:49Z")

</div>

@Christian_Dahlqvist  
I sent logs output with rubydebug and json codec in previous post, check that pls.

Update: I try to disable translate plugin, but still got error with geoip.location index

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 14, 2017, 9:55am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/15 "2017-12-14T09:55:14Z")

</div>

Have you tried using conditionals to select either group plugin or translate plugin depending on what the IP address starts with? Do you have a matching entry in your translate file?

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [December 14, 2017, 10:02am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/16 "2017-12-14T10:02:46Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> Have you tried using conditionals to select either group plugin or translate plugin depending on what the IP address starts with?

Some case i have, with internal IP, i'll use translate file. But some case, i dont use filter for that.  
But this is example of translate file

```
'10.1.11.164': '{"geoip": {"unit": "FESFB01", "unit_desc": "FESFB01", "city_name": "Hanoi", "country_name": "Vietnam", "latitude": 21.033, "longitude": 105.85, "location": [105.85, 21.033]}}'
'10.1.11.165': '{"geoip": {"unit": "FESFB02", "unit_desc": "FESFB02", "city_name": "Hanoi", "country_name": "Vietnam", "latitude": 21.033, "longitude": 105.85, "location": [105.85, 21.033]}}'

```

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [December 14, 2017, 4:24pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/17 "2017-12-14T16:24:19Z")

</div>

Solved!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 11, 2018, 4:24pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-in-es-6-0-0/111325/18 "2018-01-11T16:24:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.