ES gives me 400 for my index template but its seems to be validate

Logstash can connect to the cluster fine and I can query http://mycluster/_template/ so it looks like the 400 is just about something in the template it doesn't like.

I get this error:
2018-02-17T00:35:11,571][INFO ][logstash.outputs.elasticsearch] Installing elasticsearch template to _template/iis-request-log
[2018-02-17T00:35:11,703][ERROR][logstash.outputs.elasticsearch] Failed to install template. {:message=>"Got response code '400' contacting Elasticsearch at URL 'http://mycluster:80/_template/iis-request-log'", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError", ....

Im running ES 6.0

Here is my template, its valid json, not sure how to debug, ES gives me no useful info on what it doesn't like about it:
{
"template": "logstash-iis*",
"settings": {
"number_of_shards": 5,
"number_of_replicas": 1
},
"aliases": {
"iis-request-logs": {}
},
"order": 12,
"mappings": {
"default": {
"dynamic": false,
"_all": {
"enabled": false
},
"properties": {
"@timestamp": {
"type": "date",
"include_in_all": false
},
"@version": {
"type": "keyword",
"include_in_all": false
},
"bytesReceived": {
"type": "long"
},
"bytesSent": {
"type": "long"
},
"clientIP": {
"type": "ip"
},
"cookie": {
"type": "text",
"norms": false
},
"userAgent_device": {
"type": "keyword"
},
"userAgent_name": {
"type": "keyword"
},
"userAgent_os": {
"type": "keyword"
},
"userAgent_os_name": {
"type": "keyword"
},
"geoip": {
"dynamic": "true",
"properties": {
"city_name": {
"type": "keyword"
},
"continent_code": {
"type": "keyword"
},
"country_code2": {
"type": "keyword"
},
"country_code3": {
"type": "keyword"
},
"country_name": {
"type": "keyword"
},
"dma_code": {
"type": "long"
},
"ip": {
"type": "ip"
},
"latitude": {
"type": "half_float"
},
"location": {
"type": "geo_point"
},
"longitude": {
"type": "half_float"
},
"postal_code": {
"type": "keyword"
},
"region_code": {
"type": "keyword"
},
"region_name": {
"type": "keyword"
},
"timezone": {
"type": "keyword"
}
}
},
"host": {
"type": "ip"
},
"log_timestamp": {
"type": "keyword"
},
"message": {
"type": "text",
"norms": false
},
"method": {
"type": "keyword"
},
"port": {
"type": "long"
},
"protocolVersion": {
"type": "keyword"
},
"referer": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"requestHost": {
"type": "keyword"
},
"response": {
"type": "keyword"
},
"serverIP": {
"type": "ip"
},
"serverName": {
"type": "keyword"
},
"serviceName": {
"type": "keyword"
},
"subresponse": {
"type": "keyword"
},
"tags": {
"type": "text",
"norms": false
},
"timetaken": {
"type": "long"
},
"type": {
"type": "keyword"
},
"uriQuery": {
"type": "text",
"norms": false
},
"uriStem": {
"type": "keyword"
},
"userAgent": {
"type": "keyword"
},
"username": {
"type": "keyword"
},
"win32response": {
"type": "keyword"
},
"xForwardedFor": {
"type": "text",
"norms": false
},
"xForwardedForProto": {
"type": "text",
"norms": false
},
"xForwardedForSrc": {
"type": "ip",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
}
}

If I am not wrong in es 6 , this is not template any more but pattern.

Remove the definition to the _all field and all include_in_all attributes.

This should work...

{
  "template": "logstash-iis*",
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "aliases": {
    "iis-request-logs": {}
  },
  "order": 12,
  "mappings": {
    "doc": {
      "dynamic": false,
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "@version": {
          "type": "keyword"
        },
        "bytesReceived": {
          "type": "long"
        },
        "bytesSent": {
          "type": "long"
        },
        "clientIP": {
          "type": "ip"
        },
        "cookie": {
          "type": "text",
          "norms": false
        },
        "userAgent_device": {
          "type": "keyword"
        },
        "userAgent_name": {
          "type": "keyword"
        },
        "userAgent_os": {
          "type": "keyword"
        },
        "userAgent_os_name": {
          "type": "keyword"
        },
        "geoip": {
          "dynamic": "true",
          "properties": {
            "city_name": {
              "type": "keyword"
            },
            "continent_code": {
              "type": "keyword"
            },
            "country_code2": {
              "type": "keyword"
            },
            "country_code3": {
              "type": "keyword"
            },
            "country_name": {
              "type": "keyword"
            },
            "dma_code": {
              "type": "long"
            },
            "ip": {
              "type": "ip"
            },
            "latitude": {
              "type": "half_float"
            },
            "location": {
              "type": "geo_point"
            },
            "longitude": {
              "type": "half_float"
            },
            "postal_code": {
              "type": "keyword"
            },
            "region_code": {
              "type": "keyword"
            },
            "region_name": {
              "type": "keyword"
            },
            "timezone": {
              "type": "keyword"
            }
          }
        },
        "host": {
          "type": "ip"
        },
        "log_timestamp": {
          "type": "keyword"
        },
        "message": {
          "type": "text",
          "norms": false
        },
        "method": {
          "type": "keyword"
        },
        "port": {
          "type": "long"
        },
        "protocolVersion": {
          "type": "keyword"
        },
        "referer": {
          "type": "text",
          "norms": false,
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        },
        "requestHost": {
          "type": "keyword"
        },
        "response": {
          "type": "keyword"
        },
        "serverIP": {
          "type": "ip"
        },
        "serverName": {
          "type": "keyword"
        },
        "serviceName": {
          "type": "keyword"
        },
        "subresponse": {
          "type": "keyword"
        },
        "tags": {
          "type": "text",
          "norms": false
        },
        "timetaken": {
          "type": "long"
        },
        "type": {
          "type": "keyword"
        },
        "uriQuery": {
          "type": "text",
          "norms": false
        },
        "uriStem": {
          "type": "keyword"
        },
        "userAgent": {
          "type": "keyword"
        },
        "username": {
          "type": "keyword"
        },
        "win32response": {
          "type": "keyword"
        },
        "xForwardedFor": {
          "type": "text",
          "norms": false
        },
        "xForwardedForProto": {
          "type": "text",
          "norms": false
        },
        "xForwardedForSrc": {
          "type": "ip",
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        }
      }
    }
  }
}

Hopefully that helps.

Rob

Robert Cowart (rob@koiossian.com)
www.koiossian.com
True Turnkey SOLUTIONS for the Elastic Stack

Thank you!!

I should have stated I used this template with 5.4 previously and I am know trying to use it with 6.0.

I did some googling after your post and found those fields are no longer valid. Thanks for looking at this for me!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.