No raw fields, not using template

Elastic Stack Logstash
1

I saw the question come by several times but here I am, same problem, everything seems to be in order but I'm not getting my raw fields :frowning:

I haven't changed anything about the index naming in the Elasticsearch Logstash output. My index names look like logstash-2016.10.30

If I do a GET /_template I see the following output:

{
  "logstash": {
    "order": 0,
    "template": "logstash-*",
    "settings": {
      "index": {
        "refresh_interval": "5s"
      }
    },
    "mappings": {
      "_default_": {
        "dynamic_templates": [
          {
            "message_field": {
              "mapping": {
                "index": "analyzed",
                "omit_norms": true,
                "fielddata": {
                  "format": "disabled"
                },
                "type": "string"
              },
              "match_mapping_type": "string",
              "match": "message"
            }
          },
          {
            "string_fields": {
              "mapping": {
                "index": "analyzed",
                "omit_norms": true,
                "fielddata": {
                  "format": "disabled"
                },
                "type": "string",
                "fields": {
                  "raw": {
                    "index": "not_analyzed",
                    "ignore_above": 256,
                    "type": "string"
                  }
                }
              },
              "match_mapping_type": "string",
              "match": "*"
            }
          }
        ],
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "geoip": {
            "dynamic": true,
            "properties": {
              "location": {
                "type": "geo_point"
              },
              "longitude": {
                "type": "float"
              },
              "latitude": {
                "type": "float"
              },
              "ip": {
                "type": "ip"
              }
            }
          },
          "@version": {
            "index": "not_analyzed",
            "type": "string"
          }
        },
        "_all": {
          "enabled": true,
          "omit_norms": true
        }
      }
    },
    "aliases": {}
  }
}

But if I look at the mapping of a logstash index, it says

{
  "logstash-2016.11.01": {
    "mappings": {
      "collectd": {
        "properties": {
          "@timestamp": {
            "type": "date",
            "format": "strict_date_optional_time||epoch_millis"
          },
          "@version": {
            "type": "string"
          },
          "collectd_type": {
            "type": "string"
          },
          "host": {
            "type": "string"
          },
          "longterm": {
            "type": "double"
          },
          "midterm": {
            "type": "double"
          },
          "plugin": {
            "type": "string"
          },
          "plugin_instance": {
            "type": "string"
          },
          "rx": {
            "type": "long"
          },
          "shortterm": {
            "type": "double"
          },
          "tags": {
            "type": "string"
          },
          "tx": {
            "type": "long"
          },
          "type": {
            "type": "string"
          },
          "type_instance": {
            "type": "string"
          },
          "value": {
            "type": "long"
          }
        }
      },
...

No not_analyzed strings :scream: Any idea what could be wrong?

2

Which version of Elasticsearch are you sending to?

Have you checked the log files for Elasticsearch? It seems strange that the template is there, but the mapping is not. It seems there should be a logged reason why that is the case.

3

It seems like a fluke. I got fed up with it so I restarted Elasticsearch, deleted today's index and it created it with the right mapping. I think that "fixed" it :slight_smile: Thanks for your reply though