No result when searching .raw field in Kibana

Elastic Stack Kibana
1

Hello,

I'm using Kibana 4 with ElasticSearch 2.3.5 to store and search logs. I have a "Message" field which is analyzed and a "Message.raw" field which is not analyzed, this is configured in the mapping template.

From the Settings tab in Kibana I can see the Message.raw field is exist.

Settings_Kibana.png1234×768 41.6 KB

But when I search for "Message.raw" field I can't get any result.

Pasted image1586×663 68.3 KB

Pasted image1095×392 49.7 KB

My template mapping looks like

{
  "template": "aa*",
  "settings": {
    "index": {
      "index": {
        "translog": {
          "sync_interval": "30s",
          "durability": "async"
        },
        "requests": {
          "cache": {
            "enable": "true"
          }
        }
      },
      "refresh_interval": "30s"
    }
  },
  "mappings": {
    "usage": {
      "_source": {
        "enabled": false
      },
      "_all": {
        "enabled": false
      },
      "properties": {
        "Type": {
          "index": "not_analyzed",
          "type": "string",
          "doc_values": true
        },
        "Message": {
          "norms": {
            "enabled": false
          },
          "index": "analyzed",
          "type": "string",
          "fields": {
            "raw": {
              "ignore_above": 1024,
              "index": "not_analyzed",
              "type": "string"
            }
          }
        },
        "Zone": {
          "index": "not_analyzed",
          "type": "string",
          "doc_values": true
        },
        "Hostname": {
          "index": "not_analyzed",
          "type": "string",
          "doc_values": true
        },
        "Levelname": {
          "index": "not_analyzed",
          "type": "string",
          "doc_values": true
        },
        "Process": {
          "index": "not_analyzed",
          "type": "string",
          "doc_values": true
        },
        "Time": {
          "index": "not_analyzed",
          "type": "string",
          "doc_values": true
        },
        "Logger": {
          "index": "not_analyzed",
          "type": "string",
          "doc_values": true
        },
        "Timestamp": {
          "format": "strict_date_optional_time||epoch_millis",
          "type": "date",
          "doc_values": true
        },
        "Lineno": {
          "index": "not_analyzed",
          "type": "string",
          "doc_values": true
        },
        "Name": {
          "index": "not_analyzed",
          "type": "string",
          "doc_values": true
        }
      }
    }
  },
  "aliases": {}
}

Do you know what I'm doing wrong here and how can I fix this?

Thanks.

2

Did you add the Message.raw field to the mapping after you had already indexed data?

3

Yes, I added the Message.raw field after the index was created. But the index is created daily, I think the mapping setting should be working for the new indices. Is this the issue? Should I reindex the old indices or delete all the indices will work?

My indices

aa-mktint-common_log-2016.09.12
aa-mktint-common_log-2016.09.11
aa-mktint-common_log-2016.09.14
aa-mktint-common_log-2016.09.13
aa-mktint-common_log-2016.09.10
4

Documents/indexes that were created after the mapping changed should be searchable, so I don't think that's the issue.

I created an index with the mapping you supplied, and indexed some documents into it. I was able to find them in kibana. So, I'm not sure what the issue is yet.

When you add the Message.raw: * query, can you expand the spy panel and post the request body?

5

The request body from Visualize tab

{
  "size": 0,
  "query": {
    "filtered": {
      "query": {
        "query_string": {
          "analyze_wildcard": true,
          "query": "Message.raw: *"
        }
      },
      "filter": {
        "bool": {
          "must": [
            {
              "range": {
                "Timestamp": {
                  "gte": 1474263189601,
                  "lte": 1474264089601,
                  "format": "epoch_millis"
                }
              }
            }
          ],
          "must_not": []
        }
      }
    }
  },
  "aggs": {
    "3": {
      "terms": {
        "field": "Message.raw",
        "size": 20,
        "order": {
          "_count": "desc"
        }
      }
    }
  }
}