Script field for search msg

[Ali4 (ALI)]

Hello,

Please i need your help.

I need to create script field to search msg "error" in field message.

the field message is:
'''
{"@timestamp":"2020-07-20T12:14:43+02:00","@version":"1","message":" time="Jul 20 12:14:43" level=debug msg=Finished code=403 error="No token provided" mw=CoProcessMiddleware ns=1981606 "severity":"debug","facility":"kern","programname":""}
'''
i created
if (doc['message.keyword'].value.contains ("error") {
return "Error"
}

its not works

Help please

[tylersmalley]

First of all, I would highly recommend setting this during index time as each document needs analyzed using this method.

If you're OK with the performance cost here, this can be done.

Here is my test data:

DELETE /discuss-241879

PUT /discuss-241879
{
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 0
    }
  },
  "mappings": {
    "properties": {
      "message": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}

POST /discuss-241879/_doc
{
    "@timestamp" : "July 20th 2020, 16:17:55.029",
    "message" : "This is an error"
}

POST /discuss-241879/_doc
{
    "@timestamp" : "July 20th 2020, 16:18:55.029",
    "message" : "This was a success"
}

I have then created an index pattern on discuss-241879 with the following scriped field:

if (doc.containsKey('message.keyword') && 
    doc['message.keyword'].value.contains("error")) {
  return true
}

return false

What error were you getting?

[Ali4 (ALI)]

Hello,

Thank you for your response.

I shared wih you my output json.

{
"_index": "tyk_gateway_prod-2020-07-21",
"_type": "doc",
"_version": 1,
"_score": null,
"_source": {
"tags": [
"_grokparsefailure"
],
"@version": "1",
"type": "syslog",
"host": "X.X.X.X",
"@timestamp": "2020-07-21T07:33:20.441Z",
"message": "{"@timestamp":"2020-07-21T09:33:20+02:00","@version":"1","message":" time=\\\"Jul 21 09:33:20\\\" level=debug msg=Finished api_id=xxxxxxxxxxxxxxx api_name=xxxxxxxxxxxx code=403 error=\\\"No token provided\\\" mw=CoProcessMiddleware ns=2075904 org_id=xxxxxxx origin=xxxxxxxxx path=xxxxxxxxx "@sysloghost":"xxxxxxxxxx","severity":"debug","facility":"kern**"}**\n"
},
"fields": {
"@timestamp": [
"2020-07-21T07:33:20.441Z"
]
},
"sort": [
1595316800441
]
}

i have message on elasticsearch : Fielddata is disabled on text fields by default. Set fielddata=true on [message] in order to load fielddata in memory by uninverting the inverted index.

the problem is the field "message" in to "_source"

Have you idea how to enabled this field please.

I tried with :
PUT tyk_gateway_prod
{

"mappings": {
"properties": {
"_source": {
"message": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}

but i have message error:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Mapping definition for [_source] has unsupported parameters: [message : {type=text, fields={keyword={ignore_above=256, type=keyword}}}]"
}
],

Thank you :