Type keyword stills appearing as text


(Damian Correa) #1

Hi guys. I need help.

My goal is to have 2 fields as type "keyword", meaning no text type. I want to have these fields like core data, in my case I want this for processid and jobid
Here is my logstash/conf.d/
input {
beats {
port => ####
}
}

filter {

    grok    {
            patterns_dir => "/etc/logstash/patterns/"
            break_on_match => false
            keep_empty_captures => false

            match => { "message" => [ "%{SYSLOGBASE} %{JOBID:jobid} %{PROCESSID:processid} %{GREEDYDATA:syslog_message}"
                                    ]
                     }
        }
           if ![syslog_message] or [syslog_message] == "" { drop {}}

}

output {

elasticsearch {
hosts => ["myURLxxxxxx:9200"]
index => "logstash-%{+YYYY.MM.dd}"

}
#stdout {}
}

Now here is part of my patterns config

                    "logsource": {
                        "fields": {
                            "keyword": {
                                "ignore_above": 256,
                                "type": "keyword"
                            }
                        },
                        "norms": false,
                        "type": "text"
                    },
                    "message": {
                        "norms": false,
                        "type": "text"
                    },
                    "offset": {
                        "type": "long"
                    },
                    "processid": {
                        "fields": {
                            "keyword": {
                                "ignore_above": 256,
                                "type": "keyword"
                            }
                        },
                        "norms": false,
                        "type": "keyword"
                    },
                    "program": {
                        "fields": {
                            "keyword": {
                                "ignore_above": 256,
                                "type": "keyword"
                            }
                        },
                        "norms": false,
                        "type": "text"
                    },
                    "prospector": {
                        "properties": {
                            "type": {
                                "fields": {
                                    "keyword": {
                                        "ignore_above": 256,
                                        "type": "keyword"
                                    }
                                },
                                "norms": false,
                                "type": "text"
                            }
                        }
                    },
"source": {
                        "fields": {
                            "keyword": {
                                "ignore_above": 256,
                                "type": "keyword"
                            }
                        },
                        "norms": false,
                        "type": "text"
                    },
                    "syslog_message": {
                        "fields": {
                            "keyword": {
                                "ignore_above": 256,
                                "type": "keyword"
                            }
                        },
                        "norms": false,
                        "type": "keyword"
                    },
                    "tags": {
                        "fields": {
                            "keyword": {
                                "ignore_above": 256,
                                "type": "keyword"
                            }
                        },
                        "norms": false,
                        "type": "text"
                    },
                    "timestamp": {
                        "fields": {
                            "keyword": {
                                "ignore_above": 256,
                                "type": "keyword"
                            }
                        },
                        "norms": false,
                        "type": "text"
                    }
                }
            }
        }
    }

############################
Here is my template conf.
{
"template": "mytemplate",
"version": "1001",
"settings": {
"index.refresh_interval": "5s"
},
"mappings" : {
"default" : {
"properties" : {
"jobid" : { "type" : "keyword", "index": "not_analyzed" },
"processid" : { "type" : "keyword", "index": "not_analyzed" },
"syslog_message" : { "type" : "keyword", "index": "not_analyzed" },
}
}
}
}

So as result: by going to kibana > management > elasticSearch Management > select index > "Under Mapping" =
the fields processid and jobid are still present as text.

 "jobid": {
      "type": "text",
      "norms": false,
      "fields": {
        "keyword": {
          "type": "keyword",
          "ignore_above": 256
        }
      }
    },

Here is a sample of core data:

"pipeline": {
"properties": {
"batch_size": {
"type": "integer"
},
"ephemeral_id": {
"type": "keyword"
},
"format": {
"type": "keyword"
},
"hash": {
"type": "keyword"
},
"id": {
"type": "keyword"
},
"representation": {
"type": "object",
"enabled": false
},
"version": {
"type": "keyword"
},
"workers": {
"type": "short"
}
}
},

Please help...


(Damian Correa) #2

Can someone please help :sob:


(system) closed #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.