Hi guys. I need help.
My goal is to have 2 fields as type "keyword", meaning no text type. I want to have these fields like core data, in my case I want this for processid and jobid
Here is my logstash/conf.d/
input {
beats {
port => ####
}
}
filter {
grok {
patterns_dir => "/etc/logstash/patterns/"
break_on_match => false
keep_empty_captures => false
match => { "message" => [ "%{SYSLOGBASE} %{JOBID:jobid} %{PROCESSID:processid} %{GREEDYDATA:syslog_message}"
]
}
}
if ![syslog_message] or [syslog_message] == "" { drop {}}
}
output {
elasticsearch {
hosts => ["myURLxxxxxx:9200"]
index => "logstash-%{+YYYY.MM.dd}"
}
#stdout {}
}
Now here is part of my patterns config
"logsource": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"norms": false,
"type": "text"
},
"message": {
"norms": false,
"type": "text"
},
"offset": {
"type": "long"
},
"processid": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"norms": false,
"type": "keyword"
},
"program": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"norms": false,
"type": "text"
},
"prospector": {
"properties": {
"type": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"norms": false,
"type": "text"
}
}
},
"source": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"norms": false,
"type": "text"
},
"syslog_message": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"norms": false,
"type": "keyword"
},
"tags": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"norms": false,
"type": "text"
},
"timestamp": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"norms": false,
"type": "text"
}
}
}
}
}
############################
Here is my template conf.
{
"template": "mytemplate",
"version": "1001",
"settings": {
"index.refresh_interval": "5s"
},
"mappings" : {
"default" : {
"properties" : {
"jobid" : { "type" : "keyword", "index": "not_analyzed" },
"processid" : { "type" : "keyword", "index": "not_analyzed" },
"syslog_message" : { "type" : "keyword", "index": "not_analyzed" },
}
}
}
}
So as result: by going to kibana > management > elasticSearch Management > select index > "Under Mapping" =
the fields processid and jobid are still present as text.
"jobid": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
Here is a sample of core data:
"pipeline": {
"properties": {
"batch_size": {
"type": "integer"
},
"ephemeral_id": {
"type": "keyword"
},
"format": {
"type": "keyword"
},
"hash": {
"type": "keyword"
},
"id": {
"type": "keyword"
},
"representation": {
"type": "object",
"enabled": false
},
"version": {
"type": "keyword"
},
"workers": {
"type": "short"
}
}
},
Please help...
