Thanks... I appreciate you patience and detail, and I am happy to provide an example / solutions, I have just learned that we will both get to and answer quicker if I have a good understanding exactly what you are looking for... your initial post was not enough detail.
To add a runtime field that you want to search on you have to add it to the mappings see here and some examples... and remember it is added as a field not to the _source so you will be able to search on it, use it in visualizations and access but it will not be in the _source json
My index is named
discuss-openvpn
I indexed your data into the message field and then I added this
This is THE most Brute Force way to do this ... there are probably more elegant ways.. but as you said you just wanted and example.
This basically will parse use the grok and for the lines that do not match the username field is not created for those that do.. username will be emitted
PUT discuss-openvpn/_mappings
{
"runtime": {
"username": {
"type": "keyword",
"script": """
String username=grok('%{GREEDYDATA:leading_data}/CN=%{DATA:username}\'').extract(doc["message"].value)?.username;
if (username != null) emit(username);
"""
}
}
}
Then when I search for this and note it is in the fields not _source
GET discuss-openvpn/_search
{
"fields": [
"*"
],
"query": {
"exists": {
"field": "username"
}
}
}
#results
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "discuss-openvpn",
"_id": "uYPrgIIBkiWX4SWKF193",
"_score": 1,
"_source": {
"@timestamp": "2022-08-09T00:57:30.000Z",
"message": "2019-09-23T17:49:32+0000 [stdout#info] [OVPN 1] OUT: 'Mon Sep 23 17:49:32 2019 Hide-my-ip:7305 VERIFY OK: depth=1, /CN=OpenVPN CA'"
},
"fields": {
"@timestamp": [
"2022-08-09T00:57:30.000Z"
],
"message": [
"2019-09-23T17:49:32+0000 [stdout#info] [OVPN 1] OUT: 'Mon Sep 23 17:49:32 2019 Hide-my-ip:7305 VERIFY OK: depth=1, /CN=OpenVPN CA'"
],
"username": [
"OpenVPN CA"
]
}
},
{
"_index": "discuss-openvpn",
"_id": "u4PrgIIBkiWX4SWKoF-7",
"_score": 1,
"_source": {
"@timestamp": "2022-08-09T00:57:30.000Z",
"message": "2019-09-23T17:49:32+0000 [stdout#info] [OVPN 1] OUT: 'Mon Sep 23 17:49:32 2019 Hide-my-ip:7305 VERIFY OK: depth=0, /CN=lan-with-inet_AUTOLOGIN'"
},
"fields": {
"@timestamp": [
"2022-08-09T00:57:30.000Z"
],
"message": [
"2019-09-23T17:49:32+0000 [stdout#info] [OVPN 1] OUT: 'Mon Sep 23 17:49:32 2019 Hide-my-ip:7305 VERIFY OK: depth=0, /CN=lan-with-inet_AUTOLOGIN'"
],
"username": [
"lan-with-inet_AUTOLOGIN"
]
}
}
]
}
}
And it also shows up in Discover
