I have a message index pattern which has the below string
[INFO][callback][views][2020-03-18 13:06:21][xce_web][PROD][user__id_1010510-18-03-2020-13-03-26][167]:For ABC start time is 2020-03-18 13:03:26.999778 & end time is 2020-03-18 13:06:21.507542 and pipeline takes 174507 milliseconds to complete.
How to get the substring i.e user_id and 174507 in 2 diferent script fields.
Thanks for the reply. I tried using the below script
def path = doc['message.keyword'].value;
if (path != null) {
int lastSlashIndex = path.IndexOf('[doc');
if (lastSlashIndex > 0) {
return path.substring(lastSlashIndex,lastSlashIndex +10);
}
}
return "";
But all in vain after saving and going to discover.it is showing the warning This field is present in your elasticsearch mapping but not in any documents in the search results. You may still be able to visualize or search on it.
Based on the input, I slightly modified the script:
def path = doc['message.keyword'].value;
String userIdPrefix = '[user__id_';
int userIdLen = 7;
if (path != null) {
int lastSlashIndex = path.indexOf(userIdPrefix);
if (lastSlashIndex > 0) {
return path.substring(lastSlashIndex + userIdPrefix.length(), lastSlashIndex + userIdPrefix.length() + userIdLen);
}
}
return "";
With your input, that returns 1010510 as the user_id. I didn't see '[doc' in your sample input, so I replaced it with [user__id_ and I assumed your userId was always length 7.
IMPORTANT if you are modifying your scripted field in the kibana scripted field editor, you MUST refresh discover before it will use the updated script.
Kibana caches the scripted fields for an index when you load Discover.
It seems the message field is of text type. Will this script will work on this.
You'd have to change def path = doc['message.keyword'].value;
to def path = params['_source']['message'];
if field data is disabled. This change accesses the _source to get at the raw data.
However, if field data is enabled (and depending how it is tokenized), then doc['message'] will be a list of strings, so you'll have to look for values in that start with user__id_. Here's one way you can do that:
def path = doc['message'];
for (String m: path) {
if (m.startsWith('user__id_')) {
// returns user__id_1010510, use the techniques above to extract 1010510
return m;
}
}
return "";
Our Field Context documentation may be helpful to you.
please provide your inputs for above query.
I am using the example log line you provided in your initial post.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.