How to use the script field for string

[somya]

I have a message index pattern which has the below string

[INFO][callback][views][2020-03-18 13:06:21][xce_web][PROD][user__id_1010510-18-03-2020-13-03-26][167]:For ABC start time is 2020-03-18 13:03:26.999778 & end time is 2020-03-18 13:06:21.507542 and pipeline takes 174507 milliseconds to complete.

How to get the substring i.e user_id and 174507 in 2 diferent script fields.

[LizaD (Liza Dayoub)]

Hi @somya,

Welcome to our community! Here is a blog post shows how to get a substring:

Using Painless in Kibana scripted fields

Hope this helps,

Thanks,
Liza

[somya]

Hello @LizaD

Thanks for the reply. I tried using the below script
def path = doc['message.keyword'].value;
if (path != null) {
int lastSlashIndex = path.IndexOf('[doc');
if (lastSlashIndex > 0) {
return path.substring(lastSlashIndex,lastSlashIndex +10);
}
}
return "";

But all in vain after saving and going to discover.it is showing the warning This field is present in your elasticsearch mapping but not in any documents in the search results. You may still be able to visualize or search on it.

[stu (Stuart Tettemer)]

Hi @somya

Based on the input, I slightly modified the script:

def path = doc['message.keyword'].value;
String userIdPrefix = '[user__id_';
int userIdLen = 7;
if (path != null) {
  int lastSlashIndex = path.indexOf(userIdPrefix);
  if (lastSlashIndex > 0) {
    return path.substring(lastSlashIndex + userIdPrefix.length(), lastSlashIndex + userIdPrefix.length() + userIdLen);
  }
}
return "";

With your input, that returns 1010510 as the user_id. I didn't see '[doc' in your sample input, so I replaced it with [user__id_ and I assumed your userId was always length 7.

IMPORTANT if you are modifying your scripted field in the kibana scripted field editor, you MUST refresh discover before it will use the updated script.

Kibana caches the scripted fields for an index when you load Discover.

[somya]

Hello Stuart Hello Liza,

It seems the message field is of text type. Will this script will work on this.
If no please provide your suggestion

Regards,
Somya Chawla

[somya]

Is there any way to export the saved search data in CSV format from UI or from API

[somya]

@stu @LizaD please provide your inputs for above query

[LizaD (Liza Dayoub)]

Hi @somya

For the CSV question, you can save your data into CSV from Discover -> Share -> CSV Reports.

Thanks,
Liza

[stu (Stuart Tettemer)]

It seems the message field is of text type. Will this script will work on this.

You'd have to change
def path = doc['message.keyword'].value;
to
def path = params['_source']['message'];
if field data is disabled. This change accesses the _source to get at the raw data.

However, if field data is enabled (and depending how it is tokenized), then doc['message'] will be a list of strings, so you'll have to look for values in that start with user__id_. Here's one way you can do that:

def path = doc['message'];
for (String m: path) {
    if (m.startsWith('user__id_')) {
        // returns user__id_1010510, use the techniques above to extract 1010510
        return m;
    }
}
return "";

Our Field Context documentation may be helpful to you.

please provide your inputs for above query.

I am using the example log line you provided in your initial post.