How to split array or string with "Ingest Node Pipeline"

[HaKuNa (Remo)]

Hi
I'm new to ECE and have a filebeat pipeline which sends data to ECE.
This is the format of the message field which I have in ECE (separator would be -> \t):

2021-06-25T13:08:39.559+0200 INFO    12409 com.l7tech.traffic: 2021-06-25T11:08:39.554Z\tno_endpoint\tmonitoring\tno_operation\tno_consumer_timestamp\tCOM-556\t0\ttrue\t0\tno_consumer_certificate\t13\tnon-routing\t\t10.183.135.249\tundefined\tundefined\tundefined\tundefined\tundefined\tundefined\tundefined\t/monitoring\tundefined

The aim is to separate the values by \t and add them to a new field.

I'm able to get the first value (date) with the SPLIT processor

regex *([\s]+)(.*)

-> the regex which should work is ([^\s]+) but I had to add (.*) at the end ... strange. I have not yet understood how the regex processor is working in the SPLIT processor.

But then I'm now struggeling with the other 22 parts.
I tried to first move to a sting value with:
SET processor, value={{field_message}}, which leads to a string:

{0=2021-06-25T13:08:39.559+0200 INFO    12409 com.l7tech.traffic: 2021-06-25T11:08:39.554Z, 1=no_endpoint, 2=monitoring, 3=no_operation, 4=no_consumer_timestamp, 5=COM-556, 6=0, 7=true, 8=0, 9=no_consumer_certificate, 10=13, 11=non-routing, 12=, 13=10.183.135.249, 14=undefined, 15=undefined, 16=undefined, 17=undefined, 18=undefined, 19=undefined, 20=undefined, 21=/monitoring, 22=undefined}

Then I tried to separate the new string value with separate SPLIT processors for each value.
The separator regex is working on https://regex101.com/ with the string value above.

0=(.*?),
1=(.*?),
2=(.*?),
...
22=(.*?),

But this is not working in the Ingest Node Pipeline.

Is there a better way to store all the individual values from the input to new fields ?

Would be great if someone can put me in the right direction.

[spinscale (Alexander Reelsen)]

the split processor sounds right. Can you share a fully reproducible example using the Simulate Ingest Pipeline API so that others can try to build the same?

Thank you!

[HaKuNa (Remo)]

Thanks for the response.
In the meantime I solved my issue by using a "Script" processor

int tempStart = ctx.tempFormatted.indexOf("19=",0);
int tempEnd = ctx.tempFormatted.indexOf("20=",0);
ctx.providerMessageId = ctx.tempFormatted.substring(tempStart+3,tempEnd-2);