How to create a new field combining values of other field values

Elastic Stack Logstash
21

OK, so in elasticsearch you have three documents, which means in logstash you will have three separate events. What ties them together? Is it originLogId?

Do you want a fourth document that contains the three strings in a single field?

[ "Identity merged", "Identity Accepted", "Identity Rejected" ]

This can be done using an aggregate filter.

If you want all three documents updated with that then I am unsure whether that can be done.

Also, "Identity merged" comes from the [service] field, not the [result] field. How should logstash know which field to look in? Should it just test both and use whichever one starts with Identity?

22

well yes i want all those three strings in a single field.

that's why i want to use the add_field of mutate after using this filter

if [service] == "Identity merged" and [result] == "Identity Rejected" or [result] == "Identity Accepted"

well will the above condition isn't enough for logstash to look into the service field and compare the complete string "Identity merged"

23

You are not answering my questions. These are separate events. What fields should be tested to determine whether two different events should be combined?

Do you want a fourth document created?

24

yes a fourth document needs to be created.

result and the service field would be tested to determine the combination of the events.

25

How many documents exist in total? Are there only three? Or are there multiple sets of three?

26

there are multiple sets of these 3 documents

27

Which fields can be examined to determine that those three document are part of the same set?

28

result and service field

29

Are you able to do a search in kibaba/elasticsearch that pulls out that particular set of three documents, as opposed to some other set of three? If so, what is the search?

30

yes i can able to search in Kibana/elasticsearch to pull out the particular set of documents

here is the search value and the results in kibana

image
image1911×712 135 KB

31

if you want all those three strings in a single field that in you have to put data statically

if [result] == "Identity Accepted" or [result] == "Identity Rejected" and [service] == "Identity merged" {
mutate {
add_field => {"result1" => "Identity Accepted, Identity Rejected, %{service}"}
}
}

after this you can split the data to form an array.

32

hi @chitreshg
after the add field why you used %{service}?
Instead of if we take Identity merge will it be wrong ?
Just Asking the reason ?

33

you can use any of these, if service field contains the multiple values then then you can use %{service} otherwise for single value you can hard-code that value.

34

yes i want to use the service value as Identity merge only.

As when the result keyword is checked their respective %service value also comes in kibana visualization.

Let me implement these conditions and share you all the results

35

hi @chitreshg if i use %service then the respective services of Identity accepted and Idenetity Rejected also comes along in the result .

I achieved my result by doing this which works perfectly for me now

this is my query syntax

if [service] == "IDENTITY_MERGE"
{
   mutate {
    update => { "result" => "Identity Merged" }
    }
}

so now the result is coming as expected

image
image1444×178 35.1 KB

36

@Matish_Bhuyan if you got your solution then mark that as answer, so the topic will close.

37

sorry forgot to add that.it's done now

38

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.