Combine different logs in one data table


#1

Hi! I have 2 different source logs with several common fields, e.g. field1 and field2, other fields are different.
How can I join all fields from these different sources in one index by field1 and field2 in a data table? It returns "no results found" when selecting all these fields in data table. But there are equal values of field1 and field2 in both source logs.
Thank you!


(Chris Roberson) #2

Hi @SunGirl,

Can you see the data in the Discover tab? Could it be an issue with the time filter at the top?


#3

HI!)
Yes, I see them in discover tab.
Logs with the same field1 and field2 value have different timestamp, but I don`t select @timestamp field in the result data table.
Just an example:
Log1: ip, cmd, @timestamp
Log2: ip, country, @timestamp

And I just want ip, cmd, country in one data table. And I have records with the same ip in both logs, though @timestamp differs at several minutes.


(Chris Roberson) #4

Hi @SunGirl,

I'm not exactly sure what you want to do, but this might help.

You can create a table like this with your sample data if you use a scripted field to create a synthetic field for the two fields that do not exist in both indices.

Scripted field:

def cmd = '';
try {
  cmd = doc['cmd.keyword'].value;
} catch (Exception e) { }

def country = '';
try {
  country = doc['country.keyword'].value;
} catch (Exception e) {}

return cmd + country;

#5

Thank you very much!!!)
And maybe is there another way how to do it? Because in documentation it`s written "Computing data on the fly with scripted fields can be very resource intensive and can have a direct impact on Kibana’s performance".

So I want to join row2 and row3 by ip.


(Chris Roberson) #6

Unfortunately, that's the only way I know of. The other route is to update your mappings to include a field for this and reindex your data


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.