Associating/combining data from secondary records

Elastic Stack Kibana
1

In Kibana I often want to associate data from different records together.

For example I have a location/geopoint associated with an IP address in one record, and in another record I have the same IP associated with an Error. Now on a map I want to see all of the errors.

How can I structure a query to associate or combine these two records together?

2

Kibana visualizations all deal with aggregations of documents. So when you look at a Tile Map visualization it doesn't show you individual documents. And the only thing you can really customize is the size of the circles. You can change them from the default Count, to some other metric. So you could potentially have the size of the circles represent the count of errors.

Only Discover in Kibana shows individual documents. There is a new feature that helps people investigating incidents in logs that allows you to View surrounding documents. So I think that with a combination of filtering and/or querying would be the best way to see the data from multiple docs.

image
image.png1736×908 102 KB

Regards,
Lee

3

Thanks for the response. The issue I'm trying to solve is that the documents that have the error only have the IP address, not the geopoint. Other documents that have the geopoint and IP.

This is not limited to the Map example. Another example would be that I have a response time from a request in a document that has a request/response ID. In another document I have the request/response ID and the URL of the request. How could I make an visualization between the response time and URL?

4

I think the only way is to use Discover and filter on the IP for your first case. Some docs in that subset have the error and other docs have the geopoint.
You would probably filter on the error first,
then add a filter for the IP.
And then remove or disable the filter on the error.

Same for your second case.
First filter to find response time>x
then add a filter on a response ID
then remove the filter or query on response time

5

Hi Lee,

Wouldn't this only show the results for one IP or one response ID? I want to see all errors at their respective geo_point.

Regards,

David

6

Hi David,
You could filter your docs so that you only get errors. And if those same docs had the geo_point you could show those on the map. But it sounds like that's not how your data is loaded.

Is your data loaded using logstash? And is it doing the geopoint lookup on the IP address and adding it?

Lee

7

Right, the issue here is that there is geo_point data that is associated with the error (via IP address) but is not in the same document.

We are loading the data using logstash. However the geopoint is not being done based on IP Address; it is done based off a mobile devices location data. If we were doing geopoint lookup based on IP it would make this a non-issue, but that's not the case.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.