Totally blank on Graph, but aiming to to understand how we might prepare an index for usage in Graph in a small PoC trial. We're aiming initially to try and map netstat-ish data from a server and graph it's tcp connections, how may we need to create a mapping for source and destination data possible enriching data with geohashes.
Appreciate any hints to get started, TIA!
Edit: initially we've adapted an index to ECS 1.1 with [destination] and [source] 'objects' and are trying to map relations between destination.ip and source.ip for specific destination.port (services).
Only seems like the viewing pane of Graph is very small hight portion of the browser windows hight, wondering why it does scale to full extend of the windows hight, any hints?
First, about the viewport of the visualization - this is definitely a bug and shouldn't happen, which version and browser are you using? I will create a Github issue for this if you haven't already. It's possible that the problem is fixed with the upcoming 7.5 release because we changed a little how the graph is rendered, otherwise I can make sure it will get fixed with 7.6
In general for getting started with Graph: You probably have already checked out the documentation on our Website. Besides that if you search for "Kibana Graph" on youtube you will find multiple screencasts of how you can use the Graph UI. I especially recommend the videos of @Mark_Harwood .
In Graph each vertex is a specific term of a single field. A connection between to vertices means the terms of the fields of the vertices co-occur in the documents of your index. You can freely configure the fields you want to use as a source for vertices added to your Graph, so the Graph app is not bound to a specific mapping. About your use case to map network data (in case you ingest data from multiple servers talking to each other) - the way Graph works means if you have two separate fields for source ip and destination ip of a network traffic event, you will get two separate vertices even though they belong to the same host in the network. To create a graph where the host is shown by a single vertex for incoming and outgoing traffic, it makes sense to create a "role less field" that contains an array of both the destination and the source ip. This can be done by using the copy_to parameter in the mapping.
About geo-information: Graph is currently not especially good at handling geo data, it's only operating on specific terms of fields. That's fine for working with e.g. city names or country names, but if you want to show events where they happened on a map, maybe the Maps app is a better fit.
@Mark_Harwood do you have other recommendations to get started with Graph in the network data use case?
Joe’s comment re “role less” fields is the main point.
One other consideration is that the data being summarised can be huge - petabytes of data spread across multiple machines. For this reason we have to work with a sample to avoid blowing memory limits. There’s a danger this sample could be dominated by documents describing comms between a particularly chatty pair to the exclusion of all other pairings. This is why we have the option of a diversity setting - a choice of field where you can limit the number of documents selected for the sample being visualised. For comms data a good choice of field would be a field that contains a combination of source and destination IDs and a size of 1. That means you’d only see one document per unique communicating pair and therefore get better coverage of all the actors. Also remember to increase the number of values returned for each field in the GUI (default is a modest 5, max is 99 per expand step)
@Mark_Harwood Thanks, will attempt to view some of your UTubes on Graphs...
@flash1293 Joe thanks also for your reply, rgd the View Pane, that of course works in another browser (Chrome) The browser that failed me is Safari Version 13.0.3 (14608.3.10.10.1) but I've seen other issues from other sites with this instance of Safari on my work laptop before so it properly not an issue with Graph. Same version of Safari worked on the same sites from other of my Macs, will attempt Graph in those instance at some point (when I get Cisco AnyConnect to work in their MacOS Catalina again) so assume my work labtop Safair might be malfunctioning a bit...
Rgd Geohashes, your right, was also thinking along the line of Graphing Countries/Cities/AS organisation kind thingy
View Pane is fine in Chrome (Geo Cities in Yellow Vertices):
@flash1293 Joe this is what my Safari says in the developer Console when I load teh Graph with a small view pane, only one error is seen in line 9 of bootstrap.js:
This error is expected in the newer versions of Kibana, so nothing to worry about. About the rendering problem: It looks like I fixed this problem in Safari not long ago: https://github.com/elastic/kibana/pull/47667#discussion_r332921210 The fix will be released with the upcoming 7.5 version
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
The browser that failed me is Safari Version 13.0.3 (14608.3.10.10.1) but I've seen other issues from other sites with this instance of Safari on my work laptop before so it properly not an issue with Graph. Same version of Safari worked on the same sites from other of my Macs, will attempt Graph in those instance at some point (when I get Cisco AnyConnect to work in their MacOS Catalina again) so assume my work labtop Safair might be malfunctioning a bit...
