How to create fast network graph?

Elastic Stack Elasticsearch
1

hi, i want to create a network graph from the data available in my elasticsearch index, currently if i want to create such network graph i need to find all the relationship between returned data but my elasticsearch db is huge, so i have to query all the possible matches from search not top 100 or 200 but 5k or 10k sometimes even bigger, so it makes my graph creation really slow.

so my question here is how to make such network graph really fast, is it possible to create network graph using elasticsearch? Or we have to go for some graph database for example(neo4j etc..)?

Thank you.

2

What are the vertices in your graph?
While it’s true that the elastic graph api works with samples of all matching docs you can be smart about what docs go into that sample. Rather than collecting a bajillion docs which might be comms between the same ip addresses (producing a graph with only 2 nodes and one edge) you can ask to “diversify” the sample on a field eg ask for no more than 1 doc per IP address or pair. That way your sample will have more coverage and describe more of the conversations in the dataset.