Hi, I'm currently working on creating browser based dashboards for a large system to monitor various metrics. As everyone is working from home these will not be sitting on a wall in an office but rather on the screens of many users. My concern is the stress this will put on Elastic search. The canvas could make maybe 200 requests every minute. Times this by 30-50 users.
So my question is, is elastic search capable of this sort of usage? Has anyone else tackled this issue and are there any methods of implementation that can help ease the stress of hitting elastic search with potentially 10,000+ search filters?
Are you talking about Dashboard or Canvas here? I am not familiar with any performance concerns around either.
There is no limit on the number of client connections imposed by ES. There is connect timeout on the client though, so queries shouldn't be too complex. I assume each request separately won't contain 10000+ search filters, but you're referring to the total number of filters by all users across Kibana sessions?
Talking about canvas embedded into a webpage and shared amongst employees.
If 30-50 employees all have the canvas board open on their second monitor (and it auto updates to monitor system status).
Each user would be requesting around 100-200 filters. (querying things like cpu usage, error counts, unique user counts etc for all servers). Is this feasible from a performance perspective? is the caching on elastic search sufficient to handle this sort of use? Are there any recommendations on configuring something like this?
Or would we need to use some sort of server side processing to cache the data to ease the strain on elastic search?
No its still being designed. But it will monitor maybe 12+ systems so even if it only displays essential info it would still be pretty large. I just wondered if using canvas is a normal approach to this sort of problem or would the norm be to create a more custom solution, with a box using the Search API and caching the data to serve to the employees rather than using canvas? Is there a recommended approach to this sort of task? Or any recommendations in terms of config that could help?
Yes this is sort of similar, thanks for that. Unfortunately ours would be more complex maybe double the number of requests to elastic search. My question is more to do with the overheads of 50 users running this sort of thing from home at the same time with auto refresh. Would we be expected to create some sort of custom solution. Or is elastic search capable of handling this sort of usage? are there any recommendations on this?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.