We are looking to consolidate our technology stack into a more manageable size and one of the areas we are investigating is potentially changing our Web Analytics tool with Elastic RUM. After making a side-by-side comparison of data being collected at the front end, we realised that Elastic RUM agent is able to cover at least 70% Web Analytics data. Many of those it does not cover can be instrumented via tags and labels and custom transactions. We still need to do more in depth investigation to cover edge cases.
Another thing we are clear with is the analytics reporting and dashboards which Kibana does not have (for instance: user churn, marketing goals, funnels, etc).
However, we have been building our own web analytics dashboards and reports using Tableau and are interested more in the collection of analytics data. We have various data stores, including in-memory databases which Tableau connects to. Our BI specialists, Web Analysts, and data scientists can be tapped to create complex correlations info and generate any dashboard and visualisation business requires.
Question (mainly to the Elastic team): is using Elastic RUM agent to collect web analytics data a sound direction? Can you see any downsides/problems with this? We'd love to consolidate all our observability and analytics collection and pre-processing around Elastic. Imagine, a single agent to rule them all
Hi Ronald,
Thanks for the feedback ,
It definitely make sense to collect and consolidate the observability & analytics data.
It's one of the Themes we are discussing internally and is in our backlog, yet no plan or ETA for it.
will be happy to hear & learn more about your needs and existing dashboard capabilities .
Thanks,
Lior
Highly appreciate the feedback and I'm glad that you've considered this approach.
One of our pain points is the need to have multiple libraries included with our web apps (and even web-based downloadable clients). Not only does this add extra download time, these also result in duplicate information being sent to multiple backends. Consider RUM, Hotjar, Optimizely, and Google Analytics all needing to be included in our applications, then some crisscrossing info being sent to them. The amount of bandwidth used by these becomes unconscionable for our users, not to mention battery drain when a mobile device needs to keep sending streams of data. Since we only use specific functionalities for each of those, it makes sense to to consolidate. Consolidation also results with better correlations.
To simplify, we are starting with just Web Analytics. The important bits are:
Referrers for traffic sources
Site Journey (including visitor/returning user tracking), Funnels, Churns
Specific CTA clicks
Heat maps
Time on site/engagement tracking
AB/Multivariate test tracking
Custom transactions
For dashboards, standard Web analytics dashboards are all gonna be in the wishlist, but we'd also be needing things like traffic-vs-performance correlations as well as the ability to create goals and various drill downs.
Data management is also very important as we'd like to have a sane way roll up and aggregate multifacet data as business wants 100% sampling, but also recofnises we can't keep all that granular data forever.
From a (power) user perspective, having default dashboards in Kibana would be useful for the quick exploratory looks as well as for savvy business users, but since we have Tableau and other tools, the ability to extract data and manipulate those would still be top on our list. I think default Kibana dashboards would also be important to companies who may not immediately need the massive capabilities of Tableau or similar tools.
Thanks for Sharing!
It make sense and having both business & Performance data side by side - it's a powerful tool.
btw you can probably get the google analytics data into elasticsearch with logstash as a temp solution and build the Kibana Dashboard on top of the same data ( if it make sense to you).
You're welcome Lior. Indeed we have been correlating data but doing it on Tableau once we consolidate our data from the different sources into an in-memory store (part of our data warehouse) as we take advantage of our Business Intelligence team's skillsets on that matter.
Kibana is most useful for us for ad hoc reports especially for engineering and architecture, while dashboards (mostly for non-technical stakeholders) are created via Tableau, especially those that rely on pivoting and merged sets.
One thing we use Kibana a lot for are the queries (especially Elastic SQL recently) and histograms.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
