I am beginner started using kibana only from one month. I have 2 dataframes and number of records are approx 6,00,000(Increase on daily basis per day approx 20,000 records) in one and 1000 in another. I need columns from both dataframes in one Kibana dashboard.
Whether I should join both the dataframes and write one index for one dashboard or shall I write two dataframes in two different index and create one dashboard using two index.
PS: I am using pyspark in backend to do transformations and writing data to ES.
It all depends on your data and how you want to query it. If you don't need to use data from both dataframes in one visualization, then I would suggest to use one index per dataframe and have a dashboard with visualizations from each index.
I want to use data from both dataframes but not in one visualization. I want to use data from both dataframes in one dashboard but in two different visualization.
I want to ask why do you recommend keeping two different index for one dashboard ? I thought it will hamper performance of dashboard if we keep data in two different index rather than keeping in one index and creating two different visualization in one dashboard.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.