I am ingesting data and looking for opinions on better ways of indexing the data for later aggregation analytics. The data is currently pulled and stored in Elasticsearch as numbered q&a fields.
{
"question1": {
"question": "How are you?",
"answer": "Pumpkin Pie"
}
},
{
"question2": {
"question": "What's your favorite color?",
"answer": "Bowling ball"
}
}
The problems I am seeing with this strategy is, there's no easy way to create a Kibana visualization, that I know of, that aggregates all the questions and/or answers to answer things like "How many times is the question, 'How are you' asked?" Or a way to filter on a specific question without having to know it's 'question246'. I thought maybe I could join all the questions/answers asked into a single field of arrays, but I'm not sure how I'd go about joining a non-contiguous N number of fields like that together via Logstash.
Any thoughts on a better way to organize the data?