Distribution tables from survey data

Hi,

I am storing survey data like this:

[  
  {  
    "userid":1,
    "answers":[  
      {  
        "key":"gender",
        "value":"male"
      },
      {  
        "key":"color",
        "value":"red"
      },
      {  
        "key":"vehicle",
        "value":"car"
      }
    ]
  },
  {  
    "userid":2,
    "answers":[  
      {  
        "key":"gender",
        "value":"female"
      },
      {  
        "key":"color",
        "value":"blue"
      },
      {  
        "key":"vehicle",
        "value":"bike"
      }
    ]
  },
  ......
]

That means lots of users with lots of answers to different questions. I have to be flexible about what questions have been asked, so I chose the key/value style.

Now I want to find a query that will give me a distribution table on gender and color.
That means: A 2-dimensional table having gender as one axis and color as one axis showing me all possible terms in those fields.
I want to have a nice overview on how many women like the color red or how many men like blue and so on.

I tried a lot with nested filtered terms aggregations and did not succeed yet.

Any hints would be appreciated...