Query with elasticsearch and jcard json (array of array mapping)

I have a data set containing vcards based on the JSON Jcard Mapping (RFC 7095 - jCard: The JSON Format for vCard) The problem is that I want to search on the 'fn' field only.

The vcard data has the following format. As you can see the value don't have the "traditional" "key":"value" structure but an array with several value of which the first happen to be the key for that array.

["vcard",
[
["version", {}, "text", "4.0"],
["fn", {}, "text", "John Doe"],
["gender", {}, "text", "M"],
["categories", {}, "text", "computers", "cameras"],
...
]
]
I'm creating a vcard document like this

curl -X POST localhost:9200/vcards/id1 -d '{
"id":"id1",
"vcardArray" : ["vcard",
[
["version", {}, "text", "4.0"],
["fn", {}, "text", "John Doe"],
["gender", {}, "text", "M"]
]
],
"status":["registered"]
}'
Normally I would create a specific mapping so when the document is analysed a new index is created and searching for a fn field would look something like this....

curl -v -X POST http://localhost:9200/vcards/_search -d '{ "query" :
{ "bool" : { "must": { "match" : {
"vcardArray.vcard.fn" :
{ "query" : "Rik Ribbers" , "type" : "phrase" }
} } } }
}'
A potential mapping would look like this, but this one is not working

curl -X PUT http://localhost:9200/vcards -d '
{
"mappings": {
"vcardArray" : {
"type" : "nested",
"properties" : {
"vcard" : {
"type" : "index",
"index" : "not_analyzed"
}
}
}
}
}'
Any pointer to the correct mapping or query would be helpful.