Using ES for visual search

Sorry for my lousy explanation. Let me try that again

Basically I need to index, say, 1M (million) images. Each images is described by a number of features, each feature is a vector of 128 float numbers. Different images could have different number of features, but they could share some features. For example, image 1 might have 500 features, image 2 has 600 features, and there're 200 features in image 1 that are also in image 2. Thus, even though each image has more than 500 features, the total number of features collected from 1M images is not 500M but only, say, 2M. So each image is like a book and each feature is like a word.

Now, given a query image, I can also describe it by a number of features, each feature is also a vector of 128 float numbers, except that each feature might not be exactly the same as one of the 2M features mentioned above. Thus, when I search for each (new) feature from my index in ES, I don't search for the exact feature but the nearest neighbor instead.

My question is: is that nearest neighbor search feasible with Elasticsearch, and if such, how do I define the distance between two features (two vectors) in the query. What I mean is how to write the query to represent my distance, not the formula of the distance (could be just use L2 norm).

Thanks