Limit Results to _source

[Velusamy_Velu]

Friends:

I have this single document indexed -

{  
   "name":"Jane Doe",
   "age":20,
   "aka":"Jane"
}

When I run the query -

POST _search
{"query":{"match_all":{}}}

ES produces this result -

{  
   "took":1,
   "timed_out":false,
   "_shards":{  
      "total":5,
      "successful":5,
      "failed":0
   },
   "hits":{  
      "total":1,
      "max_score":1,
      "hits":[  
         {  
            "_index":"customer",
            "_type":"external",
            "_id":"1",
            "_score":1,
            "_source":{  
               "name":"Jane Doe",
               "age":20,
               "aka":"Jane"
            }
         }
      ]
   }
}

However, results should be limited to

{  
   "name":"Jane Doe",
   "age":20,
   "aka":"Jane"
}

Any suggestions on how to achieve it through the JSON query? Please remember, there could be more than one entity. In such cases it should be limited to [{"entity 1"}, {"entity 2"}...]

Your help will be greatly appreciated.

[dadoonet (David Pilato)]

You can try this: https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#common-options-response-filtering

[Velusamy_Velu]

Thanks David, your suggestion improved the result by minimizing what goes to client. This helps us control the cost for network use. Apps deployed in the cloud do incur a fee for network traffic.

We use a Java REST client to execute queries and changed the endpoint as below

String endPoint = localHost:9200/index/type/_search?filter_path=hits.hits._source";
WebTarget target = buildTarget(endPoint);
try {
    json = target.request().accept(JSON).post(Entity.json(json), String.class);
} catch (...) {

Now I get

{
   "hits": {
      "hits": [
         {
            "_source": {
               "name": "Jane Doe",
               "age": 20,
               "aka": "Jane"
            }
         }
      ]
   }
}

I still would like to minimize the content to

[
	{
		"name": "Jane Doe",
		"age": 20,
		"aka": "Jane"
	},
	{
		"name": "John Smith",
		"age": 21,
		"aka": "Jack"
	}
]

I believe that's a very good use case for ElasticSearch to consider.

Thank you

[dadoonet (David Pilato)]

TBH I don't think that elasticsearch should be accessible on internet.
I always think elasticsearch as a backend component. Which means to me that you should have a frontend, close to the backend.
The frontend layer can rewrite whatever response to whatever format you wish.

I don't think that we (elastic) will do more simplification than response filtering we already have.

My 2 cents

[Velusamy_Velu]

May be I mis-communicated.

Our instances of ES are insulated from the internet. A set of Java REST clients on application server(s) interact with ES.

Our application is secure and doesn't reveal any details about its data source. Only it can access the servers that run ES. In a nutshell our architecture is:

[Internet Client] <---> [[App Server]] <---> {ES]

Moving JSON between the client & server and server & ES is what we want to minimize.

[dadoonet (David Pilato)]

Where is the cloud network cost then? If your app and elasticsearch are in the same DC do you have additional costs?

[Velusamy_Velu]

I think cloud vendors charge for the network traffic in some fashion. For example the attached screen clip is from the price calculator from a major vendor. The cost/month differs when you change the Network Traffic Processed. Besides the cost, I believe data size also affects the efficiency, a factor I didn't mention in my earlier communication.

[dadoonet (David Pilato)]

Normally if you have the backend and the frontend within the same "project" (whatever it's called on your cloud platform), then you don't pay anything for it AFAIK but I might be wrong.