# Searching inner hits in ES datastore **URL:** \ **Category:** Elasticsearch\ **Created:** [August 2, 2018, 1:11am UTC](https://discuss.elastic.co/t/searching-inner-hits-in-es-datastore/142665 "2018-08-02T01:11:18Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Parul-Kudtarkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parul-kudtarkar/32/30650_2.png) [@Parul-Kudtarkar](https://discuss.elastic.co/u/Parul-Kudtarkar)\ **Post date:** [August 2, 2018, 1:11am UTC](https://discuss.elastic.co/t/searching-inner-hits-in-es-datastore/142665/1 "2018-08-02T01:11:18Z")
Hi ES community, Searching and retrieving inner hits ([https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-inner-hits.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-inner-hits.html)) is extremely slow! Any tips to expedite the search? Thanks! Parul ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [August 2, 2018, 4:27am UTC](https://discuss.elastic.co/t/searching-inner-hits-in-es-datastore/142665/2 "2018-08-02T04:27:02Z")
Is this happening for parent-child or nested searches? ---
**Author:** ![Parul-Kudtarkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parul-kudtarkar/32/30650_2.png) [@Parul-Kudtarkar](https://discuss.elastic.co/u/Parul-Kudtarkar)\ **Post date:** [August 2, 2018, 8:15pm UTC](https://discuss.elastic.co/t/searching-inner-hits-in-es-datastore/142665/3 "2018-08-02T20:15:19Z")
@NerdSec this is nested search ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [August 3, 2018, 10:03am UTC](https://discuss.elastic.co/t/searching-inner-hits-in-es-datastore/142665/4 "2018-08-03T10:03:36Z")
Can you add some more details, because this is a very generic question: - Elasticsearch version - Specific query you are running, maybe even a (simplified) mapping - What slow means specifically (in general, in comparison to other queries,...) ---
**Author:** ![Parul-Kudtarkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parul-kudtarkar/32/30650_2.png) [@Parul-Kudtarkar](https://discuss.elastic.co/u/Parul-Kudtarkar)\ **Post date:** [August 3, 2018, 7:49pm UTC](https://discuss.elastic.co/t/searching-inner-hits-in-es-datastore/142665/5 "2018-08-03T19:49:50Z")
@xeraa **Elasticsearch version** ``` curl -XGET 'localhost:9200' { "status" : 200, "name" : "Gaea", "cluster_name" : "v1-cluster", "version" : { "number" : "1.7.6", "build_hash" : "c730b59357f8ebc555286794dcd90b3411f517c9", "build_timestamp" : "2016-11-18T15:21:16Z", "build_snapshot" : false, "lucene_version" : "4.10.4" }, "tagline" : "You Know, for Search" } ``` Specific query you are running, maybe even a (simplified) mapping - the data is querying biological dataset **ES query** peak\_results = es.search(body=query, index=chromosome, doc\_type=assembly, size=99999) **About peak\_query function** ``` query = get_query(start, end, within_region=region_inside_status) def get_query(start, end, with_inner_hits=True, within_region=True): """ return peak query """ query = { 'query': { 'filtered': { 'filter': { 'nested': { 'path': 'location', 'filter': { 'bool': { 'should': [] } } } }, '_cache': True, } }, '_source': False, } search_ranges = { 'inside_range': { 'start': start, 'end': end }, 'range_inside': { 'start': end, 'end': start }, 'overlap_start_range': { 'start': start, 'end': start }, 'overlap_end_range': { 'start': end, 'end': end } } for key, value in search_ranges.items(): query['query']['filtered']['filter']['nested']['filter']['bool']['should'].append(get_bool_query(value['start'], value['end'])) if with_inner_hits: query['query']['filtered']['filter']['nested']['inner_hits'] = {'size': 99999} return query ``` **What slow means specifically (in general, in comparison to other queries,...)** To explain slowness in our use case, let me give you an overview of the mapping function, the query is executed against nested object **location** ``` 'location': { 'type': 'nested', 'properties': { 'start': { 'type': 'long' }, 'end': { 'type': 'long' }, 'state': { 'type': 'string' }, 'val': { 'type': 'string' } ``` The longer the search coordinates/values for start, end parameter longer is the query search i.e. for start = 1 and end = 100 takes 1second return the location information, however for start = 1 and end = 10000 it takes 60 seconds to return location information. Let me know, what you think. Best Regards, Parul ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [August 6, 2018, 9:49am UTC](https://discuss.elastic.co/t/searching-inner-hits-in-es-datastore/142665/6 "2018-08-06T09:49:35Z")
The bad news is that your Elasticsearch version is ancient, which hasn't been supported for quite a while and also misses some very helpful tools for finding performance issues like the [profile API](https://www.elastic.co/guide/en/elasticsearch/reference/6.3/search-profile.html). The good news is that inner hits should have improved quite a bit in more recent versions. We have a lot of [performance benchmarks](https://elasticsearch-benchmarks.elastic.co/nested/index.html) and the one you'll be most interested in is probably [major versions of nested](https://elasticsearch-benchmark-analytics.elastic.co/app/kibana#/dashboard/42939a4e-b2db-44d6-a4ab-a1cf89976d83?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-5y,mode:quick,to:now))&_a=(description:'',filters:!(),fullScreenMode:!f,options:(darkTheme:!f,useMargins:!f),panels:!((gridData:(h:4,i:'1',w:6,x:6,y:9),id:'1ee5d049-d384-4a2f-bce1-937882e99cb9',panelIndex:'1',type:visualization,version:'6.2.4'),(gridData:(h:4,i:'2',w:6,x:0,y:9),id:'4f8f976b-ec4f-47f4-b8c1-52ed85774c2b',panelIndex:'2',type:visualization,version:'6.2.4'),(gridData:(h:4,i:'3',w:6,x:6,y:13),id:'59c39104-87f1-43a9-93f7-db38927f91c9',panelIndex:'3',type:visualization,version:'6.2.4'),(gridData:(h:4,i:'4',w:6,x:0,y:13),id:'63abc3a8-b155-46f2-956b-c18d006e01aa',panelIndex:'4',type:visualization,version:'6.2.4'),(gridData:(h:4,i:'5',w:6,x:6,y:17),id:aba3c62b-56cd-4c64-aaf4-e2030bd9df98,panelIndex:'5',type:visualization,version:'6.2.4'),(gridData:(h:4,i:'6',w:6,x:0,y:17),id:d6f15d12-5a4f-4ac2-9e8b-46e578c4dc64,panelIndex:'6',type:visualization,version:'6.2.4'),(gridData:(h:5,i:'7',w:6,x:6,y:21),id:fd1c0a53-0f18-437f-adb4-84cf83b0a4eb,panelIndex:'7',type:visualization,version:'6.2.4'),(gridData:(h:5,i:'8',w:6,x:0,y:21),id:'090a6ef5-ffc1-4097-989e-e0279d9f59be',panelIndex:'8',type:visualization,version:'6.2.4'),(gridData:(h:5,i:'9',w:6,x:0,y:26),id:b0174e51-ec6b-46cc-a66b-37d76c43c259,panelIndex:'9',type:visualization,version:'6.2.4'),(gridData:(h:9,i:'10',w:12,x:0,y:0),id:e5173866-5409-4ef0-95a6-1faa7f7a62f6,panelIndex:'10',type:visualization,version:'6.2.4')),query:(language:lucene,query:(query_string:(analyze_wildcard:!t,query:'*'))),timeRestore:!f,title:release-nested,viewMode:view)) — especially about inner hits at the end: ![release-nested%20-%20Kibana%20%F0%9F%94%8A%202018-08-06%2011-42-44](https://us1.discourse-cdn.com/elastic/original/3X/2/6/26e448de8ac25eb2c875cdda874f602cbd1e8b1a.png) ![release-nested%20-%20Kibana%20%F0%9F%94%8A%202018-08-06%2011-43-13](https://us1.discourse-cdn.com/elastic/original/3X/4/3/434b4454f266f1c8c9d4feab55d33777431f6661.png) You'll have to update sooner or later, but this will probably require quite a lot of work — rewriting queries and (remote) reindexing the data. Not sure about quick wins. The `for` loop for the should criteria looks dangerous to me. Also `{'size': 99999}` for inner hits could be an issue — do you really need that much data? What's the total size of the response document? But even tweaks here won't save you from the update in the long run. ---
**Author:** ![Parul-Kudtarkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parul-kudtarkar/32/30650_2.png) [@Parul-Kudtarkar](https://discuss.elastic.co/u/Parul-Kudtarkar)\ **Post date:** [August 7, 2018, 7:19pm UTC](https://discuss.elastic.co/t/searching-inner-hits-in-es-datastore/142665/7 "2018-08-07T19:19:43Z")
@xeraa Thanks a lot! We will keep the community posted on our implementation and speed improvement ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 4, 2018, 7:19pm UTC](https://discuss.elastic.co/t/searching-inner-hits-in-es-datastore/142665/8 "2018-09-04T19:19:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.