# Filter with millions of record
**URL:** https://discuss.elastic.co/t/filter-with-millions-of-record/134561
**Category:** Elasticsearch
**Created:** [June 5, 2018, 8:28am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561 "2018-06-05T08:28:50Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@chaudharys31](https://discuss.elastic.co/u/chaudharys31)
#### Post date: [June 5, 2018, 8:28am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/1 "2018-06-05T08:28:51Z")
Can someone explain term filter or any other filter work with millions or 100 K records in the filter? How will be performed and how it will work?
---
### Author:  [@chaudharys31](https://discuss.elastic.co/u/chaudharys31)
#### Post date: [June 13, 2018, 6:29am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/2 "2018-06-13T06:29:51Z")
Anyone?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 13, 2018, 6:39am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/3 "2018-06-13T06:39:07Z")
My guess is that it will be very slow and perform badly, but I would recommend benchmarking to find out what the limit is for your setup.
---
### Author:  [@chaudharys31](https://discuss.elastic.co/u/chaudharys31)
#### Post date: [June 13, 2018, 6:51am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/4 "2018-06-13T06:51:48Z")
We have term conditions where we are passing 100 K, 200 K records and performs is very bad and most of the time it's not workign.
How we can change our approcah to make it workign?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 13, 2018, 6:53am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/5 "2018-06-13T06:53:09Z")
That does not surprise me. What is your use case? Why are you sending so many terms? What are you trying to achieve? Which version of Elasticsearch are you on?
I suspect you will need to change how you work with Elasticsearch as I can not see any tuning that will make the current approach perform well.
---
### Author:  [@Puroo\_Jain](https://discuss.elastic.co/u/Puroo_Jain)
#### Post date: [June 13, 2018, 6:58am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/6 "2018-06-13T06:58:21Z")
Hi Christian,
Currently, we r using 2.4.0 but we also switch into 6.2.4.
Basically, we show charts and tables for which we need data from many indexes, so first we find out data from one index in the form of ids and then pass in query to find out result from another index.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [June 13, 2018, 7:00am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/8 "2018-06-13T07:00:21Z")
So you are doing "joins" basically at search time...
There is no miracle to expect here.
If you can do it, do the join at index time and you'll pay the price only once.
My 2 cents
---
### Author:  [@Puroo\_Jain](https://discuss.elastic.co/u/Puroo_Jain)
#### Post date: [June 13, 2018, 7:02am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/9 "2018-06-13T07:02:21Z")
we are working on runtime approach..its change everytime whenever u load again..so during indexing time how we save data?
---
### Author:  [@Puroo\_Jain](https://discuss.elastic.co/u/Puroo_Jain)
#### Post date: [June 13, 2018, 7:04am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/10 "2018-06-13T07:04:27Z")
abcd
---
### Author:  [@Puroo\_Jain](https://discuss.elastic.co/u/Puroo_Jain)
#### Post date: [June 13, 2018, 7:06am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/11 "2018-06-13T07:06:21Z")
David, any suggestion to which we perform not badly?
---
### Author:  [@chaudharys31](https://discuss.elastic.co/u/chaudharys31)
#### Post date: [June 13, 2018, 7:29am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/12 "2018-06-13T07:29:21Z")
@dadoonet @Christian_Dahlqvist Is that possible for you guys to do a small session with us and we can show you what we are doign and by seeing that suggest us some solution?
---
### Author:  [@chaudharys31](https://discuss.elastic.co/u/chaudharys31)
#### Post date: [June 13, 2018, 7:34am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/13 "2018-06-13T07:34:14Z")
Davind, we are not doing join, we pull some ids from one index one the bases of some condition than passing these retrive ids in another index in term.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 13, 2018, 8:02am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/14 "2018-06-13T08:02:54Z")
I do not do private sessions. Please post the relevant information here so the rest of the community can benefit.
If you are not comfortable sharing details in public, Elastic offers [support subscriptions](https://www.elastic.co/subscriptions) as well as [consulting](https://www.elastic.co/consulting).
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [June 13, 2018, 9:02am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/15 "2018-06-13T09:02:52Z")
This is exactly what a join is IMO.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 13, 2018, 9:09am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/16 "2018-06-13T09:09:44Z")
If you can explain your data model and how you are updating and querying it, we might be able to provide some suggestions. It would also be useful if you could provide some guidance on data volumes and cardinality between entities modelled.
---
### Author:  [@Puroo\_Jain](https://discuss.elastic.co/u/Puroo_Jain)
#### Post date: [June 13, 2018, 9:25am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/17 "2018-06-13T09:25:57Z")
abcd
---
### Author:  [@Puroo\_Jain](https://discuss.elastic.co/u/Puroo_Jain)
#### Post date: [June 13, 2018, 9:30am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/18 "2018-06-13T09:30:44Z")
Yes Please
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 13, 2018, 9:31am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/19 "2018-06-13T09:31:42Z")
What does your documents look like? How are you storing and linking the IDs?
---
### Author:  [@Puroo\_Jain](https://discuss.elastic.co/u/Puroo_Jain)
#### Post date: [June 13, 2018, 9:40am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/20 "2018-06-13T09:40:28Z")
abcd
---
### Author:  [@Puroo\_Jain](https://discuss.elastic.co/u/Puroo_Jain)
#### Post date: [June 13, 2018, 9:48am UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561/21 "2018-06-13T09:48:49Z")
Charts are working on the basis of Ids..
Main problem is if we fetch detials from 4 to 5 lakh data without any terms filter, then query take too much time..
if also use terms filters for ids then its also take time.
Suggest some 😐
[Next page](https://discuss.elastic.co/t/filter-with-millions-of-record/134561.md?page=2)