Filter with millions of record

Elastic Stack Elasticsearch
42

Yes, its make faster

43

So we do agree that was the root cause of the problem, right?

Now we can start thinking about the use case and what you are trying to do. Could you explain that a bit?
Like share what a typical document is looking like and what is the output you'd like to see.

Ideally a full recreation script to play around with your data would help a lot.

May be you need to think about preprocessing some data before indexing? May be you need to change your model?
Giving samples would help hopefully to determine that.

1 Like
44

Now, we save simple key-value data in only one index as sum, average, and other arthimetic functionality.
Now, In one query we pass date from UI and its change regularly and we find a sum of power and after that, we divide no off days coming from UI.
So, we could not save data during indexing, because no of days changes query to query.
Now, we find band wise data in which we need the sum of power and then we divide it no of days and after that, we calculate the count that how many ids are lies in the band.
Please suggest in above query, how we find the count of the bucket.

   "Locations": {
      "cardinality": {
        "field": "AssetId",
        "aggs": {
          "AssetBucket": {
            "terms": {
              "field": "AssetId",
              "size": 10000
            },
            "aggs": {
              "PowerOffDuration": {
                "sum": {
                  "field": "SumOfPowerOffDuration"
                }
              },
              "conversion_ratio": {
                "bucket_script": {
                  "buckets_path": {
                    "total_clicks": "PowerOffDuration"
                  },
                  "script": {
                    "source": "return [params.total_clicks].sum() /3100/31 >= 8 && [params.total_clicks].sum() /3100/31 < 12 ? 1 : 0"
                  }
                }
              }
            }
          }
        }
      }
    }
45

Anyone can help us?

46

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.