Enrich processor time range

Elastic Stack Elasticsearch
1

Hi,

i have a pipeline with enrich processor.

When enriching is called.. what time range is used for enrich?
All documents in index(defined in enrich policy) will be scanned? or maybe from last 15minutes?

Elasticsearch 7.17

2

The lookup is performed I simply comparing the match_field to the field that's passed in the enrich processor. Basically an exact match time is not considered.

Typically the enrich / look up data is not time-based.

What are you trying to accomplish?

3

I'm trying to proof, that while max_matches = 1, then enriched data can be random.

Example :

POST /test-users/_doc
{
  "@timestamp": "2022-10-10T18:40:00.000+02:00",
  "id": "1",
  "name": "John",
  "surname": "Derre"
}

POST /test-users/_doc
{
  "@timestamp": "2022-10-10T18:45:00.000+02:00",
  "id": "2",
  "name": "Michael",
  "surname": "Frank"
}

POST /test-mails/_doc
{
  "@timestamp": "2022-10-10T18:45:00.000+02:00",
  "id": "1",
  "email": "test2@test.com",
  "real": "true"
}

POST /test-mails/_doc
{
  "@timestamp": "2022-10-10T18:45:00.000+02:00",
  "id": "2",
  "email": "test3@test.com",
  "real": "true"
}

Does user with id=1 will have random email?

4

Apologies I am not clear what you mean.

So if that data above is the Lookup Data with duplicate ids ... then yes I would consider the match random but you can also return up to 128 Matches so you could get both and look for the latest.

Or fix your enrich data to only have the latest.

Oh and Welcome to the Community!

5

Thank you for explain :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.