Enrich processor time range

[Marcin_Frankiewicz]

Hi,

i have a pipeline with enrich processor.

When enriching is called.. what time range is used for enrich?
All documents in index(defined in enrich policy) will be scanned? or maybe from last 15minutes?

Elasticsearch 7.17

[stephenb (Stephen Brown)]

The lookup is performed I simply comparing the match_field to the field that's passed in the enrich processor. Basically an exact match time is not considered.

Typically the enrich / look up data is not time-based.

What are you trying to accomplish?

[Marcin_Frankiewicz]

I'm trying to proof, that while max_matches = 1, then enriched data can be random.

Example :

POST /test-users/_doc
{
  "@timestamp": "2022-10-10T18:40:00.000+02:00",
  "id": "1",
  "name": "John",
  "surname": "Derre"
}

POST /test-users/_doc
{
  "@timestamp": "2022-10-10T18:45:00.000+02:00",
  "id": "2",
  "name": "Michael",
  "surname": "Frank"
}

POST /test-mails/_doc
{
  "@timestamp": "2022-10-10T18:45:00.000+02:00",
  "id": "1",
  "email": "test2@test.com",
  "real": "true"
}

POST /test-mails/_doc
{
  "@timestamp": "2022-10-10T18:45:00.000+02:00",
  "id": "2",
  "email": "test3@test.com",
  "real": "true"
}

Does user with id=1 will have random email?

[stephenb (Stephen Brown)]

@Marcin_Frankiewicz

I'm trying to proff

Apologies I am not clear what you mean.

So if that data above is the Lookup Data with duplicate ids ... then yes I would consider the match random but you can also return up to 128 Matches so you could get both and look for the latest.

Or fix your enrich data to only have the latest.

Oh and Welcome to the Community!

[Marcin_Frankiewicz]

Thank you for explain