Tracking the execution of Scroll API in Elasticsearch

Elastic Stack Elasticsearch
1

Hi everyone,

I'm currently working with Elasticsearch and I'm interested in tracking the execution of the Scroll API. Specifically, I want to audit the use of the Scroll API to keep track of when it is executed and by which user.

Is it possible to achieve this through Elasticsearch or Kibana auditing? If so, could anyone provide guidance or share their experiences on how to set this up?

Alternatively, if there are better methods or tools for achieving this kind of tracking, I would appreciate any suggestions.

Thank you!

2

Hi @yago82 !

You could use Audit Logging in Elasticsearch.

Take into account that audit logging is only available in specific subscriptions - please check that out.

Using a proxy in front of Elasticsearch would make it possible to access that information from the headers and path of the Elasticsearch requests.

3

Hi Carlos,

First of all, thank you for the information.

In order to avoid overloading the cluster by enabling all audit settings, could you please guide me on the specific settings needed to track the use of the Scroll API?

Any detailed instructions or experiences you can share would be greatly appreciated.

Thank you!

4

Given your use case, I'd say that you need to track access granted operations to the scroll API (see audit events for REST event types):

  • xpack.security.audit.logfile.events.include: access_granted

You could also add ignore policies, and ignore actions you're not interested on (which will be all actions except for the scroll API):

  • xpack.security.audit.logfile.events.ignore_filters.<policy_name>.actions

I'd recommend to setup a audit logging on a test cluster and experiment with these options until you get the audit log that you need for the action you're interested in.

Hope that helps!