Elastic Search 5.1.1 - How to fetch all documents that contains/ startswith operator on a field in ElasticSearch

[HS161148 (Harneet Singh)]

How to fetch all documents that contains/ startswith with value on a field in ElasticSearch.

[Christian_Dahlqvist]

What is the mapping for the field in question?

[HS161148 (Harneet Singh)]

Hi,

I am new to Elastic Search.. Is the below information you are looking for?

[Christian_Dahlqvist]

Please use the get mapping API to retrieve the full mapping.

[HS161148 (Harneet Singh)]

{"events_2021_2":{"mappings":{"event":{"_all":{"enabled":false},"dynamic_templates":[{"event_string_fields":{"match":"*","mapping":{"analyzer":"event_custom_field_search_analyzer","fields":{"keyword_value":{"ignore_above":10922,"type":"keyword","doc_values":true}},"index_options":"docs","norms":{"enabled":false},"type":"text"}}}],"properties":{"acl type":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"action type":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"alert rule names":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"alert_rule_description":{"type":"text"},"application type":{"type":"text","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"attribute name":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"attribute new value":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"attribute old value":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"attributechanges":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"attributename":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"attributenewvalue":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"attributeoldvalue":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"authentication":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"bam id":{"type":"long"},"bam name":{"type":"text","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"bam type id":{"type":"long"},"changedpermissions":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"classification category":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"creation_timestamp":{"type":"date"},"customaction":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"destinationfileextension":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"destinationrelativeurl":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"domain":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"eventdata":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"eventinfo":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"extradetails":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"file extension":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"gpolinktoou":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"gponame":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"gpopropertyorigin":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"gpopropertypath":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"group name":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"ip address":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"itemtype":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"linkexpirationdate":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"logical bam ids":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"logical paths":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"membername":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"modifiedobject":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"modifiedproperties":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"new name":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"new path":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"newobjectcn":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"newobjectdn":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"object class":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"object name":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"objectcn":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"objecttype":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"old name":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"old path":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"originating server":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"path":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"permission inheritance":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"permission subject":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"permission type":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"permissionaction":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"permissiongroup":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"permissionname":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"permissionuser":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"siteid":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"snapshotdirectory":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"sourcerelativeurl":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"subjecttype":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"target entity":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"target entity type":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"threshold_alert_rule_candidates":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"threshold_alert_rule_ids":{"type":"long"},"time of day":{"type":"date","format":"epoch_millis"},"timestamp":{"type":"date"},"user full name":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"user name":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"useragent":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"usersharedwith":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"},"wssuserid":{"type":"text","norms":false,"index_options":"docs","fields":{"keyword_value":{"type":"keyword","ignore_above":10922}},"analyzer":"event_custom_field_search_analyzer"}}}}}}

[HS161148 (Harneet Singh)]

Hi Team,

Does anyone knows how to achieve this?

[dadoonet (David Pilato)]

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.

[dadoonet (David Pilato)]

Could you provide a full recreation script as described in About the Elasticsearch category. It will help to better understand what you are doing. Please, try to keep the example as simple as possible.

A full reproduction script is something anyone can copy and paste in Kibana dev console, click on the run button to reproduce your use case. It will help readers to understand, reproduce and if needed fix your problem. It will also most likely help to get a faster answer.

[dadoonet (David Pilato)]

@HS161148

How to fetch all documents that [...]

You can use:

• the size and from parameters to display by default up to 10000 records to your users. If you want to change this limit, you can change index.max_result_window setting but be aware of the consequences (ie memory).
• the search after feature to do deep pagination.
• the Scroll API if you want to extract a resultset to be consumed by another tool later.

[dadoonet (David Pilato)]

5.1.1 is not supported anymore. Please upgrade. 7.12.0 is now available.