# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=795 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 796 --- ## [Filter latest record for each user and aggregate on different field in elasticsearch](https://discuss.elastic.co/t/filter-latest-record-for-each-user-and-aggregate-on-different-field-in-elasticsearch/250435)
**Author:** [@matrix1](https://discuss.elastic.co/u/matrix1)\ **Replies:** 15\ **Last updated:** [October 2, 2020, 10:05am UTC](https://discuss.elastic.co/t/filter-latest-record-for-each-user-and-aggregate-on-different-field-in-elasticsearch/250435 "2020-10-02T10:05:18Z")
Need to filter the latest record for each user and aggregate the count on each vote type. Still cannot able to figure out. any help pls? {user:"sam", vote: 20, "createdDate": 1601285178238} {user:"sam", vote: 21, "crea… --- ## [Elasticsearch SQL retrieve more than 1000](https://discuss.elastic.co/t/elasticsearch-sql-retrieve-more-than-1000/250614)
**Author:** [@ananya\_ramesh](https://discuss.elastic.co/u/ananya_ramesh)\ **Replies:** 1\ **Last updated:** [October 2, 2020, 9:38am UTC](https://discuss.elastic.co/t/elasticsearch-sql-retrieve-more-than-1000/250614 "2020-10-02T09:38:02Z")
Implementing SQL query in Elasticsearch, not able to extract records more than 1000 in spite of LIMIT \> 1000. Tried "index.mapping.total\_fields.limit": 5000 , but did not help. POST \_sql?format=txt { "query": "SELECT… --- ## [Ldap user authen sam account option](https://discuss.elastic.co/t/ldap-user-authen-sam-account-option/250767)
**Author:** [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Replies:** 0\ **Last updated:** [October 2, 2020, 9:04am UTC](https://discuss.elastic.co/t/ldap-user-authen-sam-account-option/250767 "2020-10-02T09:04:54Z")
Hi all our organization have some specific rule to the naming of dn for example mine would me "cn=Something something,ou=test..." and my samaccount name would be soso@test.com and now when i have the role mapping to a… --- ## [How many shards for 5 millions of groups?](https://discuss.elastic.co/t/how-many-shards-for-5-millions-of-groups/250756)
**Author:** [@harisk](https://discuss.elastic.co/u/harisk)\ **Replies:** 3\ **Last updated:** [October 2, 2020, 8:26am UTC](https://discuss.elastic.co/t/how-many-shards-for-5-millions-of-groups/250756 "2020-10-02T08:26:28Z")
Now we have about 5 millions of groups and 100 millions of users. There are about 120 million messages every day users sent to the group, approximate 12GiB. We'd like to use Elasticsearch to search group messages with te… --- ## [Exporting data from field with HUGE content](https://discuss.elastic.co/t/exporting-data-from-field-with-huge-content/250760)
**Author:** [@prabhatdreamz](https://discuss.elastic.co/u/prabhatdreamz)\ **Replies:** 0\ **Last updated:** [October 2, 2020, 8:14am UTC](https://discuss.elastic.co/t/exporting-data-from-field-with-huge-content/250760 "2020-10-02T08:14:40Z")
I want to export my ES (v6.0.0) data to csv, for which I have the tool (es2csv). However, one of the fields (message) stores commit messages, and quite a lot of the entries are HUGE. The content can run to multiple pages… --- ## [How to process again documents already ingested](https://discuss.elastic.co/t/how-to-process-again-documents-already-ingested/250716)
**Author:** [@rschirin](https://discuss.elastic.co/u/rschirin)\ **Replies:** 3\ **Last updated:** [October 2, 2020, 6:59am UTC](https://discuss.elastic.co/t/how-to-process-again-documents-already-ingested/250716 "2020-10-02T06:59:53Z")
Hi all, I would like to know if there is a way to process again documents already saved into the indices. I noticed that my Logstash pipeline was not completed so now I know like to process a specific field. in my doc… --- ## [504 Error due to large number of routing parameters](https://discuss.elastic.co/t/504-error-due-to-large-number-of-routing-parameters/250742)
**Author:** [@prosens](https://discuss.elastic.co/u/prosens)\ **Replies:** 2\ **Last updated:** [October 1, 2020, 9:42pm UTC](https://discuss.elastic.co/t/504-error-due-to-large-number-of-routing-parameters/250742 "2020-10-01T21:42:50Z")
We have a large number of shards. The documents are routed using user ids. There are use cases where we need to search across multiple users. When we specify a large number of user ids as the routing parameter, we see a … --- ## [Elasticdump users: does anyone else worry about WARNings on dependencies?](https://discuss.elastic.co/t/elasticdump-users-does-anyone-else-worry-about-warnings-on-dependencies/250480)
**Author:** [@viu](https://discuss.elastic.co/u/viu)\ **Replies:** 2\ **Last updated:** [October 1, 2020, 8:27pm UTC](https://discuss.elastic.co/t/elasticdump-users-does-anyone-else-worry-about-warnings-on-dependencies/250480 "2020-10-01T20:27:57Z")
Hi there, I am a newbie in this community, but lurked as a visitor for some time. I work with a very security aware IT/server administration, and it has made me semi-paranoid. The Elastick stack in use is an older ver… --- ## [Installing Curator add-on with ElasticSearch on Ubuntu Focal Fossa](https://discuss.elastic.co/t/installing-curator-add-on-with-elasticsearch-on-ubuntu-focal-fossa/250727)
**Author:** [@xberg3rmanx](https://discuss.elastic.co/u/xberg3rmanx)\ **Replies:** 0\ **Last updated:** [October 1, 2020, 6:22pm UTC](https://discuss.elastic.co/t/installing-curator-add-on-with-elasticsearch-on-ubuntu-focal-fossa/250727 "2020-10-01T18:22:44Z")
When attempting to install the Curator add-on with ElasticSearch, I am receiving an error on Ubuntu 20.04. The error states "The repository 'http://ppa.launchpad.net/webupd8team/java/ubuntu focal Release' does not have … --- ## [Machine learning](https://discuss.elastic.co/t/machine-learning/250719)
**Author:** [@liorhalperin](https://discuss.elastic.co/u/liorhalperin)\ **Replies:** 1\ **Last updated:** [October 1, 2020, 4:34pm UTC](https://discuss.elastic.co/t/machine-learning/250719 "2020-10-01T16:34:28Z")
I am beginner in the Machine learning module in ES i created single metric job and started it it creates results but I dont see that the job continues to run each 15 minutes or so and create new results what do I mi… --- ## [SAML Issue in ELK 7.6.4](https://discuss.elastic.co/t/saml-issue-in-elk-7-6-4/249890)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 5\ **Last updated:** [October 1, 2020, 3:35pm UTC](https://discuss.elastic.co/t/saml-issue-in-elk-7-6-4/249890 "2020-10-01T15:35:01Z")
Hi All, I was trying to integrate Okta with elasticsearch, getting below error while logging into console, {"statusCode":401,"error":"Unauthorized","message":"\[security\_exception\] unable to authenticate user \[\ **Author:** [@rajkumar.m](https://discuss.elastic.co/u/rajkumar.m)\ **Replies:** 5\ **Last updated:** [October 1, 2020, 2:26pm UTC](https://discuss.elastic.co/t/elastic-api-is-not-working-in-elk6/248751 "2020-10-01T14:26:49Z") Hi Team, I am unable to create search by using elastic API, the same thing is wokring on ELK5 but not working in ELK6. Please help. Please find the below command & error: curl -H 'Authorization:Basic XXXXXXXXXX' -H … --- ## [License question - am I basic or open source?](https://discuss.elastic.co/t/license-question-am-i-basic-or-open-source/247692)
**Author:** [@bn1980](https://discuss.elastic.co/u/bn1980)\ **Replies:** 6\ **Last updated:** [October 1, 2020, 2:10pm UTC](https://discuss.elastic.co/t/license-question-am-i-basic-or-open-source/247692 "2020-10-01T14:10:47Z")
I have installed ELK stack on FreeBSD using pkg manager. Kibana reports my license as BASIC. I don't appear to have the suricata module. Can someone please confirm which license i have? --- ## [Searching through PDF attachments and other documents in ElasticSearch with one query](https://discuss.elastic.co/t/searching-through-pdf-attachments-and-other-documents-in-elasticsearch-with-one-query/250519)
**Author:** [@Bertrand\_Nau](https://discuss.elastic.co/u/Bertrand_Nau)\ **Replies:** 5\ **Last updated:** [October 1, 2020, 1:46pm UTC](https://discuss.elastic.co/t/searching-through-pdf-attachments-and-other-documents-in-elasticsearch-with-one-query/250519 "2020-10-01T13:46:57Z")
Hello everybody, I'm using the ingest-attachment plugin to parse PDF files in an ElasticSearch 7 cluster. Each PDF file gives additional informations to an already existing document. I try to create a query which retri… --- ## [Return only needed fields in response without metadata](https://discuss.elastic.co/t/return-only-needed-fields-in-response-without-metadata/250639)
**Author:** [@Nick\_Name](https://discuss.elastic.co/u/Nick_Name)\ **Replies:** 3\ **Last updated:** [October 1, 2020, 1:24pm UTC](https://discuss.elastic.co/t/return-only-needed-fields-in-response-without-metadata/250639 "2020-10-01T13:24:37Z")
Can I specify my SearchSourceBuilder or SearchRequest for return in SearchResponse only fields without metadata? Include fields and Exclude don't work. I want to return only code.keyword and id.keyword fields in response… --- ## [Upgrading from 2.4 -\> 7.9, Synonym reserved words?](https://discuss.elastic.co/t/upgrading-from-2-4-7-9-synonym-reserved-words/250197)
**Author:** [@ronchalant](https://discuss.elastic.co/u/ronchalant)\ **Replies:** 2\ **Last updated:** [October 1, 2020, 1:22pm UTC](https://discuss.elastic.co/t/upgrading-from-2-4-7-9-synonym-reserved-words/250197 "2020-10-01T13:22:35Z")
We're doing a big upgrade on various indexes, going from 2.4 -\> 7.9. One thing we're running into is we had a number of synonyms that had the words 'and' and 'or' in them, and they're being rejected by 7.9. One of them… --- ## [Bootstrap checks failed](https://discuss.elastic.co/t/bootstrap-checks-failed/250679)
**Author:** [@muthug](https://discuss.elastic.co/u/muthug)\ **Replies:** 4\ **Last updated:** [October 1, 2020, 1:11pm UTC](https://discuss.elastic.co/t/bootstrap-checks-failed/250679 "2020-10-01T13:11:48Z")
\[2020-10-01T14:44:51,442\]\[INFO \]\[o.e.e.NodeEnvironment \] \[graylogES-1\] using \[1\] data paths, mounts \[\[/opt (/dev/mapper/vg00-lvopt)\]\], net usable\_space \[12.8gb\], net total\_space \[25.4gb\], types \[ext4\] \[2020-10-01T14:… --- ## [Querying parent-child documents](https://discuss.elastic.co/t/querying-parent-child-documents/250631)
**Author:** [@Priyam\_Maheshwari](https://discuss.elastic.co/u/Priyam_Maheshwari)\ **Replies:** 1\ **Last updated:** [October 1, 2020, 1:08pm UTC](https://discuss.elastic.co/t/querying-parent-child-documents/250631 "2020-10-01T13:08:46Z")
I have a following index which stores the course details (I have truncated some attributes for brevity): { "settings": { "index": { "number\_of\_replicas": "1", "number\_of\_shards": "1" } }, "alia… --- ## [Change 'total\_auto\_throttle\_in\_bytes' in Elasticsearch](https://discuss.elastic.co/t/change-total-auto-throttle-in-bytes-in-elasticsearch/249463)
**Author:** [@Bahar\_Yilmaz](https://discuss.elastic.co/u/Bahar_Yilmaz)\ **Replies:** 3\ **Last updated:** [October 1, 2020, 12:35pm UTC](https://discuss.elastic.co/t/change-total-auto-throttle-in-bytes-in-elasticsearch/249463 "2020-10-01T12:35:21Z")
Is there any way to change ' total\_auto\_throttle\_in\_bytes' value in Elasticsearch. Is this value changable ? Thanks in advance. --- ## [Complex query with priorities](https://discuss.elastic.co/t/complex-query-with-priorities/250667)
**Author:** [@Patrik\_Vaclavek](https://discuss.elastic.co/u/Patrik_Vaclavek)\ **Replies:** 0\ **Last updated:** [October 1, 2020, 12:00pm UTC](https://discuss.elastic.co/t/complex-query-with-priorities/250667 "2020-10-01T12:00:24Z")
Hello, I am struggling to figure this out. I've got products with many prices based on available discounts and price lists with assigned priorities. The 'price' part mapping looks like this: "price": { "properties"… --- ## [Double documents?](https://discuss.elastic.co/t/double-documents/248991)
**Author:** [@Tuckson](https://discuss.elastic.co/u/Tuckson)\ **Replies:** 13\ **Last updated:** [October 1, 2020, 10:56am UTC](https://discuss.elastic.co/t/double-documents/248991 "2020-10-01T10:56:48Z")
Hi, For some reason, all documents end up twice in my indexes. and I do not have the slightest idea why... I mean... Even the doc\_id seems the same. Consider these 2: Both exactly the same. When I log this on the a… --- ## [System Requirements](https://discuss.elastic.co/t/system-requirements/250649)
**Author:** [@PatNae](https://discuss.elastic.co/u/PatNae)\ **Replies:** 0\ **Last updated:** [October 1, 2020, 10:42am UTC](https://discuss.elastic.co/t/system-requirements/250649 "2020-10-01T10:42:33Z")
Hi all I am working on a project where data is delivered to elasticsearch using winlogbeat - logstash - kiibana. There are 4 source servers, each 1 file with 1 GB, so 4 \* 1 GB, these data should be stored 180 Tg, so 411… --- ## [Elasticsearch stopped due to (code = kill, signal = kill)](https://discuss.elastic.co/t/elasticsearch-stopped-due-to-code-kill-signal-kill/250622)
**Author:** [@sai92](https://discuss.elastic.co/u/sai92)\ **Replies:** 1\ **Last updated:** [October 1, 2020, 8:48am UTC](https://discuss.elastic.co/t/elasticsearch-stopped-due-to-code-kill-signal-kill/250622 "2020-10-01T08:48:49Z")
Hi, I use elasticsearch to store data from different IoT sensors and it was working good but elasticsearch stopped and when i check the log it show me like this can you please say which thing caused this error. elastic… --- ## [Esrally command line parameters](https://discuss.elastic.co/t/esrally-command-line-parameters/250507)
**Author:** [@Oleg\_Ruchovets](https://discuss.elastic.co/u/Oleg_Ruchovets)\ **Replies:** 1\ **Last updated:** [October 1, 2020, 7:04am UTC](https://discuss.elastic.co/t/esrally-command-line-parameters/250507 "2020-10-01T07:04:58Z")
Hello, I am running a set of rally tests and I would like to preserve data/index and run next time with existing data in elastic. I tried to use --preserve-install=true option but it is not accepted. esrally: error: a… --- ## [Elastic Search Warm node taking too much of Ram than the Heap assigned to that](https://discuss.elastic.co/t/elastic-search-warm-node-taking-too-much-of-ram-than-the-heap-assigned-to-that/250605)
**Author:** [@Ozil](https://discuss.elastic.co/u/Ozil)\ **Replies:** 0\ **Last updated:** [October 1, 2020, 6:53am UTC](https://discuss.elastic.co/t/elastic-search-warm-node-taking-too-much-of-ram-than-the-heap-assigned-to-that/250605 "2020-10-01T06:53:45Z")
Hi All I am using two data nodes to achieve hot and warm architecture. Both are assigned with 8 GB RAM as Heap Size. The total available is 64 GB RAM. I am using the rollover and allocation settings to achieve my hot wa… --- ## [Index.lifecycle.rollover\_alias \[filebeat-7.8.0\] does not point to index](https://discuss.elastic.co/t/index-lifecycle-rollover-alias-filebeat-7-8-0-does-not-point-to-index/245642)
**Author:** [@thomas7467](https://discuss.elastic.co/u/thomas7467)\ **Replies:** 4\ **Last updated:** [October 1, 2020, 6:23am UTC](https://discuss.elastic.co/t/index-lifecycle-rollover-alias-filebeat-7-8-0-does-not-point-to-index/245642 "2020-10-01T06:23:56Z")
Hi All, I've the following errors il eleastic log files for the indexes Lifecycle Management for the indexes and don't know how to solve them... Could someone give me some clues? Thanks, \*java.lang.IllegalArgumentEx… --- ## [Filebeat is not sending logs to Logstash, logstash is localhost](https://discuss.elastic.co/t/filebeat-is-not-sending-logs-to-logstash-logstash-is-localhost/250588)
**Author:** [@Sravan\_Kumar\_Guduru](https://discuss.elastic.co/u/Sravan_Kumar_Guduru)\ **Replies:** 1\ **Last updated:** [October 1, 2020, 3:54am UTC](https://discuss.elastic.co/t/filebeat-is-not-sending-logs-to-logstash-logstash-is-localhost/250588 "2020-10-01T03:54:04Z")
Hi All, I am having trouble with my filebeat/logstash. connectivity seems to be inconsistenst. When i try it in debug mode it works fine but when i use it with systemctl start it fails . Below are my filebeat and logsta… --- ## [Convert Elasticsearch output to CSV file using "jq"](https://discuss.elastic.co/t/convert-elasticsearch-output-to-csv-file-using-jq/250587)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 1\ **Last updated:** [October 1, 2020, 3:49am UTC](https://discuss.elastic.co/t/convert-elasticsearch-output-to-csv-file-using-jq/250587 "2020-10-01T03:49:01Z")
Hi Team Could you please help me with syntax of jq to be used for getting , entire "\_source" contents of elasticsearch index into single row of csv file. I am trying out multiple ways but its not working ... Thanks T… --- ## [ELK Architecture considerations in AWS before deployment](https://discuss.elastic.co/t/elk-architecture-considerations-in-aws-before-deployment/250143)
**Author:** [@myrsecurity](https://discuss.elastic.co/u/myrsecurity)\ **Replies:** 1\ **Last updated:** [October 1, 2020, 1:44am UTC](https://discuss.elastic.co/t/elk-architecture-considerations-in-aws-before-deployment/250143 "2020-10-01T01:44:02Z")
Hi all, we are planning to deploy ELK in AWS infra to receive feeds from around \<500 linux beats ( Metricbeats feed only for now ). In terms of minimum capacity lowest cost what would you estimate we need to cover such n… --- ## [Enable ILM for different filebeat indices](https://discuss.elastic.co/t/enable-ilm-for-different-filebeat-indices/250350)
**Author:** [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)\ **Replies:** 1\ **Last updated:** [October 1, 2020, 1:40am UTC](https://discuss.elastic.co/t/enable-ilm-for-different-filebeat-indices/250350 "2020-10-01T01:40:22Z")
Hi, I have two different indices in my filebeat. one for cisco module (cisco-beats-0001) and one for the rest of the logs(default filebeat index). so far: I've disabled ILM in filebeat I've created an index lifecycle… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=794) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=796)