# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=221 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 222 --- ## [Elasticsearch Python Lib](https://discuss.elastic.co/t/elasticsearch-python-lib/339751)
**Author:** [@Vivek\_Burman](https://discuss.elastic.co/u/Vivek_Burman)\ **Replies:** 0\ **Last updated:** [August 1, 2023, 6:17am UTC](https://discuss.elastic.co/t/elasticsearch-python-lib/339751 "2023-08-01T06:17:58Z")
Hi, I'm using Elastic Search python lib (8.8.2) to bulk insert data into a index. There are almost 2lakh+ documents I need to insert. I'm using parallel\_bulk api to sync them, but as I track the process RAM usage I see … --- ## [At which step does Bulkresponse occur](https://discuss.elastic.co/t/at-which-step-does-bulkresponse-occur/339740)
**Author:** [@Logan-lxw](https://discuss.elastic.co/u/Logan-lxw)\ **Replies:** 0\ **Last updated:** [August 1, 2023, 3:40am UTC](https://discuss.elastic.co/t/at-which-step-does-bulkresponse-occur/339740 "2023-08-01T03:40:24Z")
At which stage does the BulkResponse corresponding to BulkRequest occur? Which step does this response specifically refer to before returning? Does it mean that the request was successfully written to the translog and fl… --- ## [Runtime get month()](https://discuss.elastic.co/t/runtime-get-month/339721)
**Author:** [@Murilo\_Livorato](https://discuss.elastic.co/u/Murilo_Livorato)\ **Replies:** 1\ **Last updated:** [August 1, 2023, 3:07am UTC](https://discuss.elastic.co/t/runtime-get-month/339721 "2023-08-01T03:07:22Z")
hello , I have this runtime - GET my-index-000006-test/\_search { "runtime\_mappings": { "day\_of\_week": { "type": "keyword", "script": { "source": "emit(doc\['timestamp'\].value.dayOfWeek… --- ## [An index has stayed on the same action longer than expected](https://discuss.elastic.co/t/an-index-has-stayed-on-the-same-action-longer-than-expected/339636)
**Author:** [@nitinkapoor](https://discuss.elastic.co/u/nitinkapoor)\ **Replies:** 2\ **Last updated:** [August 1, 2023, 1:46am UTC](https://discuss.elastic.co/t/an-index-has-stayed-on-the-same-action-longer-than-expected/339636 "2023-08-01T01:46:06Z")
I have upgraded my Elastic stack from 8.82 to 8.9 and facing this warning for one of my indices Deployment\_management Automatic index lifecycle and data retention management cannot make progress on one or more indices.… --- ## [Select latest docs for each transactions and apply filters on selections with pagination capability](https://discuss.elastic.co/t/select-latest-docs-for-each-transactions-and-apply-filters-on-selections-with-pagination-capability/339730)
**Author:** [@pramodbhade](https://discuss.elastic.co/u/pramodbhade)\ **Replies:** 0\ **Last updated:** [July 31, 2023, 8:12pm UTC](https://discuss.elastic.co/t/select-latest-docs-for-each-transactions-and-apply-filters-on-selections-with-pagination-capability/339730 "2023-07-31T20:12:31Z")
I have a situation where I need to load 100 records per page with the latest values and be able to paginate as well. The selected latest records also get filtered for a set of filter values. I'm using aggregation in th… --- ## [It is that possible to I return in my Elasticsearch query a new field , that does not exist in the mapping . With a new format from other field?](https://discuss.elastic.co/t/it-is-that-possible-to-i-return-in-my-elasticsearch-query-a-new-field-that-does-not-exist-in-the-mapping-with-a-new-format-from-other-field/339616)
**Author:** [@Murilo\_Livorato](https://discuss.elastic.co/u/Murilo_Livorato)\ **Replies:** 4\ **Last updated:** [July 31, 2023, 6:46pm UTC](https://discuss.elastic.co/t/it-is-that-possible-to-i-return-in-my-elasticsearch-query-a-new-field-that-does-not-exist-in-the-mapping-with-a-new-format-from-other-field/339616 "2023-07-31T18:46:41Z")
It is that possible to I return in my Elasticsearch query a new field ( that does not exist in the mapping ), with a new format from other field ? something like that - This is my Mapping - PUT /user-product-2023-06… --- ## [Cardinality Limitation Work Around](https://discuss.elastic.co/t/cardinality-limitation-work-around/339453)
**Author:** [@edang](https://discuss.elastic.co/u/edang)\ **Replies:** 0\ **Last updated:** [July 27, 2023, 2:44pm UTC](https://discuss.elastic.co/t/cardinality-limitation-work-around/339453 "2023-07-27T14:44:22Z")
Hi All, With my data set I have seen a mismatch of data between ELK and my DB. For my purpose, I have used the cardinality aggregation to count the unique ids of a field but ran into some issues. The issues comes from t… --- ## [Autocomplete using Completion Suggesters](https://discuss.elastic.co/t/autocomplete-using-completion-suggesters/338823)
**Author:** [@Senchok](https://discuss.elastic.co/u/Senchok)\ **Replies:** 7\ **Last updated:** [July 31, 2023, 5:12pm UTC](https://discuss.elastic.co/t/autocomplete-using-completion-suggesters/338823 "2023-07-31T17:12:04Z")
Hello, I want to write Autocomplete using Elasticsearch. I use Completion Suggesters and I have problems with it. For example I have fullName and title fields "fullName": "John Smith" "title": "Python Developer" B… --- ## [Collecting Sophos XG logs using the Sophos integration feature](https://discuss.elastic.co/t/collecting-sophos-xg-logs-using-the-sophos-integration-feature/339565)
**Author:** [@TIT](https://discuss.elastic.co/u/TIT)\ **Replies:** 5\ **Last updated:** [July 31, 2023, 5:10pm UTC](https://discuss.elastic.co/t/collecting-sophos-xg-logs-using-the-sophos-integration-feature/339565 "2023-07-31T17:10:00Z")
Hello, I'm currently trying to integrate Sophos XG firewall logs into my ELK stack via the Filebeat Sophos module. My setup involves sending logs directly from my Sophos XG device to Elasticsearch, bypassing Logstash. T… --- ## [Where is the certificate authority that signs elastic images using cosign?](https://discuss.elastic.co/t/where-is-the-certificate-authority-that-signs-elastic-images-using-cosign/339699)
**Author:** [@data\_smith](https://discuss.elastic.co/u/data_smith)\ **Replies:** 0\ **Last updated:** [July 31, 2023, 1:45pm UTC](https://discuss.elastic.co/t/where-is-the-certificate-authority-that-signs-elastic-images-using-cosign/339699 "2023-07-31T13:45:05Z")
Elastic now signs images using cosign to strengthen the supply chain. I'd like to run these images in Kubernetes and use Kyverno to verify the images but i get the error: "certificate signed by unknown authority". Wher… --- ## [Searching with runtime, without mapping with runtime](https://discuss.elastic.co/t/searching-with-runtime-without-mapping-with-runtime/339696)
**Author:** [@Murilo\_Livorato](https://discuss.elastic.co/u/Murilo_Livorato)\ **Replies:** 2\ **Last updated:** [July 31, 2023, 1:48pm UTC](https://discuss.elastic.co/t/searching-with-runtime-without-mapping-with-runtime/339696 "2023-07-31T13:48:55Z")
hello , I am trying to learn runtime . it uses in the script the painless language , It wold be nice to I understand this language , that I could test it before mapping . the way to test it before mapping it is search … --- ## [Filter vector search by similarity value](https://discuss.elastic.co/t/filter-vector-search-by-similarity-value/339654)
**Author:** [@john\_nicolas](https://discuss.elastic.co/u/john_nicolas)\ **Replies:** 1\ **Last updated:** [July 31, 2023, 11:53am UTC](https://discuss.elastic.co/t/filter-vector-search-by-similarity-value/339654 "2023-07-31T11:53:40Z")
i used in mappings file a dense\_vector for knn "esvector": { "type": "dense\_vector", "dims": 768, "index": true, "similarity": "cosine" }, I want to use a similarity threshold ,… --- ## [Split index into subindexes by dates like 'my-index-yyyy.MM.dd'](https://discuss.elastic.co/t/split-index-into-subindexes-by-dates-like-my-index-yyyy-mm-dd/339330)
**Author:** [@tyro\_plotter](https://discuss.elastic.co/u/tyro_plotter)\ **Replies:** 4\ **Last updated:** [July 31, 2023, 11:49am UTC](https://discuss.elastic.co/t/split-index-into-subindexes-by-dates-like-my-index-yyyy-mm-dd/339330 "2023-07-31T11:49:04Z")
I am considering two options: Time Series Ingest pipeline (Date index name processor) I am aware that they differ in their intended use, but I am trying to understand what risks there are. As a newbie to the time ser… --- ## [Too many fields in an index](https://discuss.elastic.co/t/too-many-fields-in-an-index/339679)
**Author:** [@Jurrien](https://discuss.elastic.co/u/Jurrien)\ **Replies:** 0\ **Last updated:** [July 31, 2023, 10:55am UTC](https://discuss.elastic.co/t/too-many-fields-in-an-index/339679 "2023-07-31T10:55:28Z")
We have an index with the following structure (see below) So basically, we have our index business\_objects with a link and no. We add objects to this index (doc\_type1, doc\_type2, ....). These objects are linked via no … --- ## [Word count using Logstash Pipeline](https://discuss.elastic.co/t/word-count-using-logstash-pipeline/339678)
**Author:** [@rvadiga](https://discuss.elastic.co/u/rvadiga)\ **Replies:** 0\ **Last updated:** [July 31, 2023, 10:53am UTC](https://discuss.elastic.co/t/word-count-using-logstash-pipeline/339678 "2023-07-31T10:53:13Z")
Hi Team, I am trying to build and design a logstash pipeline where the count of different words tracked against the timestamp. I need to classify every word based on length of texts in three segments, say words with … --- ## [Support hieroglyphs and symbols](https://discuss.elastic.co/t/support-hieroglyphs-and-symbols/339677)
**Author:** [@viachaslau](https://discuss.elastic.co/u/viachaslau)\ **Replies:** 0\ **Last updated:** [July 31, 2023, 10:45am UTC](https://discuss.elastic.co/t/support-hieroglyphs-and-symbols/339677 "2023-07-31T10:45:39Z")
What analyzer I should use for support hieroglyphs and symbols. I cant use ICU because It remove symbols. --- ## [Maximum normal shards open achived](https://discuss.elastic.co/t/maximum-normal-shards-open-achived/339529)
**Author:** [@Patryk\_Ostrowski](https://discuss.elastic.co/u/Patryk_Ostrowski)\ **Replies:** 6\ **Last updated:** [July 31, 2023, 9:10am UTC](https://discuss.elastic.co/t/maximum-normal-shards-open-achived/339529 "2023-07-31T09:10:58Z")
Hello, I have one node ELK, I know that is not the best solution, but I cannot change that. I put logs to ELK, and every day I have new index for example: alerts-2023-07-23. But after few months of working filebeat showe… --- ## [How to add thousand of objects](https://discuss.elastic.co/t/how-to-add-thousand-of-objects/339124)
**Author:** [@senadk](https://discuss.elastic.co/u/senadk)\ **Replies:** 2\ **Last updated:** [July 31, 2023, 8:46am UTC](https://discuss.elastic.co/t/how-to-add-thousand-of-objects/339124 "2023-07-31T08:46:08Z")
Hi everyone, Im new to Elastic and i can't find a way to add big data (read 150k SQL rows) at once to my index. Im using postman to execute the endpoints like \_bulk. What i would like is to copy my 150k rows from my S… --- ## [\[o.e.t.TransportService\] Received response for a request that has timed out](https://discuss.elastic.co/t/o-e-t-transportservice-received-response-for-a-request-that-has-timed-out/339056)
**Author:** [@EVINDX](https://discuss.elastic.co/u/EVINDX)\ **Replies:** 16\ **Last updated:** [July 31, 2023, 7:54am UTC](https://discuss.elastic.co/t/o-e-t-transportservice-received-response-for-a-request-that-has-timed-out/339056 "2023-07-31T07:54:54Z")
We are receiving the following error {ElasticsearchLogger} \[o.e.t.TransportService\] Received response for a request that has timed out, sent \[21.3s/21361ms\] ago, timed out \[5.6s/5682ms\] ago, action \[indices:monitor/stat… --- ## [Does ES security settings "xpack.security.transport.filter.allow" conflict with eBPF program?](https://discuss.elastic.co/t/does-es-security-settings-xpack-security-transport-filter-allow-conflict-with-ebpf-program/339652)
**Author:** [@Michael\_K\_Aboagye](https://discuss.elastic.co/u/Michael_K_Aboagye)\ **Replies:** 0\ **Last updated:** [July 31, 2023, 7:17am UTC](https://discuss.elastic.co/t/does-es-security-settings-xpack-security-transport-filter-allow-conflict-with-ebpf-program/339652 "2023-07-31T07:17:23Z")
ES documentation states that developers/admins can filter IP addresses at the transport layer via this parameter: xpack.security.transport.filter.allow . So let's assume I have configured the parameter xpack.security.t… --- ## [Auto authentication of user in python application](https://discuss.elastic.co/t/auto-authentication-of-user-in-python-application/339648)
**Author:** [@Rushi\_Bagul](https://discuss.elastic.co/u/Rushi_Bagul)\ **Replies:** 0\ **Last updated:** [July 31, 2023, 6:50am UTC](https://discuss.elastic.co/t/auto-authentication-of-user-in-python-application/339648 "2023-07-31T06:50:59Z")
Hi team, I have my django application in which I am rendering iframed Kibana dashboard. If am log in django application at same time log in iframed Kibana dashboard as well, it won't log in again in Kibana dashboard. S… --- ## [Elasticsearch Memory Utilization](https://discuss.elastic.co/t/elasticsearch-memory-utilization/338928)
**Author:** [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)\ **Replies:** 8\ **Last updated:** [July 31, 2023, 6:31am UTC](https://discuss.elastic.co/t/elasticsearch-memory-utilization/338928 "2023-07-31T06:31:44Z")
Hi Team, I am new to the Elasticsearch world. I had installed the Elasticsearch in one of my lab VM, where it is showing memory utilization is 8.3 GB when I check "systemctl status elasticsearch" and VM gets hang. Is t… --- ## [Group By Datetime fields While querying](https://discuss.elastic.co/t/group-by-datetime-fields-while-querying/339639)
**Author:** [@cybercom](https://discuss.elastic.co/u/cybercom)\ **Replies:** 0\ **Last updated:** [July 31, 2023, 4:03am UTC](https://discuss.elastic.co/t/group-by-datetime-fields-while-querying/339639 "2023-07-31T04:03:12Z")
I need to group by day in my query, so i'm trying to apply group by with datetime field as below: { "\_source": "false", "query": { "match\_all": {} }, "aggs": { "group\_by\_weekday": { … --- ## [Does fscrawler support opensearch?](https://discuss.elastic.co/t/does-fscrawler-support-opensearch/339613)
**Author:** [@nadiGam](https://discuss.elastic.co/u/nadiGam)\ **Replies:** 2\ **Last updated:** [July 30, 2023, 3:26pm UTC](https://discuss.elastic.co/t/does-fscrawler-support-opensearch/339613 "2023-07-30T15:26:37Z")
--- ## [Data storage location for elasticseach on docker](https://discuss.elastic.co/t/data-storage-location-for-elasticseach-on-docker/339602)
**Author:** [@Mataz](https://discuss.elastic.co/u/Mataz)\ **Replies:** 1\ **Last updated:** [July 29, 2023, 5:47pm UTC](https://discuss.elastic.co/t/data-storage-location-for-elasticseach-on-docker/339602 "2023-07-29T17:47:31Z")
I am trying to install elasticsearch for docker but I got stuck with configuring the data storage location for the logs collected from the log aggregators coming to elasticsearch. Basically I need to store the data on th… --- ## [Add alias to existing indices (and newly created indices) using Kibana UI](https://discuss.elastic.co/t/add-alias-to-existing-indices-and-newly-created-indices-using-kibana-ui/339147)
**Author:** [@shawnmin](https://discuss.elastic.co/u/shawnmin)\ **Replies:** 3\ **Last updated:** [July 29, 2023, 2:54am UTC](https://discuss.elastic.co/t/add-alias-to-existing-indices-and-newly-created-indices-using-kibana-ui/339147 "2023-07-29T02:54:27Z")
I've set up an EFK stack on my Kubernetes cluster. I want to automatically delete indices after certain days later (i.e., log retention and rotation), so I've created an index lifecycle policy. The policy's name is dele… --- ## [Simple Anomaly Detection Question](https://discuss.elastic.co/t/simple-anomaly-detection-question/339580)
**Author:** [@McJava1967](https://discuss.elastic.co/u/McJava1967)\ **Replies:** 1\ **Last updated:** [July 29, 2023, 1:01am UTC](https://discuss.elastic.co/t/simple-anomaly-detection-question/339580 "2023-07-29T01:01:04Z")
Hi all. I'm a newbie at Anomaly Detection. Let's say I have a key, "PET", with two possible values, "CAT" and "DOG". I want to detect when there are an unusual number of CATs in an hour. Is that possible? I thought … --- ## [.NET client connect to elastic search using SSL](https://discuss.elastic.co/t/net-client-connect-to-elastic-search-using-ssl/339554)
**Author:** [@Nilesh\_Jethwani](https://discuss.elastic.co/u/Nilesh_Jethwani)\ **Replies:** 0\ **Last updated:** [July 28, 2023, 1:43pm UTC](https://discuss.elastic.co/t/net-client-connect-to-elastic-search-using-ssl/339554 "2023-07-28T13:43:53Z")
We want to connect our .NET application to elasticsearch 8.5.2 single node cluster. We want to use https and ssl communication. We have 3 certificates from our CA. root , intermediate and main along with private key. … --- ## [Phrase suggester giving suggestion on correct terms containing number values](https://discuss.elastic.co/t/phrase-suggester-giving-suggestion-on-correct-terms-containing-number-values/339579)
**Author:** [@Pavithra2014](https://discuss.elastic.co/u/Pavithra2014)\ **Replies:** 0\ **Last updated:** [July 28, 2023, 6:14pm UTC](https://discuss.elastic.co/t/phrase-suggester-giving-suggestion-on-correct-terms-containing-number-values/339579 "2023-07-28T18:14:33Z")
Team, We are using Phrase suggestion with below configuration. but this is returning suggestion on correct speeled words having numeric values on it. eg: Product 2023 is giving the suggestion Product 2022 . I'm expect… --- ## [Pass min\_score to shoudl close](https://discuss.elastic.co/t/pass-min-score-to-shoudl-close/339561)
**Author:** [@john\_nicolas](https://discuss.elastic.co/u/john_nicolas)\ **Replies:** 0\ **Last updated:** [July 28, 2023, 2:16pm UTC](https://discuss.elastic.co/t/pass-min-score-to-shoudl-close/339561 "2023-07-28T14:16:43Z")
i want to pass a min\_score but for only should clauses, i want filter should clause by min\_score to eliminate docs which have cosinesimilarity poor {'bool': {'filter': {'term': {'hidden': 'false'}}, 'must': \[{'bool': {'… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=220) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=222)