# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=385 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 386 --- ## [Mapper\_parsing\_exception: failed to parse ( Dataframe to Elastic - batch process)](https://discuss.elastic.co/t/mapper-parsing-exception-failed-to-parse-dataframe-to-elastic-batch-process/312930)
**Author:** [@visit2siva](https://discuss.elastic.co/u/visit2siva)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 2:10pm UTC](https://discuss.elastic.co/t/mapper-parsing-exception-failed-to-parse-dataframe-to-elastic-batch-process/312930 "2022-08-25T14:10:38Z")
We are adding documents through a pyspark dataframe to elastic. df.write.mode('append').format('org.elasticsearch.spark.sql') .option('es.nodes', 'https://xyz.com') .option('es.net.http.auth.user','userid') .optio… --- ## [Unable to run ElasticSearch after setting up SSL certificate](https://discuss.elastic.co/t/unable-to-run-elasticsearch-after-setting-up-ssl-certificate/312774)
**Author:** [@RonH8](https://discuss.elastic.co/u/RonH8)\ **Replies:** 2\ **Last updated:** [August 25, 2022, 9:34am UTC](https://discuss.elastic.co/t/unable-to-run-elasticsearch-after-setting-up-ssl-certificate/312774 "2022-08-25T09:34:09Z")
Hey I've followed the following guide to setup the SSL certificate for my Elasticsearch which is running on Ubuntu 22.04.1 LTS: https://www.elastic.co/guide/en/elasticsearch/reference/current/security-basic-setup.html … --- ## [Runtime Environment Requirement for FS Crawler](https://discuss.elastic.co/t/runtime-environment-requirement-for-fs-crawler/312753)
**Author:** [@zxuz111](https://discuss.elastic.co/u/zxuz111)\ **Replies:** 1\ **Last updated:** [August 25, 2022, 12:07pm UTC](https://discuss.elastic.co/t/runtime-environment-requirement-for-fs-crawler/312753 "2022-08-25T12:07:39Z")
We are planning to deploy FS Crawler to the VM on Azure, and I am trying to provisioning the VM. Here is the information to support the provision: OS: Ubuntu 20.4 PDF files total size: 1.5 TB FS Crawler Schedule: Set a… --- ## [Can Elasticsearch return position of the text within document](https://discuss.elastic.co/t/can-elasticsearch-return-position-of-the-text-within-document/312901)
**Author:** [@Dinesh\_Zende](https://discuss.elastic.co/u/Dinesh_Zende)\ **Replies:** 4\ **Last updated:** [August 25, 2022, 11:45am UTC](https://discuss.elastic.co/t/can-elasticsearch-return-position-of-the-text-within-document/312901 "2022-08-25T11:45:59Z")
I have created index with the attributes \, where the contents are the page content of the document. When I search for the particular query, it returns the appropriate Filename, Pageno. But to b… --- ## [Failed to setup SSL Elasticsearch](https://discuss.elastic.co/t/failed-to-setup-ssl-elasticsearch/312775)
**Author:** [@Sathiya](https://discuss.elastic.co/u/Sathiya)\ **Replies:** 3\ **Last updated:** [August 25, 2022, 11:40am UTC](https://discuss.elastic.co/t/failed-to-setup-ssl-elasticsearch/312775 "2022-08-25T11:40:59Z")
Hi All, I have tried to implement SSL for ELK which has 3 nodes in the cluster. So SAN certificate has been generated with all 3 node’s hostname and FQDN and exported as .pfx (PKCS12) format to configure same in elasti… --- ## [Index size based on Kibana Query Output](https://discuss.elastic.co/t/index-size-based-on-kibana-query-output/312909)
**Author:** [@hari\_priya1](https://discuss.elastic.co/u/hari_priya1)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 11:12am UTC](https://discuss.elastic.co/t/index-size-based-on-kibana-query-output/312909 "2022-08-25T11:12:15Z")
I have an index with a key field called adapter-id which I have to aggregate by adapter\_ids and calculate the index storage size for those buckets. Is it possible to get the index size concerning a certain key field in … --- ## [Permission error while trying to access file which is under mounted volume](https://discuss.elastic.co/t/permission-error-while-trying-to-access-file-which-is-under-mounted-volume/312881)
**Author:** [@cihady](https://discuss.elastic.co/u/cihady)\ **Replies:** 1\ **Last updated:** [August 25, 2022, 11:08am UTC](https://discuss.elastic.co/t/permission-error-while-trying-to-access-file-which-is-under-mounted-volume/312881 "2022-08-25T11:08:26Z")
I cant access the files that are in mounted volume elasticsearch | {"type": "server", "timestamp": "2022-08-25T07:52:51,332Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.na… --- ## [Elastic search issue with Open Distro](https://discuss.elastic.co/t/elastic-search-issue-with-open-distro/312886)
**Author:** [@smabdulazeez](https://discuss.elastic.co/u/smabdulazeez)\ **Replies:** 3\ **Last updated:** [August 25, 2022, 10:48am UTC](https://discuss.elastic.co/t/elastic-search-issue-with-open-distro/312886 "2022-08-25T10:48:34Z")
we have defined the following in .ini file: Assigned Values: opendistro\_security.ssl.http.enabled= True Kesystore type = PKCS12 keystore\_filepath = /relativepath.pfx truststore\_type = PKCS12 truststore-filepath =/relati… --- ## [Serveur status RED](https://discuss.elastic.co/t/serveur-status-red/312891)
**Author:** [@Joachim\_Rodrigues](https://discuss.elastic.co/u/Joachim_Rodrigues)\ **Replies:** 1\ **Last updated:** [August 25, 2022, 10:16am UTC](https://discuss.elastic.co/t/serveur-status-red/312891 "2022-08-25T10:16:07Z")
Hi When i run : GET /\_cluster/health I can see that the cluster is RED : { "cluster\_name" : "es-monit-cluster", "status" : "red", "timed\_out" : false, "number\_of\_nodes" : 7, "number\_of\_data\_nodes" : 5, "acti… --- ## [Move data between data stream backing indexes](https://discuss.elastic.co/t/move-data-between-data-stream-backing-indexes/312803)
**Author:** [@KAFFEdev](https://discuss.elastic.co/u/KAFFEdev)\ **Replies:** 8\ **Last updated:** [August 25, 2022, 8:53am UTC](https://discuss.elastic.co/t/move-data-between-data-stream-backing-indexes/312803 "2022-08-25T08:53:59Z")
We collect telemetry from our systems and store each event in a service bus, ready for ingestion. We have also setup a data stream for ingesting telemetry events, with a corresponding ILM policy with automatic rollover, … --- ## [Unable to parse date value '2022-12-16T10:55:22.000Z' of property 'UserSign.signDate' with configured converters](https://discuss.elastic.co/t/unable-to-parse-date-value-2022-12-16t1022-000z-of-property-usersign-signdate-with-configured-converters/312876)
**Author:** [@zhou\_Mr](https://discuss.elastic.co/u/zhou_Mr)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 7:23am UTC](https://discuss.elastic.co/t/unable-to-parse-date-value-2022-12-16t1022-000z-of-property-usersign-signdate-with-configured-converters/312876 "2022-08-25T07:23:15Z")
Why can't the name attribute be mapped to the data in elasticsearch without adding it, but a date conversion error will occur after adding it？？？ @JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss",timezone="GMT+8") @DateTimeFo… --- ## [High Disk space utilization Elastic Indices](https://discuss.elastic.co/t/high-disk-space-utilization-elastic-indices/312855)
**Author:** [@AmarKolhapuri](https://discuss.elastic.co/u/AmarKolhapuri)\ **Replies:** 4\ **Last updated:** [August 25, 2022, 1:08am UTC](https://discuss.elastic.co/t/high-disk-space-utilization-elastic-indices/312855 "2022-08-25T01:08:53Z")
Hi All, On the ELK mount, we are seeing high diskspace utilization. We have docker image of elasticsearch 7.14 and indices is consuming 2.8T of space. Can we do housekeeping on this or update the ELK policy?. Or how to … --- ## [How to setup a single node cluster for Elasticsearch 8.3 on Windows 10 on a laptop?](https://discuss.elastic.co/t/how-to-setup-a-single-node-cluster-for-elasticsearch-8-3-on-windows-10-on-a-laptop/312530)
**Author:** [@JamesD\_7](https://discuss.elastic.co/u/JamesD_7)\ **Replies:** 13\ **Last updated:** [August 25, 2022, 12:54am UTC](https://discuss.elastic.co/t/how-to-setup-a-single-node-cluster-for-elasticsearch-8-3-on-windows-10-on-a-laptop/312530 "2022-08-25T00:54:07Z")
Hello All, I downloaded Elasticsearch 8.3 and Kibana 8.3 and I've run into lots of questions and issues about setting up a simple configuration to read a small CSV file. After running into some errors I started chang… --- ## [Using elastic 8.3.2 in java web services & connecting to multiple elastic nodes more than 3 nodes](https://discuss.elastic.co/t/using-elastic-8-3-2-in-java-web-services-connecting-to-multiple-elastic-nodes-more-than-3-nodes/312735)
**Author:** [@BaswaPrasad](https://discuss.elastic.co/u/BaswaPrasad)\ **Replies:** 1\ **Last updated:** [August 25, 2022, 12:43am UTC](https://discuss.elastic.co/t/using-elastic-8-3-2-in-java-web-services-connecting-to-multiple-elastic-nodes-more-than-3-nodes/312735 "2022-08-25T00:43:31Z")
Hello, 'I am trying to connect to multiple elastic nodes by configuring the nodes details in properties and using them i am creating the Elasticsearchclient like below and creating for across the container and by aut… --- ## [Snapshot and restore](https://discuss.elastic.co/t/snapshot-and-restore/312723)
**Author:** [@mujtabah](https://discuss.elastic.co/u/mujtabah)\ **Replies:** 1\ **Last updated:** [August 25, 2022, 12:42am UTC](https://discuss.elastic.co/t/snapshot-and-restore/312723 "2022-08-25T00:42:47Z")
Hi Is there a way if I can restore the snapshot that is stored on shared folder to a fresh independent elastic node. Here in my case i have taken a snapshot and stored on shared folder, now i created a independent elas… --- ## [Elasticsearch Java monitor](https://discuss.elastic.co/t/elasticsearch-java-monitor/312792)
**Author:** [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\ **Replies:** 0\ **Last updated:** [August 24, 2022, 10:27am UTC](https://discuss.elastic.co/t/elasticsearch-java-monitor/312792 "2022-08-24T10:27:56Z")
Hello, I am using ES 7 in docker. Which tool are you using to monitor heap in docker? I have monitoring on the servers by Node Exporter but I need a tool to collect heap from running OS like JVM. Ideally, it would be m… --- ## [Elasticsearch fleet error](https://discuss.elastic.co/t/elasticsearch-fleet-error/312117)
**Author:** [@alex\_96](https://discuss.elastic.co/u/alex_96)\ **Replies:** 4\ **Last updated:** [August 24, 2022, 11:23pm UTC](https://discuss.elastic.co/t/elasticsearch-fleet-error/312117 "2022-08-24T23:23:29Z")
Good evening, I use elastic 7.17.5 and fleet 7.17, on Sunday after 18:00 notifications to Microsoft 365 integration stopped coming, or rather they do, but a strictly fixed amount in a period of time, the error is as foll… --- ## [API: UpdateRequest question](https://discuss.elastic.co/t/api-updaterequest-question/312843)
**Author:** [@linkerc](https://discuss.elastic.co/u/linkerc)\ **Replies:** 0\ **Last updated:** [August 24, 2022, 10:32pm UTC](https://discuss.elastic.co/t/api-updaterequest-question/312843 "2022-08-24T22:32:36Z")
bulkRequest.add(new UpdateRequest(index, id).doc(asMap).upsert(asMap)); Can somebody please point me to an explanation on when to use ".doc()" and ".upsert()"? My above example works, but java memory analyzer shows the… --- ## [How to custom sort](https://discuss.elastic.co/t/how-to-custom-sort/312838)
**Author:** [@Guilherme\_Mello](https://discuss.elastic.co/u/Guilherme_Mello)\ **Replies:** 1\ **Last updated:** [August 24, 2022, 10:30pm UTC](https://discuss.elastic.co/t/how-to-custom-sort/312838 "2022-08-24T22:30:25Z")
Hello. I'm building a retail search. This search needs to give priority to products from certain stores (they pay to be more relevant). The roducts from these stores need to appear more frequently in the search. I'm … --- ## [Failed to stop elasticsearch.service: Unit elasticsearch.service not loaded](https://discuss.elastic.co/t/failed-to-stop-elasticsearch-service-unit-elasticsearch-service-not-loaded/312824)
**Author:** [@Boodle](https://discuss.elastic.co/u/Boodle)\ **Replies:** 2\ **Last updated:** [August 24, 2022, 10:21pm UTC](https://discuss.elastic.co/t/failed-to-stop-elasticsearch-service-unit-elasticsearch-service-not-loaded/312824 "2022-08-24T22:21:35Z")
Following this to upgrade Elasticsearch. When shutting down the single node with 'sudo service elasticsearch stop' gets error: "Failed to stop elasticsearch.service: Unit elasticsearch.service not loaded." Upgrading El… --- ## [Is it possible to let elasticsearch choose the index to ingest](https://discuss.elastic.co/t/is-it-possible-to-let-elasticsearch-choose-the-index-to-ingest/312343)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 3\ **Last updated:** [August 24, 2022, 9:20pm UTC](https://discuss.elastic.co/t/is-it-possible-to-let-elasticsearch-choose-the-index-to-ingest/312343 "2022-08-24T21:20:04Z")
Hi, we want to distribute / split out data to different indexes, because based on the type of the data we will have different retention times. In my last project I used logstash to parse and enrich log events and calcu… --- ## [How to call variables?](https://discuss.elastic.co/t/how-to-call-variables/312809)
**Author:** [@ToonKnight](https://discuss.elastic.co/u/ToonKnight)\ **Replies:** 0\ **Last updated:** [August 24, 2022, 12:22pm UTC](https://discuss.elastic.co/t/how-to-call-variables/312809 "2022-08-24T12:22:39Z")
Hey all, I made a script that would create a jew index, as soon as the currently running one reaches a space limit. I can't share the script but basically, I have two variables at the start that is could newindex and r… --- ## [Query Option Similar to Vega Flatten Transform](https://discuss.elastic.co/t/query-option-similar-to-vega-flatten-transform/312835)
**Author:** [@LisaJ](https://discuss.elastic.co/u/LisaJ)\ **Replies:** 0\ **Last updated:** [August 24, 2022, 6:08pm UTC](https://discuss.elastic.co/t/query-option-similar-to-vega-flatten-transform/312835 "2022-08-24T18:08:18Z")
When using the Vega visualization engine to graph data, I use the flatten transform to turn a single document that contains an array of length n into n documents with a single element from the array. From: To: Thi… --- ## [Configure TLS production environment with own CA](https://discuss.elastic.co/t/configure-tls-production-environment-with-own-ca/310776)
**Author:** [@luca.derugeriis](https://discuss.elastic.co/u/luca.derugeriis)\ **Replies:** 3\ **Last updated:** [August 24, 2022, 3:50pm UTC](https://discuss.elastic.co/t/configure-tls-production-environment-with-own-ca/310776 "2022-08-24T15:50:48Z")
Hi everyone, I'm having a bit of an hard time configuring TLS in a production environment. I already have TLS running on a test environment (with autoconfiguration, using .deb package). At the end of the procedure it pr… --- ## [Elasticsearch nested phrase search within a certain distance](https://discuss.elastic.co/t/elasticsearch-nested-phrase-search-within-a-certain-distance/312757)
**Author:** [@Sumit\_Jha1](https://discuss.elastic.co/u/Sumit_Jha1)\ **Replies:** 0\ **Last updated:** [August 24, 2022, 4:44am UTC](https://discuss.elastic.co/t/elasticsearch-nested-phrase-search-within-a-certain-distance/312757 "2022-08-24T04:44:45Z")
Sample ES Document { // other properties "transcript" : \[ { "id" : 0, "user\_type" : "A", "phrase" : "hi good afternoon" }, { "id" : 1, "user\_type" : "B"… --- ## [Data split into multiple Documents](https://discuss.elastic.co/t/data-split-into-multiple-documents/312808)
**Author:** [@Maruthappan\_Muthu](https://discuss.elastic.co/u/Maruthappan_Muthu)\ **Replies:** 1\ **Last updated:** [August 24, 2022, 12:24pm UTC](https://discuss.elastic.co/t/data-split-into-multiple-documents/312808 "2022-08-24T12:24:13Z")
I am writing the Prometheus collected metrics into Elasticsearch for every 2 minutes. I see multiple documents for a single data collection and each documents has different fields. Is this the way Elasticsearch stores th… --- ## [Search as you type with 100k rows, help scoring](https://discuss.elastic.co/t/search-as-you-type-with-100k-rows-help-scoring/312412)
**Author:** [@samtwilliams](https://discuss.elastic.co/u/samtwilliams)\ **Replies:** 2\ **Last updated:** [August 24, 2022, 11:59am UTC](https://discuss.elastic.co/t/search-as-you-type-with-100k-rows-help-scoring/312412 "2022-08-24T11:59:47Z")
Hi All, I have an index that contains drug names (100k of them),I'm using a react front end to search as you type which is working quite well except the scoring coming back is not what I expect despite endless fiddling. … --- ## [Add a new field in all the existing documents in ES](https://discuss.elastic.co/t/add-a-new-field-in-all-the-existing-documents-in-es/312739)
**Author:** [@ImranArif](https://discuss.elastic.co/u/ImranArif)\ **Replies:** 2\ **Last updated:** [August 24, 2022, 10:22am UTC](https://discuss.elastic.co/t/add-a-new-field-in-all-the-existing-documents-in-es/312739 "2022-08-24T10:22:25Z")
Hi, I want to update existing documents by adding a new field, the value of which will be looked up from a csv file hosted on an external path (ftp, probably). I read about the translate filter that can be used to look … --- ## [Restore are 20x slower after migrating from 5.x to 7.x](https://discuss.elastic.co/t/restore-are-20x-slower-after-migrating-from-5-x-to-7-x/312281)
**Author:** [@bqk](https://discuss.elastic.co/u/bqk)\ **Replies:** 2\ **Last updated:** [August 24, 2022, 10:16am UTC](https://discuss.elastic.co/t/restore-are-20x-slower-after-migrating-from-5-x-to-7-x/312281 "2022-08-24T10:16:17Z")
Hi, Found some related topics but no actual answer. We run daily snapshots in production, and every morning we wipe our test cluster and do a full restore. This way we test our backups and our staff has playground envi… --- ## [The \_id field deprecated warning in the elasticsearch\_dsl python package](https://discuss.elastic.co/t/the-id-field-deprecated-warning-in-the-elasticsearch-dsl-python-package/312785)
**Author:** [@Ivo\_Tavares](https://discuss.elastic.co/u/Ivo_Tavares)\ **Replies:** 0\ **Last updated:** [August 24, 2022, 9:41am UTC](https://discuss.elastic.co/t/the-id-field-deprecated-warning-in-the-elasticsearch-dsl-python-package/312785 "2022-08-24T09:41:36Z")
I'm using a python client. There's a Document of the document class, with a data field called id. I've added a document to the default index, without defining a value for that id field. I was expecting ES to auto-popu… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=384) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=386)