# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1077 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1078 --- ## [Tracking searched JSON documents](https://discuss.elastic.co/t/tracking-searched-json-documents/295554)
**Author:** [@Mohan\_Radhakrishnan](https://discuss.elastic.co/u/Mohan_Radhakrishnan)\ **Replies:** 0\ **Last updated:** [January 27, 2022, 7:51am UTC](https://discuss.elastic.co/t/tracking-searched-json-documents/295554 "2022-01-27T07:51:20Z")
Hello, We are evaluating based on our requirements and not sure how to handle these. We have decided to sink our events as JSON using Kafka Elasticsearch Sink Connector. The source is the DB. Now our primary document s… --- ## [How to convert timestamp in milliseconds i,e long using elastic-sql](https://discuss.elastic.co/t/how-to-convert-timestamp-in-milliseconds-i-e-long-using-elastic-sql/295543)
**Author:** [@David\_Samm](https://discuss.elastic.co/u/David_Samm)\ **Replies:** 1\ **Last updated:** [January 27, 2022, 7:39am UTC](https://discuss.elastic.co/t/how-to-convert-timestamp-in-milliseconds-i-e-long-using-elastic-sql/295543 "2022-01-27T07:39:33Z")
Hi, I have timestamp fields where I want to convert this timestamp to long i,e milliseconds GET /\_sql { "query":"select "2017-11-13T13:51:00.000Z" as date from employee " } here I want the timestamp "201… --- ## [Set kibanaConfig via Helm in Terraform](https://discuss.elastic.co/t/set-kibanaconfig-via-helm-in-terraform/295547)
**Author:** [@florian.besser](https://discuss.elastic.co/u/florian.besser)\ **Replies:** 1\ **Last updated:** [January 27, 2022, 7:23am UTC](https://discuss.elastic.co/t/set-kibanaconfig-via-helm-in-terraform/295547 "2022-01-27T07:23:42Z")
I'm using terraform to deploy Kibana using helm: resource "helm\_release" "kibana" { name = "kibana" namespace = "elastic" create\_namespace = true repository = "https://helm.elastic.co" char… --- ## [Elasticsearch memory issue?](https://discuss.elastic.co/t/elasticsearch-memory-issue/295449)
**Author:** [@catadetest](https://discuss.elastic.co/u/catadetest)\ **Replies:** 2\ **Last updated:** [January 27, 2022, 7:13am UTC](https://discuss.elastic.co/t/elasticsearch-memory-issue/295449 "2022-01-27T07:13:41Z")
Hi everyone, As my manager told me we should expect more traffic in the following days, I would like to be prepared with the Elasticsearch servers performance. So I did the following investigation: Verified my load /… --- ## [Enquiry on performance issue using Virtual Data Optimizer (VDO) to save storage space](https://discuss.elastic.co/t/enquiry-on-performance-issue-using-virtual-data-optimizer-vdo-to-save-storage-space/295550)
**Author:** [@anon17386273](https://discuss.elastic.co/u/anon17386273)\ **Replies:** 0\ **Last updated:** [January 27, 2022, 7:06am UTC](https://discuss.elastic.co/t/enquiry-on-performance-issue-using-virtual-data-optimizer-vdo-to-save-storage-space/295550 "2022-01-27T07:06:02Z")
Hi I came across this RedHat Virtual Data Optimizer (VDO) storage software that can be used as an additional storage layer on top of my physical volume to perform compression & deduplication. This can significantly save… --- ## [How to convert timestamp in milliseconds i,e long using elastic-sql](https://discuss.elastic.co/t/how-to-convert-timestamp-in-milliseconds-i-e-long-using-elastic-sql/295347)
**Author:** [@David\_Samm](https://discuss.elastic.co/u/David_Samm)\ **Replies:** 2\ **Last updated:** [January 27, 2022, 5:50am UTC](https://discuss.elastic.co/t/how-to-convert-timestamp-in-milliseconds-i-e-long-using-elastic-sql/295347 "2022-01-27T05:50:30Z")
Hi, I have timestamp fields where I want to convert this timestamp to long i,e milliseconds GET /\_sql { "query":"select "2017-11-13T13:51:00.000Z" as date from employee " } here I want the timestamp "2017-… --- ## [Search in two indexes](https://discuss.elastic.co/t/search-in-two-indexes/295506)
**Author:** [@TomTom](https://discuss.elastic.co/u/TomTom)\ **Replies:** 2\ **Last updated:** [January 27, 2022, 4:50am UTC](https://discuss.elastic.co/t/search-in-two-indexes/295506 "2022-01-27T04:50:00Z")
I have two indices. Customer: List of all customers, their internal identifier (id) and their name (name); Order: List of all purchases made, with purchase date (order\_date), customer id (customer.id) and name (custo… --- ## [Split a doc to multiple docs](https://discuss.elastic.co/t/split-a-doc-to-multiple-docs/295531)
**Author:** [@cris](https://discuss.elastic.co/u/cris)\ **Replies:** 6\ **Last updated:** [January 27, 2022, 4:41am UTC](https://discuss.elastic.co/t/split-a-doc-to-multiple-docs/295531 "2022-01-27T04:41:44Z")
Hello friends. I am trying to reindex some data to another index but with a little differences. In the first index I have this kind of hit: { "ENVA": { "Login": { "status": "passed" } }, "ENVB": { … --- ## [Accessing specific datafields for Prometheus Exporter](https://discuss.elastic.co/t/accessing-specific-datafields-for-prometheus-exporter/295424)
**Author:** [@purple\_key](https://discuss.elastic.co/u/purple_key)\ **Replies:** 2\ **Last updated:** [January 27, 2022, 4:35am UTC](https://discuss.elastic.co/t/accessing-specific-datafields-for-prometheus-exporter/295424 "2022-01-27T04:35:15Z")
Using Postgres\_Exporter on my PSQL servers and piping them to the Prometheus MetricBeat with a sample config of # Module: prometheus # Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.6/metricbeat-module-prometh… --- ## [Logstash 7.16.3 Error： (LoadError) load error: rubygems/text -- java.lang.NoClassDefFoundError: org/jruby/util/Sprintf](https://discuss.elastic.co/t/logstash-7-16-3-error-loaderror-load-error-rubygems-text-java-lang-noclassdeffounderror-org-jruby-util-sprintf/295536)
**Author:** [@seth\_qiang](https://discuss.elastic.co/u/seth_qiang)\ **Replies:** 0\ **Last updated:** [January 27, 2022, 4:21am UTC](https://discuss.elastic.co/t/logstash-7-16-3-error-loaderror-load-error-rubygems-text-java-lang-noclassdeffounderror-org-jruby-util-sprintf/295536 "2022-01-27T04:21:58Z")
hello, everyone logstash 7.16.3 on docker-compose version: '3.4' services: new-elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.16.3 container\_name: new-elasticsearch environment:… --- ## [Filebeat console repeat logs for filestream input](https://discuss.elastic.co/t/filebeat-console-repeat-logs-for-filestream-input/295435)
**Author:** [@zhiweiv](https://discuss.elastic.co/u/zhiweiv)\ **Replies:** 2\ **Last updated:** [January 27, 2022, 12:26am UTC](https://discuss.elastic.co/t/filebeat-console-repeat-logs-for-filestream-input/295435 "2022-01-27T00:26:17Z")
After switch input type from log to filestream, there are repeat logs every 10s in filebeat console like following: 2022-01-26T09:06:45.287Z \[file\_watcher\] filestream/fswatch.go:137 Start next scan 2022-01-26T09:06:55.2… --- ## [No Sort Options on Kibana Lens Table](https://discuss.elastic.co/t/no-sort-options-on-kibana-lens-table/295436)
**Author:** [@Johanes\_Mistrialdo](https://discuss.elastic.co/u/Johanes_Mistrialdo)\ **Replies:** 3\ **Last updated:** [January 27, 2022, 3:22am UTC](https://discuss.elastic.co/t/no-sort-options-on-kibana-lens-table/295436 "2022-01-27T03:22:04Z")
Hi All, I am new to kibana and doesn't know why there is no options to sort by columns... discussion on Sorting in Kibana Data Table Visualization shows that i can sort the table by columns, but i cannot do the same… --- ## [Brew services starts elasticsearch as root](https://discuss.elastic.co/t/brew-services-starts-elasticsearch-as-root/295409)
**Author:** [@Stephen\_Moore](https://discuss.elastic.co/u/Stephen_Moore)\ **Replies:** 0\ **Last updated:** [January 26, 2022, 12:51am UTC](https://discuss.elastic.co/t/brew-services-starts-elasticsearch-as-root/295409 "2022-01-26T00:51:48Z")
Hello, I've installed Elasticsearch with brew tap elastic/tap brew install elastic/tap/elasticsearch-full And it seems the brew service for it wants to run as root and so Elasticsearch fails to start ╰─ brew services… --- ## [Failed to run "elasticsearch-setup-passwords auto" on ES 7.16.2](https://discuss.elastic.co/t/failed-to-run-elasticsearch-setup-passwords-auto-on-es-7-16-2/295316)
**Author:** [@asagiv](https://discuss.elastic.co/u/asagiv)\ **Replies:** 1\ **Last updated:** [January 27, 2022, 2:45am UTC](https://discuss.elastic.co/t/failed-to-run-elasticsearch-setup-passwords-auto-on-es-7-16-2/295316 "2022-01-27T02:45:40Z")
Hi I am trying to generate the users/password for the built-in users , and fail with the below errors. can someone help to understand the issue ? Regards Avi Failed to determine the health of the cluster running at h… --- ## [ES Best Practices for collecting public distributed logs](https://discuss.elastic.co/t/es-best-practices-for-collecting-public-distributed-logs/295336)
**Author:** [@Alberto\_Manzano](https://discuss.elastic.co/u/Alberto_Manzano)\ **Replies:** 1\ **Last updated:** [January 27, 2022, 2:43am UTC](https://discuss.elastic.co/t/es-best-practices-for-collecting-public-distributed-logs/295336 "2022-01-27T02:43:43Z")
Hello everyone, I've been using ES Stack for quite a while but I'm still struggling to figure out what's the best architecture to ingest logs for a given use case. I've developed an application customers have to instal… --- ## [Why am I getting warning about "Number of open shards exceeds cluster soft limit"?](https://discuss.elastic.co/t/why-am-i-getting-warning-about-number-of-open-shards-exceeds-cluster-soft-limit/295356)
**Author:** [@willdennis](https://discuss.elastic.co/u/willdennis)\ **Replies:** 8\ **Last updated:** [January 27, 2022, 2:14am UTC](https://discuss.elastic.co/t/why-am-i-getting-warning-about-number-of-open-shards-exceeds-cluster-soft-limit/295356 "2022-01-27T02:14:53Z")
I have a single-node ES 6.8 cluster with ~5500 shards. I have set the cluster.max\_shards\_per\_node to '10000' in /etc/elasticsearch/elasticsearch.yml as per this article: But even after restarting ES, I get the "Number … --- ## [Could not create the CloudFormation stack request](https://discuss.elastic.co/t/could-not-create-the-cloudformation-stack-request/295437)
**Author:** [@Jason\_Zhang2](https://discuss.elastic.co/u/Jason_Zhang2)\ **Replies:** 4\ **Last updated:** [January 27, 2022, 1:12am UTC](https://discuss.elastic.co/t/could-not-create-the-cloudformation-stack-request/295437 "2022-01-27T01:12:01Z")
Hi, There We're trying to use functionbeat to add cloudwatch log into Elasticsearch. And it always gave this error 'Could not create the CloudFormation stack request'. This page mentioned that it's caused by region se… --- ## [ERROR COMPILING LOGSTASH CODEBASE TO GENERATE THE .JAR FILE CONTAINING THE JAVA PLUGIN API](https://discuss.elastic.co/t/error-compiling-logstash-codebase-to-generate-the-jar-file-containing-the-java-plugin-api/295528)
**Author:** [@uday22](https://discuss.elastic.co/u/uday22)\ **Replies:** 0\ **Last updated:** [January 27, 2022, 12:34am UTC](https://discuss.elastic.co/t/error-compiling-logstash-codebase-to-generate-the-jar-file-containing-the-java-plugin-api/295528 "2022-01-27T00:34:13Z")
Error compiling logstash codebase to generate the java plugin API to develop logstash java plugin. running gradlew.bat assemble from root directory of logstash codebase getting below error. Exception in thread "main" … --- ## [Reducing the number of replicas will release disk space an each node of the cluster?](https://discuss.elastic.co/t/reducing-the-number-of-replicas-will-release-disk-space-an-each-node-of-the-cluster/295526)
**Author:** [@Guillermo\_Garcia1](https://discuss.elastic.co/u/Guillermo_Garcia1)\ **Replies:** 2\ **Last updated:** [January 27, 2022, 12:01am UTC](https://discuss.elastic.co/t/reducing-the-number-of-replicas-will-release-disk-space-an-each-node-of-the-cluster/295526 "2022-01-27T00:01:11Z")
Hello there, We have a problem with shard allocation due to disk space usage in several indices (yellow status). We have 2 replicas for each primary shard. We want to update the index settings to 1 replica instead of 2… --- ## [Logstash not connecting to Elasticsearch - using Docker-Compose](https://discuss.elastic.co/t/logstash-not-connecting-to-elasticsearch-using-docker-compose/295512)
**Author:** [@userR](https://discuss.elastic.co/u/userR)\ **Replies:** 1\ **Last updated:** [January 26, 2022, 11:31pm UTC](https://discuss.elastic.co/t/logstash-not-connecting-to-elasticsearch-using-docker-compose/295512 "2022-01-26T23:31:42Z")
Both containers are running and are attempting to communicate with each other. I see Filebeat connecting to Logstash and send information, however I don't see any data being read into Elasticsearch. Here are the logs th… --- ## [After upgrading ES version, new node cannot join to the cluster](https://discuss.elastic.co/t/after-upgrading-es-version-new-node-cannot-join-to-the-cluster/295520)
**Author:** [@judy\_gao](https://discuss.elastic.co/u/judy_gao)\ **Replies:** 0\ **Last updated:** [January 26, 2022, 11:03pm UTC](https://discuss.elastic.co/t/after-upgrading-es-version-new-node-cannot-join-to-the-cluster/295520 "2022-01-26T23:03:45Z")
After upgrading one node to ES 7.16.3, the new node cannot join the cluster, error from log : \[2022-01-26T22:33:34,365\]\[DEBUG\]\[o.e.d.PeerFinder \] \[ip-10-68-41-201.ec2.internal\] address \[\[::1\]:9304\], node \[null\],… --- ## [Insure rebalance data between data node](https://discuss.elastic.co/t/insure-rebalance-data-between-data-node/295503)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 2\ **Last updated:** [January 26, 2022, 10:05pm UTC](https://discuss.elastic.co/t/insure-rebalance-data-between-data-node/295503 "2022-01-26T22:05:11Z")
Hi I'm wondering why it was occurred, from the beginning I used the schema of rebalance date through shards (7shards per index). I have 7 data node and one of additional as coordinator. But one of them is going to reach… --- ## [Discover on dashboard changes Add filter source of filter](https://discuss.elastic.co/t/discover-on-dashboard-changes-add-filter-source-of-filter/295380)
**Author:** [@Michael\_Ubry](https://discuss.elastic.co/u/Michael_Ubry)\ **Replies:** 4\ **Last updated:** [January 26, 2022, 7:44pm UTC](https://discuss.elastic.co/t/discover-on-dashboard-changes-add-filter-source-of-filter/295380 "2022-01-26T19:44:08Z")
Hi, I have dashboard which has multiple visualizations from one index "index1". Add filter offers fields from this index. When I add discover list from "index2" to dashboard, it suddenly changes the options in Add filte… --- ## [JDBC input, kafka output, syntax error, unexpected end-of-file Java:: after pipeline starts](https://discuss.elastic.co/t/jdbc-input-kafka-output-syntax-error-unexpected-end-of-file-java-after-pipeline-starts/294452)
**Author:** [@rschmerb](https://discuss.elastic.co/u/rschmerb)\ **Replies:** 6\ **Last updated:** [January 26, 2022, 7:09pm UTC](https://discuss.elastic.co/t/jdbc-input-kafka-output-syntax-error-unexpected-end-of-file-java-after-pipeline-starts/294452 "2022-01-26T19:09:32Z")
We are seeing the following error messages after starting our pipeline: \[2022-01-13T19:48:06,009\]\[FATAL\]\[org.logstash.Logstash \] Logstash stopped processing because of an error: (SyntaxError) (eval):1: syntax error, un… --- ## [Visualization of the most frequent words](https://discuss.elastic.co/t/visualization-of-the-most-frequent-words/295494)
**Author:** [@meyer1](https://discuss.elastic.co/u/meyer1)\ **Replies:** 3\ **Last updated:** [January 26, 2022, 6:54pm UTC](https://discuss.elastic.co/t/visualization-of-the-most-frequent-words/295494 "2022-01-26T18:54:13Z")
Hello ! i am new to kibana and elastic My need is to display the most used words in a text type field on several documents. Is it possible to do this with kibana and elastic? If so, how is this possible? With scripted … --- ## [Logstash http poller](https://discuss.elastic.co/t/logstash-http-poller/295441)
**Author:** [@Ingram\_Gultom](https://discuss.elastic.co/u/Ingram_Gultom)\ **Replies:** 3\ **Last updated:** [January 26, 2022, 6:03pm UTC](https://discuss.elastic.co/t/logstash-http-poller/295441 "2022-01-26T18:03:04Z")
Hi, I am using logstash 7.13.4 and I want to use http poller for this url http://10.10.10.10:8080/dbonequerydata/?username=user/password=password/encrypted=false/conversion=true/DT=csv -H "Content-Type: application/xml… --- ## [Getting search\_phase\_execution\_exception when making a SearchRequest with sort options](https://discuss.elastic.co/t/getting-search-phase-execution-exception-when-making-a-searchrequest-with-sort-options/295402)
**Author:** [@ridhwaans](https://discuss.elastic.co/u/ridhwaans)\ **Replies:** 0\ **Last updated:** [January 25, 2022, 9:23pm UTC](https://discuss.elastic.co/t/getting-search-phase-execution-exception-when-making-a-searchrequest-with-sort-options/295402 "2022-01-25T21:23:28Z")
Used the Java RestHighLevelClient 7.4.2 to make an org.elasticsearch.action.search.SearchRequest in conjunction with org.elasticsearch.search.sort.FieldSortBuilder. It returned the following ElasticsearchStatusExceptio… --- ## [Index delete error with ILM](https://discuss.elastic.co/t/index-delete-error-with-ilm/295462)
**Author:** [@ASRLO](https://discuss.elastic.co/u/ASRLO)\ **Replies:** 1\ **Last updated:** [January 26, 2022, 6:54pm UTC](https://discuss.elastic.co/t/index-delete-error-with-ilm/295462 "2022-01-26T18:54:04Z")
Hello everyone, I have a problem with indexes. Indeed, it no longer deletes itself automatically as expected with my Index Policy management. When I look I get this error message "Index lifecycle error : security\_ex… --- ## [Replicate HOT node](https://discuss.elastic.co/t/replicate-hot-node/295493)
**Author:** [@ima](https://discuss.elastic.co/u/ima)\ **Replies:** 1\ **Last updated:** [January 26, 2022, 5:47pm UTC](https://discuss.elastic.co/t/replicate-hot-node/295493 "2022-01-26T17:47:56Z")
Hello everyone, Is there a way to replicate a data hot node to another node to maitain high availability (HA). Thank you --- ## [Split text field in array with multiple value](https://discuss.elastic.co/t/split-text-field-in-array-with-multiple-value/295492)
**Author:** [@meyer1](https://discuss.elastic.co/u/meyer1)\ **Replies:** 1\ **Last updated:** [January 26, 2022, 5:45pm UTC](https://discuss.elastic.co/t/split-text-field-in-array-with-multiple-value/295492 "2022-01-26T17:45:51Z")
Hello ! I am new to kibana and Elasticsearch. My need : my document : { "text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone." } and i want to create scripted field and i want this result { "splitText… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1076) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1078)