# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1308 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1309 --- ## [Elasticsearch is still indexing but can not see in kiabana](https://discuss.elastic.co/t/elasticsearch-is-still-indexing-but-can-not-see-in-kiabana/279567)
**Author:** [@anhthang88](https://discuss.elastic.co/u/anhthang88)\ **Replies:** 2\ **Last updated:** [July 26, 2021, 2:45am UTC](https://discuss.elastic.co/t/elasticsearch-is-still-indexing-but-can-not-see-in-kiabana/279567 "2021-07-26T02:45:49Z")
Hi everyone, I have a problem with ES 6.8 I try to index with index rate ~ 1700. But I can not see realtime log in kibana. After the file was truncate, I can see all the logs. every 2 hours I can see the log. I … --- ## [Metricbeat SQL Module : Unable to extract data from a custom query](https://discuss.elastic.co/t/metricbeat-sql-module-unable-to-extract-data-from-a-custom-query/279360)
**Author:** [@VinodKumarPotta](https://discuss.elastic.co/u/VinodKumarPotta)\ **Replies:** 6\ **Last updated:** [July 26, 2021, 2:35am UTC](https://discuss.elastic.co/t/metricbeat-sql-module-unable-to-extract-data-from-a-custom-query/279360 "2021-07-26T02:35:45Z")
Hi Team, Seek your expert advise on this. I have recently enabled Metricbeat sql module to extract data from a custom query. I have followed the instructions given in sql.yml file (SQL module | Metricbeat Reference \[7… --- ## [How to prevent kubernetes bring down more than 1 data node during the deployment](https://discuss.elastic.co/t/how-to-prevent-kubernetes-bring-down-more-than-1-data-node-during-the-deployment/279104)
**Author:** [@junhuangli](https://discuss.elastic.co/u/junhuangli)\ **Replies:** 6\ **Last updated:** [July 26, 2021, 2:01am UTC](https://discuss.elastic.co/t/how-to-prevent-kubernetes-bring-down-more-than-1-data-node-during-the-deployment/279104 "2021-07-26T02:01:58Z")
Not sure if this is a question for kubernetes or for elasticsearch. We are hosting elasticsearch on kubernetes. And during the deployment, the kubernetes brings down the pods 1 by 1. But it doesn't wait for the data nod… --- ## [Kibana Error After Connecting Node to a Cluster](https://discuss.elastic.co/t/kibana-error-after-connecting-node-to-a-cluster/279202)
**Author:** [@IWannaDance](https://discuss.elastic.co/u/IWannaDance)\ **Replies:** 1\ **Last updated:** [July 26, 2021, 12:29am UTC](https://discuss.elastic.co/t/kibana-error-after-connecting-node-to-a-cluster/279202 "2021-07-26T00:29:50Z")
I originally had a single-node cluster set on machine 1, and everything was working fine. Then I tried to add another node to the cluster on machine 2 in the local area network. I had started elasticsearch on machine 2 … --- ## [How to find disk space usage of logs returned by query](https://discuss.elastic.co/t/how-to-find-disk-space-usage-of-logs-returned-by-query/279220)
**Author:** [@jteare](https://discuss.elastic.co/u/jteare)\ **Replies:** 1\ **Last updated:** [July 26, 2021, 12:29am UTC](https://discuss.elastic.co/t/how-to-find-disk-space-usage-of-logs-returned-by-query/279220 "2021-07-26T00:29:05Z")
I am attempting to determine storage requirements for logging various systems. It's convenient when all logs for the system in question are going into one index e.g. all firewall logs going into a firewall index, I can j… --- ## [Index lifecycle management not working. automatic reverting of policy](https://discuss.elastic.co/t/index-lifecycle-management-not-working-automatic-reverting-of-policy/279509)
**Author:** [@mw77](https://discuss.elastic.co/u/mw77)\ **Replies:** 2\ **Last updated:** [July 26, 2021, 12:22am UTC](https://discuss.elastic.co/t/index-lifecycle-management-not-working-automatic-reverting-of-policy/279509 "2021-07-26T00:22:10Z")
Hi Guys, I am reposting this. I have create a ILM policy which has around 90 indices attached to it. from the UI I have created a policy and below is the request when I click on show request. I am enabling the delete… --- ## [Snapshot policy which is only triggered by ILM and not on its own schedule](https://discuss.elastic.co/t/snapshot-policy-which-is-only-triggered-by-ilm-and-not-on-its-own-schedule/277758)
**Author:** [@pk.241011](https://discuss.elastic.co/u/pk.241011)\ **Replies:** 6\ **Last updated:** [July 25, 2021, 11:43pm UTC](https://discuss.elastic.co/t/snapshot-policy-which-is-only-triggered-by-ilm-and-not-on-its-own-schedule/277758 "2021-07-25T23:43:53Z")
Hi, Is it possible to have a snapshot policy which does not have a schedule? I wanted the ILM to create a snapshot by invoking the snapshot policy just before the index is deleted at the end of cold phase. I want to us… --- ## [Monitoring cluster](https://discuss.elastic.co/t/monitoring-cluster/279331)
**Author:** [@jungsoo](https://discuss.elastic.co/u/jungsoo)\ **Replies:** 1\ **Last updated:** [July 25, 2021, 11:21pm UTC](https://discuss.elastic.co/t/monitoring-cluster/279331 "2021-07-25T23:21:23Z")
Hello, My company is using monitoring cluster and want to upgrade production cluster from 7.11 to 7.13 where monitoring cluster is 7.11. The document says " \* Ideally the monitoring cluster and the production cluster … --- ## [Basic License XPack Security for ES 5.2.1](https://discuss.elastic.co/t/basic-license-xpack-security-for-es-5-2-1/279396)
**Author:** [@spr](https://discuss.elastic.co/u/spr)\ **Replies:** 2\ **Last updated:** [July 25, 2021, 11:20pm UTC](https://discuss.elastic.co/t/basic-license-xpack-security-for-es-5-2-1/279396 "2021-07-25T23:20:12Z")
Hi, I'm trying to use the following basic license which is supposed to have xpack security available. However when I deploy the same. I get the following error {"error":{"root\_cause":\[{"type":"security\_exception","reas… --- ## [Fleet not working](https://discuss.elastic.co/t/fleet-not-working/279411)
**Author:** [@Prakul\_Tomar](https://discuss.elastic.co/u/Prakul_Tomar)\ **Replies:** 1\ **Last updated:** [July 25, 2021, 11:19pm UTC](https://discuss.elastic.co/t/fleet-not-working/279411 "2021-07-25T23:19:31Z")
Hello Elastic Community, Fleet module is not working. Requesting the community to share the solution, ho to resolve this issue. --- ## [Problems adding the prerequisites for cross cluster replication](https://discuss.elastic.co/t/problems-adding-the-prerequisites-for-cross-cluster-replication/279430)
**Author:** [@Francisco\_Yanez](https://discuss.elastic.co/u/Francisco_Yanez)\ **Replies:** 1\ **Last updated:** [July 25, 2021, 11:18pm UTC](https://discuss.elastic.co/t/problems-adding-the-prerequisites-for-cross-cluster-replication/279430 "2021-07-25T23:18:02Z")
I am trying to add the prerequisites as pointed on the Tutorial: Set up cross-cluster replication | Elasticsearch Guide \[7.13\] | Elastic so I added to my k8s elastic yaml in the local cluster: ccr\_user: cluster: - man… --- ## [Lens pie chart displays exponential values instead of 0 with filters function](https://discuss.elastic.co/t/lens-pie-chart-displays-exponential-values-instead-of-0-with-filters-function/279480)
**Author:** [@BitBucketUser\_user](https://discuss.elastic.co/u/BitBucketUser_user)\ **Replies:** 2\ **Last updated:** [July 25, 2021, 6:59pm UTC](https://discuss.elastic.co/t/lens-pie-chart-displays-exponential-values-instead-of-0-with-filters-function/279480 "2021-07-25T18:59:57Z")
Hello, I'm using Kibana 7.12.0 When I use Lens Pie chart with "Filters" function, and for a given timeframe when there is no data matching my filter, I expect 0 to be shown as the "count" metrics value. But I see data… --- ## [Are the logs lost if elastic search down](https://discuss.elastic.co/t/are-the-logs-lost-if-elastic-search-down/279561)
**Author:** [@frank\_rib](https://discuss.elastic.co/u/frank_rib)\ **Replies:** 3\ **Last updated:** [July 25, 2021, 6:04pm UTC](https://discuss.elastic.co/t/are-the-logs-lost-if-elastic-search-down/279561 "2021-07-25T18:04:39Z")
Hello every body, I have a question about the fate of the logs when the elasticsearch is down. if the logs are lost during the inactivity of elasticsearch, how can we get around this problem? Best regards, --- ## [Fscrawler image file text extraction](https://discuss.elastic.co/t/fscrawler-image-file-text-extraction/279538)
**Author:** [@Jenya87](https://discuss.elastic.co/u/Jenya87)\ **Replies:** 6\ **Last updated:** [July 25, 2021, 5:50pm UTC](https://discuss.elastic.co/t/fscrawler-image-file-text-extraction/279538 "2021-07-25T17:50:54Z")
Hello! I use FScrawler 2.7 on Windows 7 to ingest files content into Elastic 7.13 node on centos 7. It works really greate for .pdf and .docx extensions, but not for imge files such as .jpg or .png and .txt files. My… --- ## [Why .monitoring-\* indices has large size of data](https://discuss.elastic.co/t/why-monitoring-indices-has-large-size-of-data/278027)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 3\ **Last updated:** [July 25, 2021, 3:26pm UTC](https://discuss.elastic.co/t/why-monitoring-indices-has-large-size-of-data/278027 "2021-07-25T15:26:53Z")
I have 11 elasticsearch nodes 3 master node 6 data node and 2 coordinate node.We are running latest version of elasticsearch 7.13.2 we have installed metricbeat and configured in all elasticsearch node we are monitorin… --- ## [Logstash filter json warning](https://discuss.elastic.co/t/logstash-filter-json-warning/279352)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 6\ **Last updated:** [July 25, 2021, 3:16pm UTC](https://discuss.elastic.co/t/logstash-filter-json-warning/279352 "2021-07-25T15:16:56Z")
Using ELK 7.13.2 The logstash is consuming logs from kafka in json format earlier it was parsing the data but now it is not working \[2021-07-22T12:18:44,401\]\[WARN \]\[logstash.filters.json \]\[ise\]\[d51b308934c5e59604230… --- ## [Howto configure log4j2.properties to log to /dev/log](https://discuss.elastic.co/t/howto-configure-log4j2-properties-to-log-to-dev-log/279566)
**Author:** [@f1outsourcing](https://discuss.elastic.co/u/f1outsourcing)\ **Replies:** 0\ **Last updated:** [July 25, 2021, 2:01pm UTC](https://discuss.elastic.co/t/howto-configure-log4j2-properties-to-log-to-dev-log/279566 "2021-07-25T14:01:30Z")
I can get this to work, but I prefer to log to /dev/log appender.syslog.type = Syslog appender.syslog.name = syslog appender.syslog.host = xxxxxxxx.local appender.syslog.port = 514 appender.syslog.protocol = UDP ap… --- ## [How to run multiple nodes with esplugin / testsClusters?](https://discuss.elastic.co/t/how-to-run-multiple-nodes-with-esplugin-testsclusters/279562)
**Author:** [@Dor\_Levi](https://discuss.elastic.co/u/Dor_Levi)\ **Replies:** 0\ **Last updated:** [July 25, 2021, 1:15pm UTC](https://discuss.elastic.co/t/how-to-run-multiple-nodes-with-esplugin-testsclusters/279562 "2021-07-25T13:15:13Z")
Hi, I'm working on a plugin for Elasticsearch, and want to try and test it in a multi-node cluster. I'm building it with gradle and the esplugin/testcluster plugins. Everything works great when it's just one node, but… --- ## [I forgot to index a field](https://discuss.elastic.co/t/i-forgot-to-index-a-field/279552)
**Author:** [@Zongxian\_Feng](https://discuss.elastic.co/u/Zongxian_Feng)\ **Replies:** 1\ **Last updated:** [July 25, 2021, 9:27am UTC](https://discuss.elastic.co/t/i-forgot-to-index-a-field/279552 "2021-07-25T09:27:53Z")
Hi. I forgot to index a field called "n\_citation" which is in type "integer". When mapping the dataset, I decide to map the necessary fields (like "title" and "abstract") only to save my time. Now, it turns out that th… --- ## [Logstash mixing up log entries despite multip. pipelines](https://discuss.elastic.co/t/logstash-mixing-up-log-entries-despite-multip-pipelines/279557)
**Author:** [@pdanjou](https://discuss.elastic.co/u/pdanjou)\ **Replies:** 1\ **Last updated:** [July 25, 2021, 12:56pm UTC](https://discuss.elastic.co/t/logstash-mixing-up-log-entries-despite-multip-pipelines/279557 "2021-07-25T12:56:39Z")
So I have the following. 2 conf files with different ports, machine A sends to port1 and machine B sends to port2. I have pipelines.yml: pipeline.id: nginx path.config: "/etc/logstash/conf.d/logstash-nginx-es.conf" p… --- ## [Is there any character limit on term filter?](https://discuss.elastic.co/t/is-there-any-character-limit-on-term-filter/279544)
**Author:** [@dotel](https://discuss.elastic.co/u/dotel)\ **Replies:** 4\ **Last updated:** [July 25, 2021, 5:47am UTC](https://discuss.elastic.co/t/is-there-any-character-limit-on-term-filter/279544 "2021-07-25T05:47:29Z")
I'm trying to check if a certain attachment is previously indexed or not to avoid reindexing that document. But the problem is, for longer text fields, the term filter isn't working. I didn't find any mention of this lim… --- ## [Monitor servers/workstations in different locations over internet](https://discuss.elastic.co/t/monitor-servers-workstations-in-different-locations-over-internet/279524)
**Author:** [@nino](https://discuss.elastic.co/u/nino)\ **Replies:** 2\ **Last updated:** [July 24, 2021, 3:42pm UTC](https://discuss.elastic.co/t/monitor-servers-workstations-in-different-locations-over-internet/279524 "2021-07-24T15:42:50Z")
Hello, i would like to know some best practices about monitoring remote servers/workstations over internet. I mean, servers in different geographical locations other than ELK server: how beats reach ELK?. Ports how be… --- ## [Conditional Coloring on Pie Chart on Canvas](https://discuss.elastic.co/t/conditional-coloring-on-pie-chart-on-canvas/279358)
**Author:** [@Mayank\_Malhotra](https://discuss.elastic.co/u/Mayank_Malhotra)\ **Replies:** 3\ **Last updated:** [July 24, 2021, 3:29pm UTC](https://discuss.elastic.co/t/conditional-coloring-on-pie-chart-on-canvas/279358 "2021-07-24T15:29:19Z")
Hi, I have created a Pie Chart on canvas which consists a field severity which as two values one is ERROR and the other is WARNING. I want to show the part of WARNING in pie-chart as orange and ERROR as red. I Used belo… --- ## [Kibana "server is not ready yet" after enabling Basic Security on Elasticsearch cluster](https://discuss.elastic.co/t/kibana-server-is-not-ready-yet-after-enabling-basic-security-on-elasticsearch-cluster/279415)
**Author:** [@panagiss](https://discuss.elastic.co/u/panagiss)\ **Replies:** 3\ **Last updated:** [July 24, 2021, 3:28pm UTC](https://discuss.elastic.co/t/kibana-server-is-not-ready-yet-after-enabling-basic-security-on-elasticsearch-cluster/279415 "2021-07-24T15:28:59Z")
I enabled basic security doing step by step everything in the documentation. After i put that code xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification\_mode: certificate xpack.security… --- ## [What's the difference between Id's query and Term Query when finding documents by “\_id”?](https://discuss.elastic.co/t/whats-the-difference-between-ids-query-and-term-query-when-finding-documents-by-id/279523)
**Author:** [@Nishikant\_Tayade](https://discuss.elastic.co/u/Nishikant_Tayade)\ **Replies:** 2\ **Last updated:** [July 24, 2021, 7:11am UTC](https://discuss.elastic.co/t/whats-the-difference-between-ids-query-and-term-query-when-finding-documents-by-id/279523 "2021-07-24T07:11:39Z")
I want to get document by "\_id", I have 3 choices: GET document by "\_id" GET order/\_doc/001 Use Id's Query, GET order/\_search { "query": { "ids" : { "values" : \["001"\] } } } Though Id's query takes array of Id's but I … --- ## [How to remove %3A and such hash in text while parsing?](https://discuss.elastic.co/t/how-to-remove-3a-and-such-hash-in-text-while-parsing/279421)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 5\ **Last updated:** [July 24, 2021, 5:54am UTC](https://discuss.elastic.co/t/how-to-remove-3a-and-such-hash-in-text-while-parsing/279421 "2021-07-24T05:54:44Z")
Hello, I have a field in my logs: "name+Of+the%3A%C2%AE+field" which sometimes can be like this: "%D9+%81+%D8+%B1+%D8+%A7+%D8" How can I remove extra expressions in logstash filters? The desired output for the first … --- ## [Logstash ConcurrentModificationException: null](https://discuss.elastic.co/t/logstash-concurrentmodificationexception-null/278759)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 10\ **Last updated:** [July 24, 2021, 5:53am UTC](https://discuss.elastic.co/t/logstash-concurrentmodificationexception-null/278759 "2021-07-24T05:53:22Z")
Hi, has anyone encountered this issue? Logstash instances keep restarting after a while with this error: \[2021-07-14T14:35:23,972\]\[FATAL\]\[logstash.runner \] An unexpected error occurred! {:error=\>java.lang.Illeg… --- ## [Show custom geoip fields](https://discuss.elastic.co/t/show-custom-geoip-fields/279266)
**Author:** [@Velon](https://discuss.elastic.co/u/Velon)\ **Replies:** 4\ **Last updated:** [July 24, 2021, 2:19am UTC](https://discuss.elastic.co/t/show-custom-geoip-fields/279266 "2021-07-24T02:19:48Z")
I have add custom fields AcmeCorp.DeptName and AcmeCorp.DeptName whith maxmind\_mmdb, but I can't find it in the last result with logstash geoip plugins, so I'm asking some help ~ Thanks you~ --- ## [How to parse nginx error\_log in Kibana from file?](https://discuss.elastic.co/t/how-to-parse-nginx-error-log-in-kibana-from-file/279460)
**Author:** [@station72](https://discuss.elastic.co/u/station72)\ **Replies:** 1\ **Last updated:** [July 24, 2021, 12:53am UTC](https://discuss.elastic.co/t/how-to-parse-nginx-error-log-in-kibana-from-file/279460 "2021-07-24T00:53:39Z")
Hello! I have an error log file from nginx. I am trying to upload and parse my file with Data Visualizer in Kibana v 7.10.1 Example of my error log 2021/07/21 04:44:36 \[error\] 7#7: \*24864573 foo could not be resolved… --- ## [Position(or numeric ranking) of a item in an aggreagation](https://discuss.elastic.co/t/position-or-numeric-ranking-of-a-item-in-an-aggreagation/279479)
**Author:** [@Pablo\_Albertengo](https://discuss.elastic.co/u/Pablo_Albertengo)\ **Replies:** 1\ **Last updated:** [July 24, 2021, 12:24am UTC](https://discuss.elastic.co/t/position-or-numeric-ranking-of-a-item-in-an-aggreagation/279479 "2021-07-24T00:24:29Z")
Hi! I have this problem. Suppose I have an index to store products sales. And I want to define a visualization (potentially a table) with the following information: Product Name, ranking in number of units sold, rank… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1307) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1309)