# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1659 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1660 --- ## [Custom field (filebeat) in condition in logstash filter](https://discuss.elastic.co/t/custom-field-filebeat-in-condition-in-logstash-filter/257761)
**Author:** [@arp220](https://discuss.elastic.co/u/arp220)\ **Replies:** 3\ **Last updated:** [December 6, 2020, 3:48pm UTC](https://discuss.elastic.co/t/custom-field-filebeat-in-condition-in-logstash-filter/257761 "2020-12-06T15:48:57Z")
I have this config in filebeat.yml: filebeat.inputs: - type: log enabled: true paths: - /var/log/nginx/access.log fields\_under\_root: true fields: service\_name: "nginx" ##### OutPut ####### output.logstas… --- ## [Multiline parsing error](https://discuss.elastic.co/t/multiline-parsing-error/257774)
**Author:** [@Muhammad\_Faisal](https://discuss.elastic.co/u/Muhammad_Faisal)\ **Replies:** 1\ **Last updated:** [December 6, 2020, 3:46pm UTC](https://discuss.elastic.co/t/multiline-parsing-error/257774 "2020-12-06T15:46:55Z")
hi all, i am using multiline to parse below log format...1 log entry is separated by either line which appears as "!-----New----------" or "------------!" ..... i am using below regex ,although its segregating log entri… --- ## [How to get the number of routing shards for an index](https://discuss.elastic.co/t/how-to-get-the-number-of-routing-shards-for-an-index/257634)
**Author:** [@Chelambarasan\_CR](https://discuss.elastic.co/u/Chelambarasan_CR)\ **Replies:** 1\ **Last updated:** [December 6, 2020, 3:13pm UTC](https://discuss.elastic.co/t/how-to-get-the-number-of-routing-shards-for-an-index/257634 "2020-12-06T15:13:48Z")
ES version: 7.6.1 I have mentioned the number of routing shards while index creation as below PUT my\_source\_index { "settings": { "index.number\_of\_shards" : 9, "index.number\_of\_routi… --- ## [How to parse the json string](https://discuss.elastic.co/t/how-to-parse-the-json-string/257768)
**Author:** [@stwang](https://discuss.elastic.co/u/stwang)\ **Replies:** 1\ **Last updated:** [December 6, 2020, 11:44am UTC](https://discuss.elastic.co/t/how-to-parse-the-json-string/257768 "2020-12-06T11:44:27Z")
I have a log file, which the content like below {"log":"{\\"time\\":\\"2020-12-02 22:42:00,535\\", \\"a\_key\\":\\"a\_value\\"}\\n","stream":"stdout","time":"2020-12-02T22:42:05.58702969Z"} I want to parse the log field to json o… --- ## [Separating fields using grok with custom patterns](https://discuss.elastic.co/t/separating-fields-using-grok-with-custom-patterns/257765)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 0\ **Last updated:** [December 6, 2020, 9:07am UTC](https://discuss.elastic.co/t/separating-fields-using-grok-with-custom-patterns/257765 "2020-12-06T09:07:56Z")
Hello! I have a problem while parsing my log file: I can not get the separated fields in stdout using ruby debug. The pattern is okay while using DevTools or the recommended debugger. also I have different log lines whic… --- ## [\[WARN \]\[logstash.outputs.elasticsearch\]\[main\] Marking url as dead](https://discuss.elastic.co/t/warn-logstash-outputs-elasticsearch-main-marking-url-as-dead/257752)
**Author:** [@Cristiane\_Marcarini](https://discuss.elastic.co/u/Cristiane_Marcarini)\ **Replies:** 2\ **Last updated:** [December 6, 2020, 10:01am UTC](https://discuss.elastic.co/t/warn-logstash-outputs-elasticsearch-main-marking-url-as-dead/257752 "2020-12-06T10:01:49Z")
What can I do in these situations? because the logs go up, but they don't appear in the kibana. \[2020-12-05T20:11:29,881\]\[WARN \]\[logstash.outputs.elasticsearch\]\[main\] Marking url as dead. Last error: \[LogStash::Outputs:… --- ## [Problem when adding fields to logstash](https://discuss.elastic.co/t/problem-when-adding-fields-to-logstash/257744)
**Author:** [@Gigazo1d](https://discuss.elastic.co/u/Gigazo1d)\ **Replies:** 5\ **Last updated:** [December 5, 2020, 7:57pm UTC](https://discuss.elastic.co/t/problem-when-adding-fields-to-logstash/257744 "2020-12-05T19:57:35Z")
Hi guys! There was a problem adding the add\_field via the mutate filter plugin. I do this: filter { if \[type\] == "syslog" { mutate { add\_field =\> \[ "received\_at", "%{@timestamp}" \] … --- ## [Kibana Bad Request when creating index pattern](https://discuss.elastic.co/t/kibana-bad-request-when-creating-index-pattern/257394)
**Author:** [@elady](https://discuss.elastic.co/u/elady)\ **Replies:** 2\ **Last updated:** [December 6, 2020, 9:38am UTC](https://discuss.elastic.co/t/kibana-bad-request-when-creating-index-pattern/257394 "2020-12-06T09:38:55Z")
\*\*\* SEE EDIT BELOW \*\*\* Hey. Since we have upgraded ELK stack to version 7.7.1 the Kibana keeps presenting error messages of "bad request" to elasticsearch when we perform index field refresh and now when we are trying … --- ## [Search on the same field multiple times with different filters and intersect the results](https://discuss.elastic.co/t/search-on-the-same-field-multiple-times-with-different-filters-and-intersect-the-results/257312)
**Author:** [@Amir\_Mirahmady](https://discuss.elastic.co/u/Amir_Mirahmady)\ **Replies:** 1\ **Last updated:** [December 6, 2020, 8:43am UTC](https://discuss.elastic.co/t/search-on-the-same-field-multiple-times-with-different-filters-and-intersect-the-results/257312 "2020-12-06T08:43:20Z")
Hi . I'm working on a project that records in my index have dynamic values . I mean I have a field named fieldvalue that can be a date or a text or a number and every record is piece of data from each user . The reason w… --- ## [Delete\_by\_query number of documents Error](https://discuss.elastic.co/t/delete-by-query-number-of-documents-error/257737)
**Author:** [@lstoneir](https://discuss.elastic.co/u/lstoneir)\ **Replies:** 3\ **Last updated:** [December 6, 2020, 8:31am UTC](https://discuss.elastic.co/t/delete-by-query-number-of-documents-error/257737 "2020-12-06T08:31:59Z")
Hi, I run this code: POST topf/\_delete\_by\_query?conflicts=proceed&max\_docs=500000 { "query": { "range": { "time": { "gte": "2020-09-18T03:00:00", "lte": "2020-09-18T04:00:00" } } … --- ## [Difference between latency and service\_time in esrally](https://discuss.elastic.co/t/difference-between-latency-and-service-time-in-esrally/257446)
**Author:** [@vangap](https://discuss.elastic.co/u/vangap)\ **Replies:** 2\ **Last updated:** [December 6, 2020, 8:22am UTC](https://discuss.elastic.co/t/difference-between-latency-and-service-time-in-esrally/257446 "2020-12-06T08:22:58Z")
Hi, It wasn't clear to me the difference between latency and service\_time mentioned here If I understand this correctly, service\_time sounds like a representation of "took" that ES returns in responses, or is it ev… --- ## [Nested key-values parsed with KV](https://discuss.elastic.co/t/nested-key-values-parsed-with-kv/257625)
**Author:** [@Elitlogik](https://discuss.elastic.co/u/Elitlogik)\ **Replies:** 5\ **Last updated:** [December 5, 2020, 11:21pm UTC](https://discuss.elastic.co/t/nested-key-values-parsed-with-kv/257625 "2020-12-05T23:21:36Z")
KV works great for parsing my input. One key-value, however, looks like this: waninfo="name=wan1,bytes=35911872068/568034814778,packets=135560453/438026267;name=wan2,bytes=0/0,packets=0/0;" I would like to KV-parse th… --- ## [Filter out only the \_source](https://discuss.elastic.co/t/filter-out-only-the-source/257729)
**Author:** [@manuel1](https://discuss.elastic.co/u/manuel1)\ **Replies:** 1\ **Last updated:** [December 5, 2020, 2:23pm UTC](https://discuss.elastic.co/t/filter-out-only-the-source/257729 "2020-12-05T14:23:03Z")
After performing a GET request, i get a response like the below. I only need the \_source field for the response. So my response should not include 'hits' nor '\_source'. Just the values of the \_source which are: id, house… --- ## [Transportclient 502 after Added nodes in cluster](https://discuss.elastic.co/t/transportclient-502-after-added-nodes-in-cluster/257492)
**Author:** [@arthurxx](https://discuss.elastic.co/u/arthurxx)\ **Replies:** 5\ **Last updated:** [December 5, 2020, 2:15pm UTC](https://discuss.elastic.co/t/transportclient-502-after-added-nodes-in-cluster/257492 "2020-12-05T14:15:23Z")
there is a elasticsearch cluster which version is 2.4.6 in our company. We met a strange problem recently. After Added nodes into this cluster, the client will met a large of number requests status become 499 or 502( now… --- ## [Setting Up Snapshot - Repository (What to set for Path.Repo)](https://discuss.elastic.co/t/setting-up-snapshot-repository-what-to-set-for-path-repo/257717)
**Author:** [@Sampson\_Light](https://discuss.elastic.co/u/Sampson_Light)\ **Replies:** 4\ **Last updated:** [December 5, 2020, 9:37am UTC](https://discuss.elastic.co/t/setting-up-snapshot-repository-what-to-set-for-path-repo/257717 "2020-12-05T09:37:35Z")
Hi All, So I am currently setting up my path.repo repository to enable Snapshot onto my linux server. I understand that I am suppose to create something new like a path.repo inside my elasticsearch.yml. Something like… --- ## [Error 503 on server post unblocking the index on a proxy](https://discuss.elastic.co/t/error-503-on-server-post-unblocking-the-index-on-a-proxy/257720)
**Author:** [@Anuradha12](https://discuss.elastic.co/u/Anuradha12)\ **Replies:** 7\ **Last updated:** [December 5, 2020, 8:14am UTC](https://discuss.elastic.co/t/error-503-on-server-post-unblocking-the-index-on-a-proxy/257720 "2020-12-05T08:14:45Z")
Could not save: {'index': {'\_index': 'prj\_1a9c0440af7f4ed297736cffa5591082\_connector', '\_type': '\_doc', '\_id': 'model\_d25q7003720200', 'status': 503, 'error': {'type': 'unavailable\_shards\_exception', 'reason': '\[prj\_1a9c… --- ## [Longest prefix match](https://discuss.elastic.co/t/longest-prefix-match/257722)
**Author:** [@sumanthmitra](https://discuss.elastic.co/u/sumanthmitra)\ **Replies:** 0\ **Last updated:** [December 5, 2020, 8:13am UTC](https://discuss.elastic.co/t/longest-prefix-match/257722 "2020-12-05T08:13:09Z")
Hi, I am not able to figure out how to achieve the longest prefix match between the search query and the stored doc values. Say we have the below doc values indexed. "ababab" "ab" "a" and if the input query is "ab… --- ## [Metricbeat not sending data after recreating index](https://discuss.elastic.co/t/metricbeat-not-sending-data-after-recreating-index/257694)
**Author:** [@StilgarBF](https://discuss.elastic.co/u/StilgarBF)\ **Replies:** 0\ **Last updated:** [December 4, 2020, 7:37pm UTC](https://discuss.elastic.co/t/metricbeat-not-sending-data-after-recreating-index/257694 "2020-12-04T19:37:58Z")
Hi, tldr: I had metricbeats running but after deleting and recreating template and index, I see no data comming in. I Have a remote host running elasticsearch and kibana in docker. Metricbeat with only module system w… --- ## [Visualizing top list of skiing results between split times in Kibana](https://discuss.elastic.co/t/visualizing-top-list-of-skiing-results-between-split-times-in-kibana/256936)
**Author:** [@tobias.berg](https://discuss.elastic.co/u/tobias.berg)\ **Replies:** 2\ **Last updated:** [December 5, 2020, 6:35am UTC](https://discuss.elastic.co/t/visualizing-top-list-of-skiing-results-between-split-times-in-kibana/256936 "2020-12-05T06:35:09Z")
Hi, I'm scraping some cross-country skiing results, ingesting it in Elasticsearch and visualizing it in Kibana. It works pretty well! Although there is one thing I would like to accomplish that I can't get my head aroun… --- ## [Elastic: Problem with decimal format when indexing from the 64th document (document not indexed)](https://discuss.elastic.co/t/elastic-problem-with-decimal-format-when-indexing-from-the-64th-document-document-not-indexed/257588)
**Author:** [@oldiequattro](https://discuss.elastic.co/u/oldiequattro)\ **Replies:** 5\ **Last updated:** [December 5, 2020, 4:43am UTC](https://discuss.elastic.co/t/elastic-problem-with-decimal-format-when-indexing-from-the-64th-document-document-not-indexed/257588 "2020-12-05T04:43:40Z")
Hello, I am showing you here a problem encountered in our environments with the indexing of a row in Elasticsearch which is not done as soon as the decimal value contained in the key exceeds 64 (side effect of a base 64… --- ## [Temp downtimes, OptionalDataException (500) on ALL APIs (AWS)](https://discuss.elastic.co/t/temp-downtimes-optionaldataexception-500-on-all-apis-aws/257571)
**Author:** [@aviau](https://discuss.elastic.co/u/aviau)\ **Replies:** 9\ **Last updated:** [December 5, 2020, 4:41am UTC](https://discuss.elastic.co/t/temp-downtimes-optionaldataexception-500-on-all-apis-aws/257571 "2020-12-05T04:41:59Z")
Hello! I am currently running ElasticSearch 7.9.1 on the AWS ElasticSerach service. I have successfully restored a snapshot that I have made on my self-managed 7.2 instance. The cluster works well, can easily ingest t… --- ## [TSVB annotation row template field error](https://discuss.elastic.co/t/tsvb-annotation-row-template-field-error/257158)
**Author:** [@pingz](https://discuss.elastic.co/u/pingz)\ **Replies:** 2\ **Last updated:** [December 5, 2020, 3:50am UTC](https://discuss.elastic.co/t/tsvb-annotation-row-template-field-error/257158 "2020-12-05T03:50:26Z")
Hi, there: My fields value is set to log.message and I set row template to {{log.message}} Clicking the annotated line showed value {{log.message}}. Not the actual message from index Kibana 7.9.1 is used How shal… --- ## [ILM Hot, Warm, Cold?](https://discuss.elastic.co/t/ilm-hot-warm-cold/256958)
**Author:** [@lamp123432](https://discuss.elastic.co/u/lamp123432)\ **Replies:** 8\ **Last updated:** [December 4, 2020, 11:41pm UTC](https://discuss.elastic.co/t/ilm-hot-warm-cold/256958 "2020-12-04T23:41:09Z")
Hello, When I try to setup ILM to make an index to become "Warm" after 30 days, then "Cold" after 60 days, and deleted after 90 days, I get the following: You can't control shard allocation without node attributes. Ca… --- ## [Creating a watcher](https://discuss.elastic.co/t/creating-a-watcher/257247)
**Author:** [@ChristianOelsner](https://discuss.elastic.co/u/ChristianOelsner)\ **Replies:** 2\ **Last updated:** [December 4, 2020, 11:28pm UTC](https://discuss.elastic.co/t/creating-a-watcher/257247 "2020-12-04T23:28:26Z")
Hello guys. I am still very much in a learning process when it comes to the elastic stack. Now i am trying to set up a watcher that can be user for some sort of allerting/notification, be it slack or mail. We have a l… --- ## [Event Correlation/Deviation](https://discuss.elastic.co/t/event-correlation-deviation/257572)
**Author:** [@stevezemlicka](https://discuss.elastic.co/u/stevezemlicka)\ **Replies:** 4\ **Last updated:** [December 4, 2020, 9:48pm UTC](https://discuss.elastic.co/t/event-correlation-deviation/257572 "2020-12-04T21:48:56Z")
I have a series of endpoints that have hourly readings as well as a set of event times. A specific subset of the endpoints' readings are expected to drop during the events. I am trying to figure out a visualization tha… --- ## [Elastic don´t start after set xpack.security.enabled to true](https://discuss.elastic.co/t/elastic-don-t-start-after-set-xpack-security-enabled-to-true/257596)
**Author:** [@jh0mil](https://discuss.elastic.co/u/jh0mil)\ **Replies:** 19\ **Last updated:** [December 4, 2020, 9:14pm UTC](https://discuss.elastic.co/t/elastic-don-t-start-after-set-xpack-security-enabled-to-true/257596 "2020-12-04T21:14:01Z")
I tried activate elastic security, but when i add the line xpack.security.enabled: true if send command systemctl restart elasticsearch this crashes. --- ## [Filebeat does not harvest inactive files](https://discuss.elastic.co/t/filebeat-does-not-harvest-inactive-files/257700)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 0\ **Last updated:** [December 4, 2020, 9:06pm UTC](https://discuss.elastic.co/t/filebeat-does-not-harvest-inactive-files/257700 "2020-12-04T21:06:18Z")
Hi All, I am using filebeat 7.6.2. On starting up, it does not harvest a file which has time stamp of today but not actively being written into. Is there some parameter that I could introduce in filebeat.yml to make i… --- ## [Filebeat cisco/asa module not working](https://discuss.elastic.co/t/filebeat-cisco-asa-module-not-working/257088)
**Author:** [@Rafal\_Radziejewski](https://discuss.elastic.co/u/Rafal_Radziejewski)\ **Replies:** 7\ **Last updated:** [December 4, 2020, 8:10pm UTC](https://discuss.elastic.co/t/filebeat-cisco-asa-module-not-working/257088 "2020-12-04T20:10:41Z")
Elasticsearch 7.9.2 Filebeat 7.10 Running command: sudo filebeat setup -e --dashboards --pipelines --template Produces: 2020-11-30T16:04:16.488+0100 ERROR \[load\] cfgfile/list.go:99 Error creating runner from config:… --- ## [Visualize a time counter](https://discuss.elastic.co/t/visualize-a-time-counter/257225)
**Author:** [@fastxl](https://discuss.elastic.co/u/fastxl)\ **Replies:** 4\ **Last updated:** [December 4, 2020, 7:47pm UTC](https://discuss.elastic.co/t/visualize-a-time-counter/257225 "2020-12-04T19:47:03Z")
I am new to the ELK stack and trying to learn how to visualize some things. I have data coming in from one of our machine tools and I would like to make a simple counter that show the amount of time the machine has been … --- ## [Set all nested fields value into master field](https://discuss.elastic.co/t/set-all-nested-fields-value-into-master-field/257676)
**Author:** [@Raynald](https://discuss.elastic.co/u/Raynald)\ **Replies:** 3\ **Last updated:** [December 4, 2020, 6:01pm UTC](https://discuss.elastic.co/t/set-all-nested-fields-value-into-master-field/257676 "2020-12-04T18:01:42Z")
Hi everyone, I have an index that can growth with many nested fileds, and we want to limit nested field to one sub-level for identified fields. But we want to keep value of this nested fields into the master field Exam… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1658) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1660)