# Latest **URL:** https://discuss.elastic.co/latest.md?page=1568 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1569 --- ## [Logstash and Json, How to ingest this fragment:](https://discuss.elastic.co/t/logstash-and-json-how-to-ingest-this-fragment/278966)
**Author:** [@cibernicola](https://discuss.elastic.co/u/cibernicola)\ **Replies:** 2\ **Last updated:** [July 18, 2021, 11:02am UTC](https://discuss.elastic.co/t/logstash-and-json-how-to-ingest-this-fragment/278966 "2021-07-18T11:02:59Z")
I'm trying to ingest json data with this format: {"info":{"page":1,"pages":53431,"results":53431},"plugins":\[{"name":"Contact Form 7","slug":"contact-form-7","version":"5.4.2","author":"\ **Author:** [@Jedrick](https://discuss.elastic.co/u/Jedrick)\ **Replies:** 3\ **Last updated:** [July 18, 2021, 9:24am UTC](https://discuss.elastic.co/t/failed-to-start-elastic-search/278899 "2021-07-18T09:24:36Z") Hi, Good day. Can you help me with my problem with elastic search? Currently, i am building a Wazuh siem that uses elastic search functions. As I checked, Filebeat, kibana services are running. But the elasticsearch … --- ## [Filter with Grok and KV](https://discuss.elastic.co/t/filter-with-grok-and-kv/278697)
**Author:** [@Sunflower](https://discuss.elastic.co/u/Sunflower)\ **Replies:** 5\ **Last updated:** [July 18, 2021, 6:52am UTC](https://discuss.elastic.co/t/filter-with-grok-and-kv/278697 "2021-07-18T06:52:22Z")
Hi, I'm trying to understand if I'm using the correct configuration for the following logs, or if there is a more efficient option for it. 2021-07-13T10:05:06.061Z 10.20.30.40 \<110\>1 2021-07-13T08:46:58Z 44.236.133.39… --- ## [Reindexing Old index](https://discuss.elastic.co/t/reindexing-old-index/278360)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 3\ **Last updated:** [July 18, 2021, 3:50am UTC](https://discuss.elastic.co/t/reindexing-old-index/278360 "2021-07-18T03:50:51Z")
I want to reindex old index but the problem my alias is writing to current index health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green ope… --- ## [Lost data in kibana but show in syslog logstash](https://discuss.elastic.co/t/lost-data-in-kibana-but-show-in-syslog-logstash/278894)
**Author:** [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\ **Replies:** 11\ **Last updated:** [July 17, 2021, 7:49pm UTC](https://discuss.elastic.co/t/lost-data-in-kibana-but-show-in-syslog-logstash/278894 "2021-07-17T19:49:07Z")
I have conf file in logstash like this input { beats { host =\> "0.0.0.0" port =\> 5044 ssl =\> false } } filter { if \[fileset\]\[name\] == "auth" { grok { match =\> { "message" =\> "%{SYSLOGT… --- ## [HELP! Logstash restarting after ip2location plugin update](https://discuss.elastic.co/t/help-logstash-restarting-after-ip2location-plugin-update/278957)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 4\ **Last updated:** [July 17, 2021, 6:33pm UTC](https://discuss.elastic.co/t/help-logstash-restarting-after-ip2location-plugin-update/278957 "2021-07-17T18:33:59Z")
I had an issue with logstash restarting so I updated ip2location plugin and now it keeps restarting every 15 seconds. The error before was ConcurrentModificationException: null. I use logstash v 7.4.2 and the plugin ver… --- ## [I am trying to pick a field from one event to another using aggregation filter](https://discuss.elastic.co/t/i-am-trying-to-pick-a-field-from-one-event-to-another-using-aggregation-filter/278955)
**Author:** [@Kamikaze\_K](https://discuss.elastic.co/u/Kamikaze_K)\ **Replies:** 1\ **Last updated:** [July 17, 2021, 6:10pm UTC](https://discuss.elastic.co/t/i-am-trying-to-pick-a-field-from-one-event-to-another-using-aggregation-filter/278955 "2021-07-17T18:10:01Z")
have log files that I am able to get fields based on two different if/grok statements and patterns. The output from the two are like below; { timestamp" =\> 2021-06-09T03:08:30.943Z, "Loc" =\> "91340", … --- ## [Ailed to parse field \[host\] of type \[text\] in document with id](https://discuss.elastic.co/t/ailed-to-parse-field-host-of-type-text-in-document-with-id/278963)
**Author:** [@Bhuwaneshwar](https://discuss.elastic.co/u/Bhuwaneshwar)\ **Replies:** 1\ **Last updated:** [July 17, 2021, 6:05pm UTC](https://discuss.elastic.co/t/ailed-to-parse-field-host-of-type-text-in-document-with-id/278963 "2021-07-17T18:05:00Z")
Dear Team I am getting below error on Logstash. "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse field \[host\] of type \[text\] in document with id '4qrUs3oBqKFEAtQHASue'. Preview of field's value:… --- ## [Covert logs to JSON in output section of Logstash before it leaves output](https://discuss.elastic.co/t/covert-logs-to-json-in-output-section-of-logstash-before-it-leaves-output/278975)
**Author:** [@shadu88](https://discuss.elastic.co/u/shadu88)\ **Replies:** 1\ **Last updated:** [July 17, 2021, 5:59pm UTC](https://discuss.elastic.co/t/covert-logs-to-json-in-output-section-of-logstash-before-it-leaves-output/278975 "2021-07-17T17:59:29Z")
Hello ELkan's Hope all are safe!!! I'm using kustos output plugin to forward logs to azure data explore which only supports json format, I want to convert my syslog input logs on TCP port to JSON only in kustos output … --- ## [Use common field to filter 2 indexes](https://discuss.elastic.co/t/use-common-field-to-filter-2-indexes/277677)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 10\ **Last updated:** [July 17, 2021, 5:44pm UTC](https://discuss.elastic.co/t/use-common-field-to-filter-2-indexes/277677 "2021-07-17T17:44:11Z")
I'm new to Lens and this is what I'm hoping to accomplish I have one index that contains transaction request information from clients and another index that contains the transaction response information for those client… --- ## [How to parse names like "This+is+name%2+field" in links?](https://discuss.elastic.co/t/how-to-parse-names-like-this-is-name-2-field-in-links/278858)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 8\ **Last updated:** [July 17, 2021, 5:23pm UTC](https://discuss.elastic.co/t/how-to-parse-names-like-this-is-name-2-field-in-links/278858 "2021-07-17T17:23:32Z")
Hi! I have a name field like: "This+is+name%2+field" How can I parse it in logstash grok filters so that I can have: "This is name 2 field" in Kibana? --- ## [Data Format Problem in Elastic: Direct Beats vs Logstash](https://discuss.elastic.co/t/data-format-problem-in-elastic-direct-beats-vs-logstash/278972)
**Author:** [@fynex](https://discuss.elastic.co/u/fynex)\ **Replies:** 0\ **Last updated:** [July 17, 2021, 1:20pm UTC](https://discuss.elastic.co/t/data-format-problem-in-elastic-direct-beats-vs-logstash/278972 "2021-07-17T13:20:36Z")
Hi everyone, I have currently a problem to ingest my logs into elasticsearch the correct way, because the logs look different from the beats=\>logstash=\>elastic input than the beats=\>elastic input (I set up two elasticse… --- ## [Date\_nano field is not considered as a time field in rollup](https://discuss.elastic.co/t/date-nano-field-is-not-considered-as-a-time-field-in-rollup/278933)
**Author:** [@malsioufi](https://discuss.elastic.co/u/malsioufi)\ **Replies:** 1\ **Last updated:** [July 17, 2021, 11:08am UTC](https://discuss.elastic.co/t/date-nano-field-is-not-considered-as-a-time-field-in-rollup/278933 "2021-07-17T11:08:24Z")
I have an index template with @timestamp field of type date\_nano. I can see this field in Discover, and it is treated as a normal time field, but when I want to create a rollup job for the index pattern that has this fi… --- ## [Filebeat redis module not working](https://discuss.elastic.co/t/filebeat-redis-module-not-working/278965)
**Author:** [@Mason](https://discuss.elastic.co/u/Mason)\ **Replies:** 0\ **Last updated:** [July 17, 2021, 10:35am UTC](https://discuss.elastic.co/t/filebeat-redis-module-not-working/278965 "2021-07-17T10:35:03Z")
Hi, I am using redis module from filebeat to monitor a redis docker container. (using docker image docker.elastic.co/beats/filebeat:7.13.1) # enable redis module filebeat.modules: - module: redis # redis container fi… --- ## [Version conflict for bulk index - do the all the docs included into the bulk index fail?](https://discuss.elastic.co/t/version-conflict-for-bulk-index-do-the-all-the-docs-included-into-the-bulk-index-fail/278951)
**Author:** [@smlbiobot](https://discuss.elastic.co/u/smlbiobot)\ **Replies:** 3\ **Last updated:** [July 17, 2021, 10:07am UTC](https://discuss.elastic.co/t/version-conflict-for-bulk-index-do-the-all-the-docs-included-into-the-bulk-index-fail/278951 "2021-07-17T10:07:14Z")
From time to time, I’ve seen bulk index fails because of version conflicts. Because of how things are, I’m ok with some docs failing to index because it exists already. However, in these cases, do all the documents get r… --- ## [Bulk insert is too slow](https://discuss.elastic.co/t/bulk-insert-is-too-slow/278794)
**Author:** [@Hooman\_Bahreini](https://discuss.elastic.co/u/Hooman_Bahreini)\ **Replies:** 8\ **Last updated:** [July 17, 2021, 9:35am UTC](https://discuss.elastic.co/t/bulk-insert-is-too-slow/278794 "2021-07-17T09:35:32Z")
I have a 2 node Elasticsearch (Version 6.8) cluster (they are running on 2 Ubuntu servers with 4GB RAM) and I am running MySQL on the same servers. The bulk insert is writing 100 documents at a time and each insert take… --- ## [Wrong examples in the rank feature query documentation](https://discuss.elastic.co/t/wrong-examples-in-the-rank-feature-query-documentation/278956)
**Author:** [@rueian](https://discuss.elastic.co/u/rueian)\ **Replies:** 0\ **Last updated:** [July 17, 2021, 4:52am UTC](https://discuss.elastic.co/t/wrong-examples-in-the-rank-feature-query-documentation/278956 "2021-07-17T04:52:01Z")
Hi, I noticed that the rank feature query documentation contains the following examples: PUT /test/\_doc/2?refresh { .... "topics": { "sports": 35, "formula one": 65, "brazil": 20 } } PUT /test/\_doc/3?… --- ## [Node processors](https://discuss.elastic.co/t/node-processors/273803)
**Author:** [@Cristian1988](https://discuss.elastic.co/u/Cristian1988)\ **Replies:** 7\ **Last updated:** [July 17, 2021, 12:59am UTC](https://discuss.elastic.co/t/node-processors/273803 "2021-07-17T00:59:40Z")
Estoy teniendo algunos rechazos a la hora de lanzar un bulk contra un elàstic y se producen con más frecuencia desde que subimos de versión de una versión 6 a una 7, revisando settings y mappings de todo el clúster, de u… --- ## [Parsing Syslog with Logstash Grock Filter isn't working with Kibana](https://discuss.elastic.co/t/parsing-syslog-with-logstash-grock-filter-isnt-working-with-kibana/278844)
**Author:** [@olg32](https://discuss.elastic.co/u/olg32)\ **Replies:** 2\ **Last updated:** [July 17, 2021, 12:48am UTC](https://discuss.elastic.co/t/parsing-syslog-with-logstash-grock-filter-isnt-working-with-kibana/278844 "2021-07-17T00:48:45Z")
Hi, I have crated a very basic grok filter to parse Cisco Syslogs: input { udp { port =\> 5140 type =\> syslog } } filter { grok { match =\> { "message"=\> "%{TIMESTAMP\_ISO8601:Timestamp\_Local… --- ## [Adding retry logic to java high level rest client](https://discuss.elastic.co/t/adding-retry-logic-to-java-high-level-rest-client/278950)
**Author:** [@Srinivasan\_Ramaswamy](https://discuss.elastic.co/u/Srinivasan_Ramaswamy)\ **Replies:** 0\ **Last updated:** [July 16, 2021, 11:51pm UTC](https://discuss.elastic.co/t/adding-retry-logic-to-java-high-level-rest-client/278950 "2021-07-16T23:51:05Z")
As requests over http can fail for various reasons, would it make sense to add some retry mechanism to high level rest client ? I see this : Retry Lifecycle Policy API | Java REST Client \[7.13\] | Elastic. Not sure about… --- ## [Ailed to start Elasticsearch. (code=killed, signal=KILL)](https://discuss.elastic.co/t/ailed-to-start-elasticsearch-code-killed-signal-kill/278827)
**Author:** [@Tealk](https://discuss.elastic.co/u/Tealk)\ **Replies:** 3\ **Last updated:** [July 16, 2021, 10:38pm UTC](https://discuss.elastic.co/t/ailed-to-start-elasticsearch-code-killed-signal-kill/278827 "2021-07-16T22:38:48Z")
hello i just want to move my mastodon server and i just installed Elasticsearch on a new server but i get the following error when i start it: systemctl start elasticsearch Job for elasticsearch.service failed because a… --- ## [\[Detection Rule for Security\]: Extract fields from {{context.alerts}} in the Kibana Connector Message Service](https://discuss.elastic.co/t/detection-rule-for-security-extract-fields-from-context-alerts-in-the-kibana-connector-message-service/277919)
**Author:** [@heetS](https://discuss.elastic.co/u/heetS)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 8:49pm UTC](https://discuss.elastic.co/t/detection-rule-for-security-extract-fields-from-context-alerts-in-the-kibana-connector-message-service/277919 "2021-07-16T20:49:40Z")
Hi Everyone, Basically, I was simulating the Password spraying Attack using Kibana Detection Rule (in the Security Tab). For the End-to-End pipeline, I have used WinLog Beat to directly ingest windows log events into th… --- ## [Assign Single Exception to Multiple Detection Rules](https://discuss.elastic.co/t/assign-single-exception-to-multiple-detection-rules/278930)
**Author:** [@BenB196](https://discuss.elastic.co/u/BenB196)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 8:38pm UTC](https://discuss.elastic.co/t/assign-single-exception-to-multiple-detection-rules/278930 "2021-07-16T20:38:00Z")
Hi All, I currently have a use-case where I have a few rules that looks at similar data, but are intended to detect different things. These rules have a very similar set of exceptions assigned to them. I'd like to be a… --- ## [How do I conditionally parse a CSV into different columns?](https://discuss.elastic.co/t/how-do-i-conditionally-parse-a-csv-into-different-columns/278884)
**Author:** [@nsouth](https://discuss.elastic.co/u/nsouth)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 7:57pm UTC](https://discuss.elastic.co/t/how-do-i-conditionally-parse-a-csv-into-different-columns/278884 "2021-07-16T19:57:58Z")
I have CSV log files where the columns change from line to line. The first field is an eventCode, which I can use to know which subsequent columns to expect. Here's a representative example. 1,sameTaskId,username,size 2… --- ## [Word Delimiter Graph Token + Synonym Graph Token](https://discuss.elastic.co/t/word-delimiter-graph-token-synonym-graph-token/278942)
**Author:** [@Kushikawa](https://discuss.elastic.co/u/Kushikawa)\ **Replies:** 0\ **Last updated:** [July 16, 2021, 7:30pm UTC](https://discuss.elastic.co/t/word-delimiter-graph-token-synonym-graph-token/278942 "2021-07-16T19:30:31Z")
I want to use a word delimiter graph token to avoid spliting terms with "+" and "-", e.g. so "3+c" doesn't change to \["3","c"\] and stays \["3+c"\]. I also want to define a list of synonyms to be applied to this field aft… --- ## [Minimum security priviliges for import object API?](https://discuss.elastic.co/t/minimum-security-priviliges-for-import-object-api/277888)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 7:10pm UTC](https://discuss.elastic.co/t/minimum-security-priviliges-for-import-object-api/277888 "2021-07-16T19:10:23Z")
Hi, I need to automatically import dashboards using the import object API. This is all working fine with a account that has full user permissions but I like to lock down the user. What are the minimum permissions a use… --- ## [Free Text Search ML User Interface](https://discuss.elastic.co/t/free-text-search-ml-user-interface/278935)
**Author:** [@kylepiwek](https://discuss.elastic.co/u/kylepiwek)\ **Replies:** 0\ **Last updated:** [July 16, 2021, 6:43pm UTC](https://discuss.elastic.co/t/free-text-search-ml-user-interface/278935 "2021-07-16T18:43:39Z")
Hello, I'm new to the community and to ELKs user interface capabilities and am hoping someone can point me in the right direction. Here is my situation: Basically I have a CSV that contains 2 columns: category informat… --- ## [Disk usage on benchmarks](https://discuss.elastic.co/t/disk-usage-on-benchmarks/278908)
**Author:** [@zj27](https://discuss.elastic.co/u/zj27)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 6:16pm UTC](https://discuss.elastic.co/t/disk-usage-on-benchmarks/278908 "2021-07-16T18:16:38Z")
Hi all, I'm studying the Elasticsearch Adhoc Benchmarks and am curious about the disk usage metrics. Why there is a huge gap between final index size (25GB) and total bytes written(313GB)? --- ## [Control filter between two dashboards](https://discuss.elastic.co/t/control-filter-between-two-dashboards/277767)
**Author:** [@deeksha39](https://discuss.elastic.co/u/deeksha39)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 6:13pm UTC](https://discuss.elastic.co/t/control-filter-between-two-dashboards/277767 "2021-07-16T18:13:46Z")
Hi, I have created two dashboards with similar column names "dashboard1" and "dashboard2". I applied some filters in "dashboard1" through the control panel dropdown "column1"=="some\_value". I saved the filter with the n… --- ## [Suppression of repeated alerts](https://discuss.elastic.co/t/suppression-of-repeated-alerts/277365)
**Author:** [@dhiegoalves](https://discuss.elastic.co/u/dhiegoalves)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 6:05pm UTC](https://discuss.elastic.co/t/suppression-of-repeated-alerts/277365 "2021-07-16T18:05:04Z")
Hello guys! Is it possible to configure a suppression to a SIEM's rule on ELK SIEM module? For example, if 5 events to the same user.id and source.ip in 5 minuts, so will be trigged just 1 alert. [Previous page](https://discuss.elastic.co/latest.md?page=1567) [Next page](https://discuss.elastic.co/latest.md?page=1569)