# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1318 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1319 --- ## [Covert logs to JSON in output section of Logstash before it leaves output](https://discuss.elastic.co/t/covert-logs-to-json-in-output-section-of-logstash-before-it-leaves-output/278975)
**Author:** [@shadu88](https://discuss.elastic.co/u/shadu88)\ **Replies:** 1\ **Last updated:** [July 17, 2021, 5:59pm UTC](https://discuss.elastic.co/t/covert-logs-to-json-in-output-section-of-logstash-before-it-leaves-output/278975 "2021-07-17T17:59:29Z")
Hello ELkan's Hope all are safe!!! I'm using kustos output plugin to forward logs to azure data explore which only supports json format, I want to convert my syslog input logs on TCP port to JSON only in kustos output … --- ## [Use common field to filter 2 indexes](https://discuss.elastic.co/t/use-common-field-to-filter-2-indexes/277677)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 10\ **Last updated:** [July 17, 2021, 5:44pm UTC](https://discuss.elastic.co/t/use-common-field-to-filter-2-indexes/277677 "2021-07-17T17:44:11Z")
I'm new to Lens and this is what I'm hoping to accomplish I have one index that contains transaction request information from clients and another index that contains the transaction response information for those client… --- ## [How to parse names like "This+is+name%2+field" in links?](https://discuss.elastic.co/t/how-to-parse-names-like-this-is-name-2-field-in-links/278858)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 8\ **Last updated:** [July 17, 2021, 5:23pm UTC](https://discuss.elastic.co/t/how-to-parse-names-like-this-is-name-2-field-in-links/278858 "2021-07-17T17:23:32Z")
Hi! I have a name field like: "This+is+name%2+field" How can I parse it in logstash grok filters so that I can have: "This is name 2 field" in Kibana? --- ## [Data Format Problem in Elastic: Direct Beats vs Logstash](https://discuss.elastic.co/t/data-format-problem-in-elastic-direct-beats-vs-logstash/278972)
**Author:** [@fynex](https://discuss.elastic.co/u/fynex)\ **Replies:** 0\ **Last updated:** [July 17, 2021, 1:20pm UTC](https://discuss.elastic.co/t/data-format-problem-in-elastic-direct-beats-vs-logstash/278972 "2021-07-17T13:20:36Z")
Hi everyone, I have currently a problem to ingest my logs into elasticsearch the correct way, because the logs look different from the beats=\>logstash=\>elastic input than the beats=\>elastic input (I set up two elasticse… --- ## [Date\_nano field is not considered as a time field in rollup](https://discuss.elastic.co/t/date-nano-field-is-not-considered-as-a-time-field-in-rollup/278933)
**Author:** [@malsioufi](https://discuss.elastic.co/u/malsioufi)\ **Replies:** 1\ **Last updated:** [July 17, 2021, 11:08am UTC](https://discuss.elastic.co/t/date-nano-field-is-not-considered-as-a-time-field-in-rollup/278933 "2021-07-17T11:08:24Z")
I have an index template with @timestamp field of type date\_nano. I can see this field in Discover, and it is treated as a normal time field, but when I want to create a rollup job for the index pattern that has this fi… --- ## [Filebeat redis module not working](https://discuss.elastic.co/t/filebeat-redis-module-not-working/278965)
**Author:** [@Mason](https://discuss.elastic.co/u/Mason)\ **Replies:** 0\ **Last updated:** [July 17, 2021, 10:35am UTC](https://discuss.elastic.co/t/filebeat-redis-module-not-working/278965 "2021-07-17T10:35:03Z")
Hi, I am using redis module from filebeat to monitor a redis docker container. (using docker image docker.elastic.co/beats/filebeat:7.13.1) # enable redis module filebeat.modules: - module: redis # redis container fi… --- ## [Version conflict for bulk index - do the all the docs included into the bulk index fail?](https://discuss.elastic.co/t/version-conflict-for-bulk-index-do-the-all-the-docs-included-into-the-bulk-index-fail/278951)
**Author:** [@smlbiobot](https://discuss.elastic.co/u/smlbiobot)\ **Replies:** 3\ **Last updated:** [July 17, 2021, 10:07am UTC](https://discuss.elastic.co/t/version-conflict-for-bulk-index-do-the-all-the-docs-included-into-the-bulk-index-fail/278951 "2021-07-17T10:07:14Z")
From time to time, I’ve seen bulk index fails because of version conflicts. Because of how things are, I’m ok with some docs failing to index because it exists already. However, in these cases, do all the documents get r… --- ## [Bulk insert is too slow](https://discuss.elastic.co/t/bulk-insert-is-too-slow/278794)
**Author:** [@Hooman\_Bahreini](https://discuss.elastic.co/u/Hooman_Bahreini)\ **Replies:** 8\ **Last updated:** [July 17, 2021, 9:35am UTC](https://discuss.elastic.co/t/bulk-insert-is-too-slow/278794 "2021-07-17T09:35:32Z")
I have a 2 node Elasticsearch (Version 6.8) cluster (they are running on 2 Ubuntu servers with 4GB RAM) and I am running MySQL on the same servers. The bulk insert is writing 100 documents at a time and each insert take… --- ## [Wrong examples in the rank feature query documentation](https://discuss.elastic.co/t/wrong-examples-in-the-rank-feature-query-documentation/278956)
**Author:** [@rueian](https://discuss.elastic.co/u/rueian)\ **Replies:** 0\ **Last updated:** [July 17, 2021, 4:52am UTC](https://discuss.elastic.co/t/wrong-examples-in-the-rank-feature-query-documentation/278956 "2021-07-17T04:52:01Z")
Hi, I noticed that the rank feature query documentation contains the following examples: PUT /test/\_doc/2?refresh { .... "topics": { "sports": 35, "formula one": 65, "brazil": 20 } } PUT /test/\_doc/3?… --- ## [Parsing Syslog with Logstash Grock Filter isn't working with Kibana](https://discuss.elastic.co/t/parsing-syslog-with-logstash-grock-filter-isnt-working-with-kibana/278844)
**Author:** [@olg32](https://discuss.elastic.co/u/olg32)\ **Replies:** 2\ **Last updated:** [July 17, 2021, 12:48am UTC](https://discuss.elastic.co/t/parsing-syslog-with-logstash-grock-filter-isnt-working-with-kibana/278844 "2021-07-17T00:48:45Z")
Hi, I have crated a very basic grok filter to parse Cisco Syslogs: input { udp { port =\> 5140 type =\> syslog } } filter { grok { match =\> { "message"=\> "%{TIMESTAMP\_ISO8601:Timestamp\_Local… --- ## [Adding retry logic to java high level rest client](https://discuss.elastic.co/t/adding-retry-logic-to-java-high-level-rest-client/278950)
**Author:** [@Srinivasan\_Ramaswamy](https://discuss.elastic.co/u/Srinivasan_Ramaswamy)\ **Replies:** 0\ **Last updated:** [July 16, 2021, 11:51pm UTC](https://discuss.elastic.co/t/adding-retry-logic-to-java-high-level-rest-client/278950 "2021-07-16T23:51:05Z")
As requests over http can fail for various reasons, would it make sense to add some retry mechanism to high level rest client ? I see this : Retry Lifecycle Policy API | Java REST Client \[7.13\] | Elastic. Not sure about… --- ## [Ailed to start Elasticsearch. (code=killed, signal=KILL)](https://discuss.elastic.co/t/ailed-to-start-elasticsearch-code-killed-signal-kill/278827)
**Author:** [@Tealk](https://discuss.elastic.co/u/Tealk)\ **Replies:** 3\ **Last updated:** [July 16, 2021, 10:38pm UTC](https://discuss.elastic.co/t/ailed-to-start-elasticsearch-code-killed-signal-kill/278827 "2021-07-16T22:38:48Z")
hello i just want to move my mastodon server and i just installed Elasticsearch on a new server but i get the following error when i start it: systemctl start elasticsearch Job for elasticsearch.service failed because a… --- ## [\[Detection Rule for Security\]: Extract fields from {{context.alerts}} in the Kibana Connector Message Service](https://discuss.elastic.co/t/detection-rule-for-security-extract-fields-from-context-alerts-in-the-kibana-connector-message-service/277919)
**Author:** [@heetS](https://discuss.elastic.co/u/heetS)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 8:49pm UTC](https://discuss.elastic.co/t/detection-rule-for-security-extract-fields-from-context-alerts-in-the-kibana-connector-message-service/277919 "2021-07-16T20:49:40Z")
Hi Everyone, Basically, I was simulating the Password spraying Attack using Kibana Detection Rule (in the Security Tab). For the End-to-End pipeline, I have used WinLog Beat to directly ingest windows log events into th… --- ## [How do I conditionally parse a CSV into different columns?](https://discuss.elastic.co/t/how-do-i-conditionally-parse-a-csv-into-different-columns/278884)
**Author:** [@nsouth](https://discuss.elastic.co/u/nsouth)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 7:57pm UTC](https://discuss.elastic.co/t/how-do-i-conditionally-parse-a-csv-into-different-columns/278884 "2021-07-16T19:57:58Z")
I have CSV log files where the columns change from line to line. The first field is an eventCode, which I can use to know which subsequent columns to expect. Here's a representative example. 1,sameTaskId,username,size 2… --- ## [Word Delimiter Graph Token + Synonym Graph Token](https://discuss.elastic.co/t/word-delimiter-graph-token-synonym-graph-token/278942)
**Author:** [@Kushikawa](https://discuss.elastic.co/u/Kushikawa)\ **Replies:** 0\ **Last updated:** [July 16, 2021, 7:30pm UTC](https://discuss.elastic.co/t/word-delimiter-graph-token-synonym-graph-token/278942 "2021-07-16T19:30:31Z")
I want to use a word delimiter graph token to avoid spliting terms with "+" and "-", e.g. so "3+c" doesn't change to \["3","c"\] and stays \["3+c"\]. I also want to define a list of synonyms to be applied to this field aft… --- ## [Minimum security priviliges for import object API?](https://discuss.elastic.co/t/minimum-security-priviliges-for-import-object-api/277888)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 7:10pm UTC](https://discuss.elastic.co/t/minimum-security-priviliges-for-import-object-api/277888 "2021-07-16T19:10:23Z")
Hi, I need to automatically import dashboards using the import object API. This is all working fine with a account that has full user permissions but I like to lock down the user. What are the minimum permissions a use… --- ## [Free Text Search ML User Interface](https://discuss.elastic.co/t/free-text-search-ml-user-interface/278935)
**Author:** [@kylepiwek](https://discuss.elastic.co/u/kylepiwek)\ **Replies:** 0\ **Last updated:** [July 16, 2021, 6:43pm UTC](https://discuss.elastic.co/t/free-text-search-ml-user-interface/278935 "2021-07-16T18:43:39Z")
Hello, I'm new to the community and to ELKs user interface capabilities and am hoping someone can point me in the right direction. Here is my situation: Basically I have a CSV that contains 2 columns: category informat… --- ## [Disk usage on benchmarks](https://discuss.elastic.co/t/disk-usage-on-benchmarks/278908)
**Author:** [@zj27](https://discuss.elastic.co/u/zj27)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 6:16pm UTC](https://discuss.elastic.co/t/disk-usage-on-benchmarks/278908 "2021-07-16T18:16:38Z")
Hi all, I'm studying the Elasticsearch Adhoc Benchmarks and am curious about the disk usage metrics. Why there is a huge gap between final index size (25GB) and total bytes written(313GB)? --- ## [Control filter between two dashboards](https://discuss.elastic.co/t/control-filter-between-two-dashboards/277767)
**Author:** [@deeksha39](https://discuss.elastic.co/u/deeksha39)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 6:13pm UTC](https://discuss.elastic.co/t/control-filter-between-two-dashboards/277767 "2021-07-16T18:13:46Z")
Hi, I have created two dashboards with similar column names "dashboard1" and "dashboard2". I applied some filters in "dashboard1" through the control panel dropdown "column1"=="some\_value". I saved the filter with the n… --- ## [Search - Request Entity Too Large](https://discuss.elastic.co/t/search-request-entity-too-large/277445)
**Author:** [@Intech\_Ops6](https://discuss.elastic.co/u/Intech_Ops6)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 5:16pm UTC](https://discuss.elastic.co/t/search-request-entity-too-large/277445 "2021-07-16T17:16:27Z")
Kibana version: Kibana 6.8.0 Elasticsearch version: AWS ES/Kibana 6.8.0 Describe the bug: SearchError: Request Entity Too Large at https://name-domain.com/\_plugin/kibana/bundles/kibana.bundle.js:2:583038 at… --- ## [Logstash mapping logfile time to @timestamp](https://discuss.elastic.co/t/logstash-mapping-logfile-time-to-timestamp/278772)
**Author:** [@Vajb12](https://discuss.elastic.co/u/Vajb12)\ **Replies:** 5\ **Last updated:** [July 16, 2021, 5:11pm UTC](https://discuss.elastic.co/t/logstash-mapping-logfile-time-to-timestamp/278772 "2021-07-16T17:11:18Z")
Hi Team, Im trying to map the logfile time with @timestamp in logstash. The logfile time is in IST. I would require some help in converting the logfile time to UTC and map it to @timestamp. I used the below ruby code… --- ## [Unable to login in Kibana after enable x-pack](https://discuss.elastic.co/t/unable-to-login-in-kibana-after-enable-x-pack/275862)
**Author:** [@Fabio1](https://discuss.elastic.co/u/Fabio1)\ **Replies:** 9\ **Last updated:** [July 16, 2021, 4:08pm UTC](https://discuss.elastic.co/t/unable-to-login-in-kibana-after-enable-x-pack/275862 "2021-07-16T16:08:57Z")
Hello, I was using elasticsearch and kibana without problem, but I need to enable some features related with users (to create some accounts and give some permissions to some users), so I need to enable x-pack option to … --- ## [Copy\_to field not searchable as I would expect](https://discuss.elastic.co/t/copy-to-field-not-searchable-as-i-would-expect/278727)
**Author:** [@tasslehawf](https://discuss.elastic.co/u/tasslehawf)\ **Replies:** 2\ **Last updated:** [July 16, 2021, 3:59pm UTC](https://discuss.elastic.co/t/copy-to-field-not-searchable-as-i-would-expect/278727 "2021-07-16T15:59:33Z")
Here is my index: { "settings": { "analysis": { "normalizer": { "keyword\_lowercase": { "type": "custom", "filter": \[ "lowercase", "asciifolding" … --- ## [Filebeat does not pickup rotated logfile from NLog](https://discuss.elastic.co/t/filebeat-does-not-pickup-rotated-logfile-from-nlog/278923)
**Author:** [@paleocomburo](https://discuss.elastic.co/u/paleocomburo)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 3:54pm UTC](https://discuss.elastic.co/t/filebeat-does-not-pickup-rotated-logfile-from-nlog/278923 "2021-07-16T15:54:20Z")
Hello, I'm currently trying to get our structured event logging into Elastic Search. We have a .NET 5 application that is running in a Docker container and it runs on a CentOS 7 server, where it writes the log to a serv… --- ## [Showing data hourly](https://discuss.elastic.co/t/showing-data-hourly/278777)
**Author:** [@prathamesh7](https://discuss.elastic.co/u/prathamesh7)\ **Replies:** 7\ **Last updated:** [July 16, 2021, 3:38pm UTC](https://discuss.elastic.co/t/showing-data-hourly/278777 "2021-07-16T15:38:22Z")
Hi Team, I need to display results of filter i am running in visualisation to show for each hour of a day. When i run filter, i can see date for today but not each hour of day , it is showing with gap of 3 hours, so my… --- ## [Basic understating regarding index creation](https://discuss.elastic.co/t/basic-understating-regarding-index-creation/278189)
**Author:** [@cool999](https://discuss.elastic.co/u/cool999)\ **Replies:** 9\ **Last updated:** [July 16, 2021, 3:34pm UTC](https://discuss.elastic.co/t/basic-understating-regarding-index-creation/278189 "2021-07-16T15:34:58Z")
Hi Team, I am facing issue where I can see index has not got created after some changes in logstash pipeline file. Can you please help me with below questions to understand. I am finding difficulty in getting it. I ha… --- ## [Hourly displaying of data](https://discuss.elastic.co/t/hourly-displaying-of-data/278695)
**Author:** [@cool999](https://discuss.elastic.co/u/cool999)\ **Replies:** 10\ **Last updated:** [July 16, 2021, 3:29pm UTC](https://discuss.elastic.co/t/hourly-displaying-of-data/278695 "2021-07-16T15:29:02Z")
Hi Team, I have log file from which I am filtering specific words and I want to create curl request that will show results based on each hours of current day. In below screenshot, when I select Date histogram on @times… --- ## [Custom regexes.yml](https://discuss.elastic.co/t/custom-regexes-yml/278921)
**Author:** [@gentle\_ghost](https://discuss.elastic.co/u/gentle_ghost)\ **Replies:** 0\ **Last updated:** [July 16, 2021, 3:08pm UTC](https://discuss.elastic.co/t/custom-regexes-yml/278921 "2021-07-16T15:08:52Z")
Hello, Has anyone had any success/experience replacing the user agent plugin regexes.yml file with an entirely different/custom regexes.yml file? I'm curious if anyone has successfully done it and were you required to … --- ## [In Kibana Line chart graph, unable to show values](https://discuss.elastic.co/t/in-kibana-line-chart-graph-unable-to-show-values/278864)
**Author:** [@Sandeep\_Raju](https://discuss.elastic.co/u/Sandeep_Raju)\ **Replies:** 1\ **Last updated:** [July 16, 2021, 2:57pm UTC](https://discuss.elastic.co/t/in-kibana-line-chart-graph-unable-to-show-values/278864 "2021-07-16T14:57:06Z")
Hi all, I'm unable to show values on Kibana Line chart. I'm able to see values when i scroll on value points, but can't get them to be displayed on the chart. Any Turnaround for this? --- ## [Healthcheck based on recent log entries](https://discuss.elastic.co/t/healthcheck-based-on-recent-log-entries/278679)
**Author:** [@ag\_joeb](https://discuss.elastic.co/u/ag_joeb)\ **Replies:** 5\ **Last updated:** [July 16, 2021, 2:55pm UTC](https://discuss.elastic.co/t/healthcheck-based-on-recent-log-entries/278679 "2021-07-16T14:55:46Z")
I'm looking to visualize the following in one my my dashboards. The healthcheck of one of my services would be based on the amount of log entries in the past few seconds; i.e. if there were more than 5 new log entries (… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1317) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1319)