# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1347 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1348 --- ## [XML Logstash](https://discuss.elastic.co/t/xml-logstash/277160)
**Author:** [@leggomyego](https://discuss.elastic.co/u/leggomyego)\ **Replies:** 1\ **Last updated:** [June 27, 2021, 7:21pm UTC](https://discuss.elastic.co/t/xml-logstash/277160 "2021-06-27T19:21:27Z")
Custom fields have been tricky because they are in an array. I can rename them fine and everything, but I am looking for a Ruby answer to my issue. For instance, if the field names are \[test\]\[test\]\[0\]\[test\], \[test\]\[test\]… --- ## [Help with Indexing log with custom time instead of timestamp](https://discuss.elastic.co/t/help-with-indexing-log-with-custom-time-instead-of-timestamp/277126)
**Author:** [@Core\_Sec](https://discuss.elastic.co/u/Core_Sec)\ **Replies:** 6\ **Last updated:** [June 27, 2021, 6:35pm UTC](https://discuss.elastic.co/t/help-with-indexing-log-with-custom-time-instead-of-timestamp/277126 "2021-06-27T18:35:27Z")
Hi everyone I've established ELK with GitHub - deviantony/docker-elk: The Elastic stack (ELK) powered by Docker and Compose. I have log with timestamp included - //{"type":"CustoumLogType","Param1":1,"Param2":1,"Param… --- ## [Create a straight line that is dynamically controlled in a Kibana line graph visualization](https://discuss.elastic.co/t/create-a-straight-line-that-is-dynamically-controlled-in-a-kibana-line-graph-visualization/277139)
**Author:** [@blackberrySherbet](https://discuss.elastic.co/u/blackberrySherbet)\ **Replies:** 2\ **Last updated:** [June 27, 2021, 6:26pm UTC](https://discuss.elastic.co/t/create-a-straight-line-that-is-dynamically-controlled-in-a-kibana-line-graph-visualization/277139 "2021-06-27T18:26:22Z")
Hi, I need to plot a straight line in my line graph. I'm plotting temperature over time. This line would represent the ideal expected temperature. The problem is that this ideal temperature is reevaluated every 12 hour… --- ## [In Kibana data is coming at localtime+3 instead of UTC+3](https://discuss.elastic.co/t/in-kibana-data-is-coming-at-localtime-3-instead-of-utc-3/277117)
**Author:** [@Yaniv\_Etrogi](https://discuss.elastic.co/u/Yaniv_Etrogi)\ **Replies:** 1\ **Last updated:** [June 27, 2021, 5:28pm UTC](https://discuss.elastic.co/t/in-kibana-data-is-coming-at-localtime-3-instead-of-utc-3/277117 "2021-06-27T17:28:08Z")
My system Ubuntu 18.04 Elastic version 7.12 run on docker PowerShell sending data to Elastic from Windows hosts System is set to UTC+3 Elastic is set to UTC+3 My problem In Kibana data is coming at localtime +3 in… --- ## [Field names with dot notation in ES Node.js client](https://discuss.elastic.co/t/field-names-with-dot-notation-in-es-node-js-client/277147)
**Author:** [@Cholley\_Alexandre](https://discuss.elastic.co/u/Cholley_Alexandre)\ **Replies:** 1\ **Last updated:** [June 27, 2021, 4:41pm UTC](https://discuss.elastic.co/t/field-names-with-dot-notation-in-es-node-js-client/277147 "2021-06-27T16:41:11Z")
Hi, I'm trying to write a field with dot notation in the body of the Search API in Node.js Client but it doesn't work. The mapping is : "my\_field": { "type":"text", "fields": { "keyword": { "type"… --- ## [Logstash error](https://discuss.elastic.co/t/logstash-error/277142)
**Author:** [@mangeshmj1992](https://discuss.elastic.co/u/mangeshmj1992)\ **Replies:** 2\ **Last updated:** [June 27, 2021, 2:08pm UTC](https://discuss.elastic.co/t/logstash-error/277142 "2021-06-27T14:08:13Z")
Hello team, I am getting following error on logstash pipeline. can you please help me to identify exact issue. Logstash version: 7.9 C:\\Users\\mangeshsuresh.jadhav\\Downloads\\EKL\\logstash-7.9.0\>bin\\logstash -f wubsprod.… --- ## [Harvester read/write is not working](https://discuss.elastic.co/t/harvester-read-write-is-not-working/277130)
**Author:** [@shankul](https://discuss.elastic.co/u/shankul)\ **Replies:** 2\ **Last updated:** [June 27, 2021, 2:01pm UTC](https://discuss.elastic.co/t/harvester-read-write-is-not-working/277130 "2021-06-27T14:01:10Z")
Hi Team, We are facing some intermittent issues in Kibana logs. While checking we got to know that we are not able to process the filebeat logs and rac is that harvester is not able to write the logs as its only reading… --- ## [Multiline parser](https://discuss.elastic.co/t/multiline-parser/277124)
**Author:** [@bz\_Os](https://discuss.elastic.co/u/bz_Os)\ **Replies:** 2\ **Last updated:** [June 27, 2021, 12:29pm UTC](https://discuss.elastic.co/t/multiline-parser/277124 "2021-06-27T12:29:46Z")
Hello, I am glad to joint the elastic stack community, to pars SQL logs I need to put together rows into one: juin 26 13:30:05 sqlpc mysql-slow: SELECT fk.pib, fk.id, fk.idAccount, fk.idVad, fk.somevalue, fk.somevalue… --- ## [Clean install of ES 7.13.2 on Ubuntu 20.04 hangs](https://discuss.elastic.co/t/clean-install-of-es-7-13-2-on-ubuntu-20-04-hangs/276808)
**Author:** [@Roger\_Clark](https://discuss.elastic.co/u/Roger_Clark)\ **Replies:** 6\ **Last updated:** [June 27, 2021, 11:28am UTC](https://discuss.elastic.co/t/clean-install-of-es-7-13-2-on-ubuntu-20-04-hangs/276808 "2021-06-27T11:28:02Z")
I've been trying to test a clean config of ES stack on an Ubuntu VM following Install Elasticsearch with Debian Package | Elasticsearch Guide \[7.13\] | Elastic The system was a clean install to a 4 core 4GB VM on 20.04.2… --- ## [Logstash twitter plugin parses wrong user id](https://discuss.elastic.co/t/logstash-twitter-plugin-parses-wrong-user-id/277135)
**Author:** [@Lukasz\_Geras](https://discuss.elastic.co/u/Lukasz_Geras)\ **Replies:** 2\ **Last updated:** [June 26, 2021, 9:52pm UTC](https://discuss.elastic.co/t/logstash-twitter-plugin-parses-wrong-user-id/277135 "2021-06-26T21:52:39Z")
Hi, quick question regarding parsing tweets using logstash official plugin. I had a problem parsing tweets, that comes from account other than mine. As you see, using a online converter, I identified my real twitter ID: … --- ## [Logstash not created indexes](https://discuss.elastic.co/t/logstash-not-created-indexes/276277)
**Author:** [@Ilya\_21bA](https://discuss.elastic.co/u/Ilya_21bA)\ **Replies:** 2\ **Last updated:** [June 26, 2021, 9:17pm UTC](https://discuss.elastic.co/t/logstash-not-created-indexes/276277 "2021-06-26T21:17:57Z")
Hi, pls tell me why new indexes not not displayed in Kibana logstash.json: input { file { type =\> "pikautotesttc4" path =\> "C:/Users/bezzbtsev/Desktop/pik2/RTS-PIK/dms-selenium-tests/TestSelenium/bin/Debug/Log… --- ## [Average of Date in visualisation](https://discuss.elastic.co/t/average-of-date-in-visualisation/277031)
**Author:** [@Dzious](https://discuss.elastic.co/u/Dzious)\ **Replies:** 1\ **Last updated:** [June 26, 2021, 6:08pm UTC](https://discuss.elastic.co/t/average-of-date-in-visualisation/277031 "2021-06-26T18:08:08Z")
Hi, I have logs which gives me the time taken from the beginning of the process. From this I'd like to know if i can aggregete theses date and if i can, how to do so. Thanks by advance --- ## [Enhanced Table Plugin line/ word wrap to avoid long string fields extending off screen](https://discuss.elastic.co/t/enhanced-table-plugin-line-word-wrap-to-avoid-long-string-fields-extending-off-screen/277120)
**Author:** [@rak](https://discuss.elastic.co/u/rak)\ **Replies:** 1\ **Last updated:** [June 26, 2021, 6:00pm UTC](https://discuss.elastic.co/t/enhanced-table-plugin-line-word-wrap-to-avoid-long-string-fields-extending-off-screen/277120 "2021-06-26T18:00:43Z")
I have an issue where I have long lines of text extend off the screen causing the need for back and forth scrolling (View picture). Is there a way to have long lines of text line/word wrap so that they are displayed … --- ## [Grok filter](https://discuss.elastic.co/t/grok-filter/276671)
**Author:** [@Sarthak\_Mishra](https://discuss.elastic.co/u/Sarthak_Mishra)\ **Replies:** 13\ **Last updated:** [June 26, 2021, 1:50pm UTC](https://discuss.elastic.co/t/grok-filter/276671 "2021-06-26T13:50:56Z")
Hello, I want to create a grok filter. e.g There is a column in my csv file with all the os system listed windows,ubuntu etc. If there is any typo in the column. I want logstash to create a new field with the name "is\_o… --- ## [{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}](https://discuss.elastic.co/t/statuscode-500-error-internal-server-error-message-an-internal-server-error-occurred/277013)
**Author:** [@Tirupathirao](https://discuss.elastic.co/u/Tirupathirao)\ **Replies:** 6\ **Last updated:** [June 26, 2021, 6:57am UTC](https://discuss.elastic.co/t/statuscode-500-error-internal-server-error-message-an-internal-server-error-occurred/277013 "2021-06-26T06:57:31Z")
{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."} --- ## [How to send synology device logs to elk](https://discuss.elastic.co/t/how-to-send-synology-device-logs-to-elk/269555)
**Author:** [@jubin03](https://discuss.elastic.co/u/jubin03)\ **Replies:** 6\ **Last updated:** [June 26, 2021, 5:59am UTC](https://discuss.elastic.co/t/how-to-send-synology-device-logs-to-elk/269555 "2021-06-26T05:59:38Z")
Do we need to install filbeat on Synology for sending logs to elk --- ## [Add more parameter to kafka input](https://discuss.elastic.co/t/add-more-parameter-to-kafka-input/277109)
**Author:** [@kannan\_raj](https://discuss.elastic.co/u/kannan_raj)\ **Replies:** 0\ **Last updated:** [June 26, 2021, 5:36am UTC](https://discuss.elastic.co/t/add-more-parameter-to-kafka-input/277109 "2021-06-26T05:36:44Z")
Hello, Logstash version is 7.9 and kafka version is 2.5.0 could you please confirm that can we change the below parameter in kafka input plugin? allow.auto.create.topics (default is true) group.instance.id … --- ## [Resolve unassigned shards?](https://discuss.elastic.co/t/resolve-unassigned-shards/276793)
**Author:** [@lchan](https://discuss.elastic.co/u/lchan)\ **Replies:** 5\ **Last updated:** [June 26, 2021, 3:17am UTC](https://discuss.elastic.co/t/resolve-unassigned-shards/276793 "2021-06-26T03:17:57Z")
I am an unassigned shards issue on a newly setup cluster where the ilm policy is not moving the shares from hot to warm. The ilm policy is set to rollover after 1d or 50gb but the allocation is complaining about node do… --- ## [Diagnose high CPU usage of beats deployed by Agent](https://discuss.elastic.co/t/diagnose-high-cpu-usage-of-beats-deployed-by-agent/276998)
**Author:** [@netfire](https://discuss.elastic.co/u/netfire)\ **Replies:** 2\ **Last updated:** [June 26, 2021, 2:51am UTC](https://discuss.elastic.co/t/diagnose-high-cpu-usage-of-beats-deployed-by-agent/276998 "2021-06-26T02:51:32Z")
I'm testing fleet and I have noticed that filebeat and metricbeat use quite a few CPU cycles. I'm using the Windows, System and IIS integrations, and on an 8 vcpu VM, filebeat consistently consumes 20%. Metricbeat consum… --- ## [Get rid of .keyword](https://discuss.elastic.co/t/get-rid-of-keyword/277105)
**Author:** [@svenvg93](https://discuss.elastic.co/u/svenvg93)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 9:36pm UTC](https://discuss.elastic.co/t/get-rid-of-keyword/277105 "2021-06-25T21:36:00Z")
Hi, I'm still a ELK noob so bear with me here ;). I've setup Logstash to send the output of fping command to Elasticsearch and add some geoip data to it. Fping output; ec2.eu-west-1.amazonaws.com : xmt/rcv/%loss = 3… --- ## [Unreadable "Source: (byte\[\])" gives input\_coercion\_exception](https://discuss.elastic.co/t/unreadable-source-byte-gives-input-coercion-exception/277104)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 9:26pm UTC](https://discuss.elastic.co/t/unreadable-source-byte-gives-input-coercion-exception/277104 "2021-06-25T21:26:24Z")
I'm receiving the following warning in my Logstash container. I think it's because there may be some malformed/corrupt data coming through. \[WARN \] 2021-06-25 20:38:37.941 \[\[test\]\>worker0\] elasticsearch - Could not ind… --- ## [How to create new field by comparing the data against JSON using Pipelines or Logstash](https://discuss.elastic.co/t/how-to-create-new-field-by-comparing-the-data-against-json-using-pipelines-or-logstash/277102)
**Author:** [@Mahesh\_Gadagi](https://discuss.elastic.co/u/Mahesh_Gadagi)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 8:03pm UTC](https://discuss.elastic.co/t/how-to-create-new-field-by-comparing-the-data-against-json-using-pipelines-or-logstash/277102 "2021-06-25T20:03:54Z")
Hello, I wanted to create a new field either using logstash or elastic search pipeline, can you please help me? I have JSON mapping as \[{"name":"a","age":25},{"name":"b","age":10}\] I have field already in my index as n… --- ## [Will Filebeat ever send my log file content to stdout?](https://discuss.elastic.co/t/will-filebeat-ever-send-my-log-file-content-to-stdout/276988)
**Author:** [@Wade](https://discuss.elastic.co/u/Wade)\ **Replies:** 3\ **Last updated:** [June 25, 2021, 7:39pm UTC](https://discuss.elastic.co/t/will-filebeat-ever-send-my-log-file-content-to-stdout/276988 "2021-06-25T19:39:14Z")
Is there ever a scenario where filebeat would log the contents of the files it is harvesting to standard out? I am shipping log files to elasticsearch with sensitive information and anything that is written to stdout it … --- ## [I have a use case related to Cumulative Flow Diagram](https://discuss.elastic.co/t/i-have-a-use-case-related-to-cumulative-flow-diagram/277015)
**Author:** [@aswanth\_jabba](https://discuss.elastic.co/u/aswanth_jabba)\ **Replies:** 4\ **Last updated:** [June 25, 2021, 7:32pm UTC](https://discuss.elastic.co/t/i-have-a-use-case-related-to-cumulative-flow-diagram/277015 "2021-06-25T19:32:00Z")
Hi all, I am trying to build a Cumulative Flow Diagram, you can have a look at this I will try to explain how we can view this chart, We have 8 Ready Stories on 10 June and 3 Work in Progress , we can see that the a… --- ## [Index.mapping.total\_fields.limit](https://discuss.elastic.co/t/index-mapping-total-fields-limit/275193)
**Author:** [@Bruceclegg](https://discuss.elastic.co/u/Bruceclegg)\ **Replies:** 6\ **Last updated:** [June 25, 2021, 7:01pm UTC](https://discuss.elastic.co/t/index-mapping-total-fields-limit/275193 "2021-06-25T19:01:43Z")
... We're running elasticsearch 6.8.15 For various reasons we are unable to run Lifecycle Management on our indexes - so I have set up a cron job which uses curator that when an index reaches a certain size or age it w… --- ## [Elasticsearch build failure](https://discuss.elastic.co/t/elasticsearch-build-failure/276965)
**Author:** [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\ **Replies:** 3\ **Last updated:** [June 25, 2021, 6:55pm UTC](https://discuss.elastic.co/t/elasticsearch-build-failure/276965 "2021-06-25T18:55:57Z")
Hello, I was running ELK from source, and today I run "git pull" and tried to build again, but when I run "./gradlew assemble" I am getting these errors : FAILURE: Build failed with an exception. \* What went wrong: A … --- ## [Breaking change in 7.12-13: Date processor](https://discuss.elastic.co/t/breaking-change-in-7-12-13-date-processor/277092)
**Author:** [@Dennis1](https://discuss.elastic.co/u/Dennis1)\ **Replies:** 4\ **Last updated:** [June 25, 2021, 6:22pm UTC](https://discuss.elastic.co/t/breaking-change-in-7-12-13-date-processor/277092 "2021-06-25T18:22:40Z")
I'd like to report a major breaking change that occurred in 7.12.1 (as originally reported and described here: Date filter problems after upgrade), and which I personally experienced when I upgraded from 7.11.1 to 7.13.2… --- ## [Restrict kibana space user access internal apis](https://discuss.elastic.co/t/restrict-kibana-space-user-access-internal-apis/277096)
**Author:** [@south](https://discuss.elastic.co/u/south)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 6:19pm UTC](https://discuss.elastic.co/t/restrict-kibana-space-user-access-internal-apis/277096 "2021-06-25T18:19:00Z")
Hi, I am trying to restrict the users in kibana space to access information for os, memory, authentications.... The space and role of the user only have privileges for kibana analytics(dashboard, visualizations..), but … --- ## [Logstash pipeline](https://discuss.elastic.co/t/logstash-pipeline/277082)
**Author:** [@Sarthak\_Mishra](https://discuss.elastic.co/u/Sarthak_Mishra)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 5:40pm UTC](https://discuss.elastic.co/t/logstash-pipeline/277082 "2021-06-25T17:40:11Z")
My logstash does not ship the data. Please help input { file { path =\> "C:/Users/kumar/Desktop/grok.csv" start\_position =\> "beginning" sincedb\_path =\> "NULL" } } filter { csv { separator =\> "," columns =\> \["Na… --- ## [How to grok pattern match middle of a string?](https://discuss.elastic.co/t/how-to-grok-pattern-match-middle-of-a-string/276708)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 4:51pm UTC](https://discuss.elastic.co/t/how-to-grok-pattern-match-middle-of-a-string/276708 "2021-06-25T16:51:04Z")
I have a string that looks like this: SITE;500400;PEER FLOW (AS 3535);;100G;MR;PP:10.02.01.91 97,98; I just want to extract "FLOW" into a field named peer. I tried doing this: (?\\\w+\\D\\d+\\D\\w+\\s(\\w+)) But tha… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1346) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1348)