# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=614 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 615 --- ## [Unable to extract hostname with Grok using NOTSPACE](https://discuss.elastic.co/t/unable-to-extract-hostname-with-grok-using-notspace/161607)
**Author:** [@drivera](https://discuss.elastic.co/u/drivera)\ **Replies:** 6\ **Last updated:** [January 22, 2019, 4:20pm UTC](https://discuss.elastic.co/t/unable-to-extract-hostname-with-grok-using-notspace/161607 "2019-01-22T16:20:58Z")
Hello, trying to extract a hostname from a log. I test my Grok filter in the Grok debugger using the NOTSPACE pattern and I can extract the hostname "DEV-1010-1010-D-B1F6R01". But in Kibana, I get a "invalid" message for… --- ## [Copy Xml Files from Filebeat to logstash](https://discuss.elastic.co/t/copy-xml-files-from-filebeat-to-logstash/165225)
**Author:** [@TEJASWI\_CHUNDURI](https://discuss.elastic.co/u/TEJASWI_CHUNDURI)\ **Replies:** 1\ **Last updated:** [January 22, 2019, 4:16pm UTC](https://discuss.elastic.co/t/copy-xml-files-from-filebeat-to-logstash/165225 "2019-01-22T16:16:24Z")
I just need to copy the file from filebeat to logstash . Please find the below file where i am trying to copy the file to logstash in a folder (/etc/logstash/appscan\_reports/xml) and trying to execute some script on it ,… --- ## [Ruby filter plugin: how to indicate failure](https://discuss.elastic.co/t/ruby-filter-plugin-how-to-indicate-failure/165055)
**Author:** [@darefilz](https://discuss.elastic.co/u/darefilz)\ **Replies:** 5\ **Last updated:** [January 22, 2019, 3:37pm UTC](https://discuss.elastic.co/t/ruby-filter-plugin-how-to-indicate-failure/165055 "2019-01-22T15:37:41Z")
Hi, currently I use a small Ruby filter script to request a REST API and get some data. If valid data is returned the script is "successful" and the ruby filter applies the add\_tag option and adds some tags. But of cou… --- ## [Ruby Filter Exception](https://discuss.elastic.co/t/ruby-filter-exception/165190)
**Author:** [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Replies:** 4\ **Last updated:** [January 22, 2019, 3:08pm UTC](https://discuss.elastic.co/t/ruby-filter-exception/165190 "2019-01-22T15:08:26Z")
hello, I am using this ruby filter to transform and map my data into a new field Dalist : "8~103511~0~103511~20181209|23~2660000~2382761~277239~20181209" transforming it to DAList =\> \[ \[0\] { "DA\_ID" =\> "8", "DA\_Befo… --- ## [XML filter and tags with no values](https://discuss.elastic.co/t/xml-filter-and-tags-with-no-values/165207)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 2\ **Last updated:** [January 22, 2019, 1:53pm UTC](https://discuss.elastic.co/t/xml-filter-and-tags-with-no-values/165207 "2019-01-22T13:53:51Z")
Hi, I have an XML with some tags that has no value. When parsed, the tags wuth no value are not refected in the target Does exists any option to force to parse the tag and leave it with emptu value? I have tried … --- ## [Parse csv with variable number of columns](https://discuss.elastic.co/t/parse-csv-with-variable-number-of-columns/165170)
**Author:** [@Petr.Simik](https://discuss.elastic.co/u/Petr.Simik)\ **Replies:** 2\ **Last updated:** [January 22, 2019, 1:53pm UTC](https://discuss.elastic.co/t/parse-csv-with-variable-number-of-columns/165170 "2019-01-22T13:53:43Z")
What would be the best way to parse simple file with the structure where second column "optional\_retry\_count" sometime exist sometime not. message\_type, optional\_retry\_count, date, xml\_data EURec,20190121150448,\ **Author:** [@Joel-Costamagna](https://discuss.elastic.co/u/Joel-Costamagna)\ **Replies:** 3\ **Last updated:** [January 22, 2019, 1:10pm UTC](https://discuss.elastic.co/t/logstash-geoip-filter-no-city-name/165182 "2019-01-22T13:10:32Z") Hello, I try to use geoip for my source adresse, but sometimes it shows the correct city name, sometimes it shows a wrong location and there is no geoip.city\_name field. However if i look up the ip i the maxmind demo (h… --- ## [Get an unique register in a block of traces](https://discuss.elastic.co/t/get-an-unique-register-in-a-block-of-traces/165020)
**Author:** [@Viti](https://discuss.elastic.co/u/Viti)\ **Replies:** 12\ **Last updated:** [January 22, 2019, 12:58pm UTC](https://discuss.elastic.co/t/get-an-unique-register-in-a-block-of-traces/165020 "2019-01-22T12:58:49Z")
Hello I need to get an unique register of a block of traces. For example I have the next block of traces: 2019-01-15 22:00:24,2534181078938 ERROR - ID\_TRAZA\_0001 \* FAIL\_1 \* CODE111 \* Error 1 2019-01-15 22:00:24,253418… --- ## [Logstash - IF / ELSE with document\_id](https://discuss.elastic.co/t/logstash-if-else-with-document-id/165214)
**Author:** [@Marcos\_OS](https://discuss.elastic.co/u/Marcos_OS)\ **Replies:** 4\ **Last updated:** [January 22, 2019, 11:18am UTC](https://discuss.elastic.co/t/logstash-if-else-with-document-id/165214 "2019-01-22T11:18:11Z")
Can anyone help me identify why the setup below does not work with document\_id. Context of my configuration, I am importing data from a relational database so I need the sync id. output { stdout { codec =\> rubyde… --- ## [Hourly rotation of ArcSight Indexes](https://discuss.elastic.co/t/hourly-rotation-of-arcsight-indexes/164966)
**Author:** [@fpieressa](https://discuss.elastic.co/u/fpieressa)\ **Replies:** 3\ **Last updated:** [January 22, 2019, 11:08am UTC](https://discuss.elastic.co/t/hourly-rotation-of-arcsight-indexes/164966 "2019-01-22T11:08:23Z")
Hi! We are trying the Logstash ArcSight Module, and we are seeing it creates indexes like arcsight-yyyy.mm.dd, where yyyy-mm-dd is extracted from the deviceReceiveTime field of the CEF event. Is there any way to configu… --- ## [Logstash ----No configuration found in the configured sources](https://discuss.elastic.co/t/logstash-no-configuration-found-in-the-configured-sources/165216)
**Author:** [@Manoj\_Nair](https://discuss.elastic.co/u/Manoj_Nair)\ **Replies:** 0\ **Last updated:** [January 22, 2019, 10:40am UTC](https://discuss.elastic.co/t/logstash-no-configuration-found-in-the-configured-sources/165216 "2019-01-22T10:40:32Z")
Hi All Im new to Elastic and I ave just brought up my system Centos 7 + ELK + Filebeat + Metricbeat I have configured syslog to receive logs from my Sophos firewall and that is working just fine The ELK system is loa… --- ## [Uploading an xml file into logstash](https://discuss.elastic.co/t/uploading-an-xml-file-into-logstash/165016)
**Author:** [@Alaoui](https://discuss.elastic.co/u/Alaoui)\ **Replies:** 19\ **Last updated:** [January 22, 2019, 8:56am UTC](https://discuss.elastic.co/t/uploading-an-xml-file-into-logstash/165016 "2019-01-22T08:56:57Z")
Hi , i already upload my xml file into logtsahs, everything sames okey but the result is not like i want, because i want to specify to each tag to a column so i can do my search in kibana using a name of a column but no… --- ## [Logstash 6.3 Installation Failure](https://discuss.elastic.co/t/logstash-6-3-installation-failure/164876)
**Author:** [@Manoj\_Hettiarachchi](https://discuss.elastic.co/u/Manoj_Hettiarachchi)\ **Replies:** 4\ **Last updated:** [January 22, 2019, 8:56am UTC](https://discuss.elastic.co/t/logstash-6-3-installation-failure/164876 "2019-01-22T08:56:56Z")
Hi All, I needed to install logstash -6.3 in my centos 7 VM. I have downloaded the setup (logstash-6.3.0.rpm) from https://www.elastic.co/downloads/past-releases/logstash-6-3-0 location. I have used the followin… --- ## [NMAP Codec for destination latency measurement](https://discuss.elastic.co/t/nmap-codec-for-destination-latency-measurement/165110)
**Author:** [@Suraj\_Lal\_Gopinadhan](https://discuss.elastic.co/u/Suraj_Lal_Gopinadhan)\ **Replies:** 2\ **Last updated:** [January 22, 2019, 8:03am UTC](https://discuss.elastic.co/t/nmap-codec-for-destination-latency-measurement/165110 "2019-01-22T08:03:45Z")
Im trying to use NMAP Codec to measure the latency for couple destination, and plot in kibana. --- ## [How to forward logs from multiple folder storage locations of rsyslog server to ELK?](https://discuss.elastic.co/t/how-to-forward-logs-from-multiple-folder-storage-locations-of-rsyslog-server-to-elk/164575)
**Author:** [@vignesh.s2](https://discuss.elastic.co/u/vignesh.s2)\ **Replies:** 1\ **Last updated:** [January 22, 2019, 7:53am UTC](https://discuss.elastic.co/t/how-to-forward-logs-from-multiple-folder-storage-locations-of-rsyslog-server-to-elk/164575 "2019-01-22T07:53:33Z")
My case is, I have ELK setup were rsyslog forwards the logs to it. My templates are , $template templmesg,"/data01/RemoteLogs/DLF/%$YEAR%/%$MONTH%/%HOSTNAME%/%HOSTNAME%-%$DAY%-%$MONTH%-%$YEAR%.log" $template mylogsec,… --- ## [Logstash not starting after CPU increase on VM](https://discuss.elastic.co/t/logstash-not-starting-after-cpu-increase-on-vm/165156)
**Author:** [@emailshak](https://discuss.elastic.co/u/emailshak)\ **Replies:** 0\ **Last updated:** [January 22, 2019, 6:07am UTC](https://discuss.elastic.co/t/logstash-not-starting-after-cpu-increase-on-vm/165156 "2019-01-22T06:07:05Z")
Logstash does not start on centos after the number of CPU's were increased. Not sure why it is not starting as it was working before the number of CPU's was increased for the VM. When i do "$service logstash status -l" … --- ## [Filter multiple spaces with a single Dissect field?](https://discuss.elastic.co/t/filter-multiple-spaces-with-a-single-dissect-field/165114)
**Author:** [@drivera](https://discuss.elastic.co/u/drivera)\ **Replies:** 2\ **Last updated:** [January 21, 2019, 10:40pm UTC](https://discuss.elastic.co/t/filter-multiple-spaces-with-a-single-dissect-field/165114 "2019-01-21T22:40:35Z")
Hello, I am trying to extract the message part of the event below which starts at "Not able to... " without having to use multiple %{+error\_message} fields but only a single field. Is that possible? 17.01.2019 17:16:23.… --- ## [Index not present in new ELK instance](https://discuss.elastic.co/t/index-not-present-in-new-elk-instance/164522)
**Author:** [@lynnptilby](https://discuss.elastic.co/u/lynnptilby)\ **Replies:** 1\ **Last updated:** [January 21, 2019, 9:32pm UTC](https://discuss.elastic.co/t/index-not-present-in-new-elk-instance/164522 "2019-01-21T21:32:07Z")
As root curl -s 'http://localhost:9200/\_cat/indices?v' Gives: health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open .kibana\_1 vmG8X7ucTPWOTDn-YQxaow … --- ## [Trying to make connection to secured cluster](https://discuss.elastic.co/t/trying-to-make-connection-to-secured-cluster/165121)
**Author:** [@bigdamhero](https://discuss.elastic.co/u/bigdamhero)\ **Replies:** 0\ **Last updated:** [January 21, 2019, 9:20pm UTC](https://discuss.elastic.co/t/trying-to-make-connection-to-secured-cluster/165121 "2019-01-21T21:20:00Z")
I am trying to set up my DEV cluster it sounded like it would be easier to create the certificates without a password and set the clients to use the certificate and not the full authentication (where the certificate dns … --- ## [Logstash - Aggregate Sylsog events based on multiple Fields](https://discuss.elastic.co/t/logstash-aggregate-sylsog-events-based-on-multiple-fields/165080)
**Author:** [@jorj](https://discuss.elastic.co/u/jorj)\ **Replies:** 1\ **Last updated:** [January 21, 2019, 6:37pm UTC](https://discuss.elastic.co/t/logstash-aggregate-sylsog-events-based-on-multiple-fields/165080 "2019-01-21T18:37:48Z")
Hello I am receiving Syslog event from a CGNAT device, that i am able to parse properly. My issue is I am facing a large number of logs, as for each difference Source Port (irrelevant Field for me), i i am receiving a … --- ## [Im using Logstash 5.6 restarting continuously](https://discuss.elastic.co/t/im-using-logstash-5-6-restarting-continuously/164960)
**Author:** [@Suraj\_Lal\_Gopinadhan](https://discuss.elastic.co/u/Suraj_Lal_Gopinadhan)\ **Replies:** 5\ **Last updated:** [January 21, 2019, 6:36pm UTC](https://discuss.elastic.co/t/im-using-logstash-5-6-restarting-continuously/164960 "2019-01-21T18:36:49Z")
Im using Logstash 5.6 and it is continuously restarting the process after every 12 sec, while starting with "systemctl start logstash". if im starting "bin/logstash --path.settings /etc/logstash" it is working properly. … --- ## [Logstash filter for renaming based on a substring](https://discuss.elastic.co/t/logstash-filter-for-renaming-based-on-a-substring/165089)
**Author:** [@tomasz](https://discuss.elastic.co/u/tomasz)\ **Replies:** 1\ **Last updated:** [January 21, 2019, 5:23pm UTC](https://discuss.elastic.co/t/logstash-filter-for-renaming-based-on-a-substring/165089 "2019-01-21T17:23:37Z")
I can't figure out how to rename fileds based on the pattern. At the moment, in Logstash (6.5) I have a banch of filters: mutate { rename =\> { "sensor\_source\_a" =\> "sensor.labels.a" } } mutate { rename =\> { "sensor\_sou… --- ## [Manual method for triggering logstash to process log4j2 properties?](https://discuss.elastic.co/t/manual-method-for-triggering-logstash-to-process-log4j2-properties/165093)
**Author:** [@slmingol](https://discuss.elastic.co/u/slmingol)\ **Replies:** 0\ **Last updated:** [January 21, 2019, 4:59pm UTC](https://discuss.elastic.co/t/manual-method-for-triggering-logstash-to-process-log4j2-properties/165093 "2019-01-21T16:59:32Z")
I'm in the process of making changes to our log4j2.properties file for logstash and was wondering if there was a way to trigger the running of the log4j2 rules via a Unix signal, perhaps kill -3 or some such. It would be… --- ## [Unable to start Logstash 6.5.4/I am trying the command below : \*\*C:\\Users\\logstash-6.5.4\\bin\> .\\logstash -e 'input { stdin{}} output { stdout{}}'\*\*/](https://discuss.elastic.co/t/unable-to-start-logstash-6-5-4-i-am-trying-the-command-below-c-users-logstash-6-5-4-bin-logstash-e-input-stdin-output-stdout/164932)
**Author:** [@Prasenjit](https://discuss.elastic.co/u/Prasenjit)\ **Replies:** 2\ **Last updated:** [January 21, 2019, 4:22pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-6-5-4-i-am-trying-the-command-below-c-users-logstash-6-5-4-bin-logstash-e-input-stdin-output-stdout/164932 "2019-01-21T16:22:46Z")
Copied the error in the following : https://pastebin.com/8FRMYsCN --- ## [Send unparsed JSON as a field to the elasticsearch from logstash](https://discuss.elastic.co/t/send-unparsed-json-as-a-field-to-the-elasticsearch-from-logstash/165069)
**Author:** [@dbonnin](https://discuss.elastic.co/u/dbonnin)\ **Replies:** 1\ **Last updated:** [January 21, 2019, 3:28pm UTC](https://discuss.elastic.co/t/send-unparsed-json-as-a-field-to-the-elasticsearch-from-logstash/165069 "2019-01-21T15:28:49Z")
I have this message log: 10:49:38,664 INFO (ajp-executor-threads - 156) JAR:SOME\_JAR ESB66ID:6648 ISV:2019H01B18 SOL:1 NC:1 SRV:CAHREST001 INPUT:{"someky": "somevalue"} And this grok pattern: %{TIME:hora} INFO \\(ajp-e… --- ## [Dynamic split field and nested loops](https://discuss.elastic.co/t/dynamic-split-field-and-nested-loops/164808)
**Author:** [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Replies:** 5\ **Last updated:** [January 21, 2019, 2:49pm UTC](https://discuss.elastic.co/t/dynamic-split-field-and-nested-loops/164808 "2019-01-21T14:49:30Z")
I have this Field "dAList" : "8~275273~273586~1687~20181209|23~2660000~2660000~0~20181209" each part separated by "|" is a Da Number i want to split it first to display for example Da\_1,Da\_2 inside the dAList and the va… --- ## [Index only created if sincedb\_path = /dev/null](https://discuss.elastic.co/t/index-only-created-if-sincedb-path-dev-null/164657)
**Author:** [@wholzgruber](https://discuss.elastic.co/u/wholzgruber)\ **Replies:** 2\ **Last updated:** [January 21, 2019, 2:24pm UTC](https://discuss.elastic.co/t/index-only-created-if-sincedb-path-dev-null/164657 "2019-01-21T14:24:44Z")
Dear Community, I need your help regarding following issue. I´m parsing a couple of log files from a honeywell system. Those 68 log files are quite small (about 600kb until 22mb) and have timestamps from 2014 until no… --- ## [Logstash Beats Error \[ELK Stack 6.5.4\]](https://discuss.elastic.co/t/logstash-beats-error-elk-stack-6-5-4/164965)
**Author:** [@elkonator](https://discuss.elastic.co/u/elkonator)\ **Replies:** 4\ **Last updated:** [January 21, 2019, 2:24pm UTC](https://discuss.elastic.co/t/logstash-beats-error-elk-stack-6-5-4/164965 "2019-01-21T14:24:00Z")
Hey folks, after a long time away from the elk stack I wanted to set up another testing scenario. I followed the latest stack guide from here: https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-starte… --- ## [Elastic Cloud Setup : Pipeline Worker Stops in Logstash](https://discuss.elastic.co/t/elastic-cloud-setup-pipeline-worker-stops-in-logstash/165051)
**Author:** [@Deepankar\_Biswas1](https://discuss.elastic.co/u/Deepankar_Biswas1)\ **Replies:** 4\ **Last updated:** [January 21, 2019, 2:11pm UTC](https://discuss.elastic.co/t/elastic-cloud-setup-pipeline-worker-stops-in-logstash/165051 "2019-01-21T14:11:56Z")
I have a setup of ES, Kibana on the cloud and LS and Filebeat (2 nodes). It was all working fine but suddenly i I see today that the pipelines stop in LS with the following error: \[2019-01-21T12:51:07,188\]\[ERROR\]\[logst… --- ## [Need some help with Ruby event.set / get to use multiplication in config](https://discuss.elastic.co/t/need-some-help-with-ruby-event-set-get-to-use-multiplication-in-config/165042)
**Author:** [@Datakids](https://discuss.elastic.co/u/Datakids)\ **Replies:** 2\ **Last updated:** [January 21, 2019, 1:43pm UTC](https://discuss.elastic.co/t/need-some-help-with-ruby-event-set-get-to-use-multiplication-in-config/165042 "2019-01-21T13:43:02Z")
Currency and Gross\_Value are existing fields: mutate { add\_field =\> {"Gross\_Euro" =\> "" }} mutate { convert =\> { "Gross\_Euro" =\> "float" }} if "INR" in \[Currency\] { ruby { code =\> "event \['Gross\_Euro'\] = event\['Gross\_V… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=613) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=615)