# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=830 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 831 --- ## [X-Pack & Logstash](https://discuss.elastic.co/t/x-pack-logstash/115588)
**Author:** [@pepperhat](https://discuss.elastic.co/u/pepperhat)\ **Replies:** 2\ **Last updated:** [February 1, 2018, 8:03pm UTC](https://discuss.elastic.co/t/x-pack-logstash/115588 "2018-02-01T20:03:18Z")
When using X-Pack to configure Logstash pipelines, an edit still needs to be made to the logstash.yml to enable the pipeline. This seems to negate the purpose of having centralized pipeline management if I still need to … --- ## [Optional field does not work](https://discuss.elastic.co/t/optional-field-does-not-work/118096)
**Author:** [@robertchen117](https://discuss.elastic.co/u/robertchen117)\ **Replies:** 0\ **Last updated:** [February 1, 2018, 7:52pm UTC](https://discuss.elastic.co/t/optional-field-does-not-work/118096 "2018-02-01T19:52:44Z")
2018-01-23 11:13:05,965 \[ajp-nio-8009-exec-4\] INFO com.paymentus.rotp.rt.SessionManager - Request\[26788\], \[Path: /rotp/abcd, RCID: 8065666700, USR: Consumer, GID: 0, CID: 805454546700, SID: 55EC7FA9E622B9233FBC0A687DC40… --- ## [Logstash SSL certificate](https://discuss.elastic.co/t/logstash-ssl-certificate/118095)
**Author:** [@bm1391](https://discuss.elastic.co/u/bm1391)\ **Replies:** 0\ **Last updated:** [February 1, 2018, 7:52pm UTC](https://discuss.elastic.co/t/logstash-ssl-certificate/118095 "2018-02-01T19:52:09Z")
Hi all, I am looking for a recommendation for my current situation. In our network, Ip's change pretty frequently when machines reboot. Before going forward, yes I'd love to set the IP statically to our Logstash server,… --- ## [Logstash didn't send data to output](https://discuss.elastic.co/t/logstash-didnt-send-data-to-output/118093)
**Author:** [@tracywan](https://discuss.elastic.co/u/tracywan)\ **Replies:** 0\ **Last updated:** [February 1, 2018, 7:35pm UTC](https://discuss.elastic.co/t/logstash-didnt-send-data-to-output/118093 "2018-02-01T19:35:22Z")
I confirmed that data has sent from filebeat client to logstash successfully from following logs output received {:event=\>{"message"=\>"2018-01-31T15:45:22-08:00 INFO Home path: \[/Users/tracywan/work/BoxRelay/filebeat-6.… --- ## [Cannot load Java class org.logstash.beats.Server](https://discuss.elastic.co/t/cannot-load-java-class-org-logstash-beats-server/118082)
**Author:** [@seth.yes](https://discuss.elastic.co/u/seth.yes)\ **Replies:** 1\ **Last updated:** [February 1, 2018, 6:12pm UTC](https://discuss.elastic.co/t/cannot-load-java-class-org-logstash-beats-server/118082 "2018-02-01T18:12:05Z")
Logstash version: 5.6.7, Java version: 1.8.0\_161 After no notable changes, Logstash was missing logs a couple mornings ago. In making some changes to the pipeline config, I restarted the service and it did not come bac… --- ## [Pipelines.yml still ignored](https://discuss.elastic.co/t/pipelines-yml-still-ignored/117697)
**Author:** [@jamest23](https://discuss.elastic.co/u/jamest23)\ **Replies:** 3\ **Last updated:** [February 1, 2018, 4:46pm UTC](https://discuss.elastic.co/t/pipelines-yml-still-ignored/117697 "2018-02-01T16:46:09Z")
Greetings, I have populated the pipelines.yml file but it is not getting picked up. The goal being to separate the 2 pipelines i have running. When i put them both in the /etc/logstash/conf.d they collide and logstash… --- ## [Developing a filter plugin which depends on ruby gem](https://discuss.elastic.co/t/developing-a-filter-plugin-which-depends-on-ruby-gem/118075)
**Author:** [@paulborile](https://discuss.elastic.co/u/paulborile)\ **Replies:** 0\ **Last updated:** [February 1, 2018, 4:36pm UTC](https://discuss.elastic.co/t/developing-a-filter-plugin-which-depends-on-ruby-gem/118075 "2018-02-01T16:36:55Z")
Hi, I'm developing a filter plugin which uses an external ruby gem (that requires ffi). The plugin compiles and tests ok (with bundle exec rspec) but when I run : bin/logstash-plugin install logstash-filter-wurfldevice… --- ## [Logstash 6.1 failing to parse CSV](https://discuss.elastic.co/t/logstash-6-1-failing-to-parse-csv/115485)
**Author:** [@Saket\_Kumar](https://discuss.elastic.co/u/Saket_Kumar)\ **Replies:** 6\ **Last updated:** [February 1, 2018, 3:40pm UTC](https://discuss.elastic.co/t/logstash-6-1-failing-to-parse-csv/115485 "2018-02-01T15:40:48Z")
While parsing csv file i am getting below error. Any guess or help to remove this? \[2018-01-15T12:33:39,047\]\[WARN \]\[logstash.filters.csv \] Error parsing csv {:field=\>"message", :source=\>"\\"ip\_addr\\",\\"initiator\_line\\",\\… --- ## [Connecting to SQL from Ubuntu VMPlayer](https://discuss.elastic.co/t/connecting-to-sql-from-ubuntu-vmplayer/117849)
**Author:** [@Hassan\_Shaitou](https://discuss.elastic.co/u/Hassan_Shaitou)\ **Replies:** 3\ **Last updated:** [February 1, 2018, 3:32pm UTC](https://discuss.elastic.co/t/connecting-to-sql-from-ubuntu-vmplayer/117849 "2018-02-01T15:32:32Z")
I am trying to create a conf file for logstash on my Ubuntu hosted on VMWare Player, to be able to connect to my remote SQL Server with this IP:192.168.237.1,49170. 1- I tried to ping the IP, and it's ok.. 2- I tried t… --- ## [Logstash With RabbitMQ Never Finish Process](https://discuss.elastic.co/t/logstash-with-rabbitmq-never-finish-process/117696)
**Author:** [@mpinheiro](https://discuss.elastic.co/u/mpinheiro)\ **Replies:** 4\ **Last updated:** [February 1, 2018, 1:29pm UTC](https://discuss.elastic.co/t/logstash-with-rabbitmq-never-finish-process/117696 "2018-02-01T13:29:46Z")
Hi everyone, I've a problem when i'm running my logstash config with the rabbitmq input plugin, the process never ends, it keeps running forever. I would like to finish logstash process when all my queue documents was pr… --- ## [Logstash sending data via TCP(Output Plugin) taking a lot of time](https://discuss.elastic.co/t/logstash-sending-data-via-tcp-output-plugin-taking-a-lot-of-time/118004)
**Author:** [@bioraj](https://discuss.elastic.co/u/bioraj)\ **Replies:** 1\ **Last updated:** [February 1, 2018, 12:39pm UTC](https://discuss.elastic.co/t/logstash-sending-data-via-tcp-output-plugin-taking-a-lot-of-time/118004 "2018-02-01T12:39:26Z")
I have a Windows service that is listening on TCP port , and ELK installed on Linux server I am sending data from logstash via TCP to windows server , It's working fine but taking a lot of time and slow down logstash/el… --- ## [Grok Pattern miss match](https://discuss.elastic.co/t/grok-pattern-miss-match/117992)
**Author:** [@Vigneshprasanna](https://discuss.elastic.co/u/Vigneshprasanna)\ **Replies:** 4\ **Last updated:** [February 1, 2018, 12:37pm UTC](https://discuss.elastic.co/t/grok-pattern-miss-match/117992 "2018-02-01T12:37:46Z")
Hi, For the following message i using the specified grok pattern but in the output i get grok faild i tried using the grok constructed, which is not favoring me In Message : \<2018-01-31 00:00:11\>,\<896\> \< INFO \> \< \[STD… --- ## [Logstash behaviour](https://discuss.elastic.co/t/logstash-behaviour/116912)
**Author:** [@JAYALAKSHMI](https://discuss.elastic.co/u/JAYALAKSHMI)\ **Replies:** 2\ **Last updated:** [February 1, 2018, 11:06am UTC](https://discuss.elastic.co/t/logstash-behaviour/116912 "2018-02-01T11:06:37Z")
I am loading the data into ES Cluster through logstash. If I Provide the path i.e)source path as the mentioned below. Path: /Folder/Folder1/\* How will Logstash behave if there are 5 files inside the folder1? Which one… --- ## [Logstash datetime timezone](https://discuss.elastic.co/t/logstash-datetime-timezone/117981)
**Author:** [@Anuar\_Mukatov](https://discuss.elastic.co/u/Anuar_Mukatov)\ **Replies:** 2\ **Last updated:** [February 1, 2018, 10:29am UTC](https://discuss.elastic.co/t/logstash-datetime-timezone/117981 "2018-02-01T10:29:32Z")
Hello! How i can change datetime timezone? What i else not to tr, anything didn't help me. input { jdbc { jdbc\_driver\_library =\> "/etc/logstash/drivers/sqljdbc42.jar" jdbc\_driver\_class =\> "com.microsoft.sqlserver.… --- ## [Logstash 5.5 unable to locate config file in windows 10](https://discuss.elastic.co/t/logstash-5-5-unable-to-locate-config-file-in-windows-10/116655)
**Author:** [@manisha\_2018](https://discuss.elastic.co/u/manisha_2018)\ **Replies:** 4\ **Last updated:** [February 1, 2018, 9:54am UTC](https://discuss.elastic.co/t/logstash-5-5-unable-to-locate-config-file-in-windows-10/116655 "2018-02-01T09:54:21Z")
Hi, I am using logstash-5.5.0 in Windows 10. Its unable to find config File which I have kept in logstash folder itself. Below is the error which I am getting when I do "logstash -f logstash.conf": ERROR StatusLogger… --- ## [Not able to grok filebeat's mysql logs output](https://discuss.elastic.co/t/not-able-to-grok-filebeats-mysql-logs-output/117955)
**Author:** [@rajya\_vardhan\_Mishra](https://discuss.elastic.co/u/rajya_vardhan_Mishra)\ **Replies:** 1\ **Last updated:** [February 1, 2018, 9:52am UTC](https://discuss.elastic.co/t/not-able-to-grok-filebeats-mysql-logs-output/117955 "2018-02-01T09:52:15Z")
Hi, I have been trying to grok filebeat messages(in json format), so that i can create kibana dashboard. But not able to crack this , even after spending couple of days! Step 1: My filebeat plugin puts messages in a l… --- ## [Grok filter (2018-02-01T09:33:02.000000+01:00)](https://discuss.elastic.co/t/grok-filter-2018-02-01t09-33-02-000000-01-00/117971)
**Author:** [@cdra](https://discuss.elastic.co/u/cdra)\ **Replies:** 1\ **Last updated:** [February 1, 2018, 8:55am UTC](https://discuss.elastic.co/t/grok-filter-2018-02-01t09-33-02-000000-01-00/117971 "2018-02-01T08:55:24Z")
I need a filter to this message.. \<6\>1 2018-02-01T09:33:02.000000+01:00 MSA1040\_CPD1 APPSV 2238 - \[details@347 ust="116444772000000000" lst="116444736000000000" ip="10.61.99.45" ssn="B9999" eventPriority="Informational"… --- ## [Ruby exception occurred: Detected invalid array contents due to unsynchronized modifications with concurrent users](https://discuss.elastic.co/t/ruby-exception-occurred-detected-invalid-array-contents-due-to-unsynchronized-modifications-with-concurrent-users/117972)
**Author:** [@madhur\_yadav](https://discuss.elastic.co/u/madhur_yadav)\ **Replies:** 0\ **Last updated:** [February 1, 2018, 8:52am UTC](https://discuss.elastic.co/t/ruby-exception-occurred-detected-invalid-array-contents-due-to-unsynchronized-modifications-with-concurrent-users/117972 "2018-02-01T08:52:31Z")
I use Elasticsearch 5.6.2 ; logstash 5.6.2 ; filebeat 5.6.2 This is how my filebeat looks like- filebeat.prospectors: - input\_type: log paths: - C:\\Users\\madhur\_yadav\\Documents\\My Received Files\\project177\\ss\_… --- ## [Remove specific part of url from logstash after match](https://discuss.elastic.co/t/remove-specific-part-of-url-from-logstash-after-match/117948)
**Author:** [@RamyaGowda](https://discuss.elastic.co/u/RamyaGowda)\ **Replies:** 1\ **Last updated:** [February 1, 2018, 7:30am UTC](https://discuss.elastic.co/t/remove-specific-part-of-url-from-logstash-after-match/117948 "2018-02-01T07:30:34Z")
Hi I want to remove a specific part from url , for example i have /documents/del/123456 /documents/del/15689 /documents/del/580867 like this , and i want to remove the number's from url so that it will be treated a… --- ## [How to trim zero or more leading and trailing spaces from key-value pairs?](https://discuss.elastic.co/t/how-to-trim-zero-or-more-leading-and-trailing-spaces-from-key-value-pairs/117933)
**Author:** [@jainbhavya53](https://discuss.elastic.co/u/jainbhavya53)\ **Replies:** 7\ **Last updated:** [February 1, 2018, 6:51am UTC](https://discuss.elastic.co/t/how-to-trim-zero-or-more-leading-and-trailing-spaces-from-key-value-pairs/117933 "2018-02-01T06:51:22Z")
I have used kv filter to extract key-value pairs using field\_split and have used default\_keys to provide default keys in case no value is present corresponding to that field.I have used strip of mutate filter to trim all… --- ## [Nmap Codec Not Indexing Hostname](https://discuss.elastic.co/t/nmap-codec-not-indexing-hostname/117947)
**Author:** [@forewarned](https://discuss.elastic.co/u/forewarned)\ **Replies:** 0\ **Last updated:** [February 1, 2018, 7:04am UTC](https://discuss.elastic.co/t/nmap-codec-not-indexing-hostname/117947 "2018-02-01T07:04:01Z")
When using the Nmap Logstash Codec, hostnames and their dns record type are not getting indexed into Elasticsearch, while the other Nmap data is. I'm following the Elastic blog on how to use the codec., and my Nmap data … --- ## [Alter filed in logstash filter for Metricbeat](https://discuss.elastic.co/t/alter-filed-in-logstash-filter-for-metricbeat/117935)
**Author:** [@tharu85](https://discuss.elastic.co/u/tharu85)\ **Replies:** 1\ **Last updated:** [February 1, 2018, 7:03am UTC](https://discuss.elastic.co/t/alter-filed-in-logstash-filter-for-metricbeat/117935 "2018-02-01T07:03:05Z")
I have configure Metricbeat and send system metric to logstash. Sample metricbeat output as below { "@timestamp" =\> 2018-02-01T05:36:14.330Z, "system" =\> { "network" =\> { "in" =\> { "bytes… --- ## [ERROR: Installation aborted, verification failed for --version](https://discuss.elastic.co/t/error-installation-aborted-verification-failed-for-version/117765)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 3\ **Last updated:** [February 1, 2018, 6:56am UTC](https://discuss.elastic.co/t/error-installation-aborted-verification-failed-for-version/117765 "2018-02-01T06:56:50Z")
Hi, As per the subject line, how do you install a older version of a plugin? In my case I tried "--version 3.5.1", "--version v3.5.1" and some other variations but they all fail. /usr/share/logstash# bin/logstash-plugi… --- ## [Unknown Setting in config for Custom JDBC Plugin](https://discuss.elastic.co/t/unknown-setting-in-config-for-custom-jdbc-plugin/117945)
**Author:** [@Kareny](https://discuss.elastic.co/u/Kareny)\ **Replies:** 0\ **Last updated:** [February 1, 2018, 6:52am UTC](https://discuss.elastic.co/t/unknown-setting-in-config-for-custom-jdbc-plugin/117945 "2018-02-01T06:52:12Z")
I was trying to customize the jdbc plugin and add a new config field of "jdbc\_user\_id". But after I installed the plugin onto logstash, logstash failed to start with error: \[logstash.inputs.jdbc \] Unknown setting 'j… --- ## [Custom grok pattern failure with forwardslashes](https://discuss.elastic.co/t/custom-grok-pattern-failure-with-forwardslashes/115409)
**Author:** [@prophoto](https://discuss.elastic.co/u/prophoto)\ **Replies:** 4\ **Last updated:** [January 31, 2018, 10:59pm UTC](https://discuss.elastic.co/t/custom-grok-pattern-failure-with-forwardslashes/115409 "2018-01-31T22:59:28Z")
About 1/400 log entries I'm feeding logstash will have a forwardslash "/" in it. How do I account for that and prevent a grok failure? --- ## [There's no "OUT\_BYTES" field when I parse netflow through logstash](https://discuss.elastic.co/t/theres-no-out-bytes-field-when-i-parse-netflow-through-logstash/117777)
**Author:** [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Replies:** 2\ **Last updated:** [January 31, 2018, 5:15pm UTC](https://discuss.elastic.co/t/theres-no-out-bytes-field-when-i-parse-netflow-through-logstash/117777 "2018-01-31T17:15:18Z")
As title, when I use logstash to parse netflow and ingest flow data into elasticsearch, I found the data in ES is lack of "OUT\_BYTES" field. this is the data in ES: { "\_index": "netflow-%{year}-%{month}-%{day}", "\_… --- ## ['path.config' (-f) and 'config.string' (-e) can't be used simultaneously](https://discuss.elastic.co/t/path-config-f-and-config-string-e-cant-be-used-simultaneously/117848)
**Author:** [@ahtelzhang](https://discuss.elastic.co/u/ahtelzhang)\ **Replies:** 0\ **Last updated:** [January 31, 2018, 3:35pm UTC](https://discuss.elastic.co/t/path-config-f-and-config-string-e-cant-be-used-simultaneously/117848 "2018-01-31T15:35:52Z")
Hi, when I execute "bin/logstash -e 'input { stdin { } } output { stdout {} }' , but have error: \[root@logstash-1 logstash\]# bin/logstash -e 'input { stdin { } } output { stdout {} }' Sending Logstash's logs to /var/l… --- ## [How do I index regular expressions?](https://discuss.elastic.co/t/how-do-i-index-regular-expressions/117826)
**Author:** [@Joshua\_Fox](https://discuss.elastic.co/u/Joshua_Fox)\ **Replies:** 1\ **Last updated:** [January 31, 2018, 2:30pm UTC](https://discuss.elastic.co/t/how-do-i-index-regular-expressions/117826 "2018-01-31T14:30:21Z")
For log management, I want to build Kibana reports based on log lines like this: 2018-01-31 11:50:00.212 Loading the user images took 234 ms 2018-01-31 10:23:01.984 Loading the user images took 331 ms 2018-01-31 10:1… --- ## [Regx or plugin suggestion needed](https://discuss.elastic.co/t/regx-or-plugin-suggestion-needed/117827)
**Author:** [@tjayantha](https://discuss.elastic.co/u/tjayantha)\ **Replies:** 0\ **Last updated:** [January 31, 2018, 2:25pm UTC](https://discuss.elastic.co/t/regx-or-plugin-suggestion-needed/117827 "2018-01-31T14:25:11Z")
I'm new bee to ELK This is my sshd\_log 2018-01-31 00:16:55,831|INFO |sshd-SshServer\[e4afa08\]-nio2-thread-12|ServerSession | Server session created from /10.52.3.29:43662 2018-01-31 00:16:58,122|INFO |sshd-SshServer\[e… --- ## [Logstash multipipline to replace shipper-redis-indexer?](https://discuss.elastic.co/t/logstash-multipipline-to-replace-shipper-redis-indexer/117787)
**Author:** [@widhalmt](https://discuss.elastic.co/u/widhalmt)\ **Replies:** 4\ **Last updated:** [January 31, 2018, 2:06pm UTC](https://discuss.elastic.co/t/logstash-multipipline-to-replace-shipper-redis-indexer/117787 "2018-01-31T14:06:00Z")
Hi, I have a setup with lots of SNMP Traps coming in which I started with Logstash 1.5 . So I set up a "Logstash Shipper" (just inputs and one output to two Redis servers, no filters) and "Logstash Indexer" another inst… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=829) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=831)