# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=894 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 895 --- ## [Undefined method \`sanitized'](https://discuss.elastic.co/t/undefined-method-sanitized/100588)
**Author:** [@feijiangnan](https://discuss.elastic.co/u/feijiangnan)\ **Replies:** 7\ **Last updated:** [October 11, 2017, 7:27am UTC](https://discuss.elastic.co/t/undefined-method-sanitized/100588 "2017-10-11T07:27:38Z")
Hi there, I upgraded ELK from 5.5. to 5.6, and logstash received the log, parsed, but didn't write to elasticsearch. From log, it complains ""undefined method \`sanitized' ", logstash-output-elasticsearch version is 7.4… --- ## [Code within the logstash-filter-aggregate plugin aggregate block](https://discuss.elastic.co/t/code-within-the-logstash-filter-aggregate-plugin-aggregate-block/103470)
**Author:** [@Mohammed\_Sanaulla](https://discuss.elastic.co/u/Mohammed_Sanaulla)\ **Replies:** 0\ **Last updated:** [October 11, 2017, 5:42am UTC](https://discuss.elastic.co/t/code-within-the-logstash-filter-aggregate-plugin-aggregate-block/103470 "2017-10-11T05:42:16Z")
Where can we find the syntax of the code which is written against the code attribute within: filter { aggregate { task\_id =\> "%{doc\_record\_id}" code =\> "" } } --- ## [Configure JDBC input with SSL connection for MySQL](https://discuss.elastic.co/t/configure-jdbc-input-with-ssl-connection-for-mysql/103069)
**Author:** [@Devang\_Gaglani](https://discuss.elastic.co/u/Devang_Gaglani)\ **Replies:** 5\ **Last updated:** [October 11, 2017, 5:13am UTC](https://discuss.elastic.co/t/configure-jdbc-input-with-ssl-connection-for-mysql/103069 "2017-10-11T05:13:48Z")
I want to connect to MySQL database for collecting logs in logstash. But database is having secured connection. How do I set SSL in JDBC input plugin of logstash for MySQL database? --- ## [Split long log into small parts and apply grok in logstash filter](https://discuss.elastic.co/t/split-long-log-into-small-parts-and-apply-grok-in-logstash-filter/103363)
**Author:** [@tharu85](https://discuss.elastic.co/u/tharu85)\ **Replies:** 2\ **Last updated:** [October 11, 2017, 4:05am UTC](https://discuss.elastic.co/t/split-long-log-into-small-parts-and-apply-grok-in-logstash-filter/103363 "2017-10-11T04:05:48Z")
I have a log which cannot directly extract using grok patterns under logstash filter. I have tested full grok pattern for the log in grok debugger and it works fine. But when I run the grok pattern under logstash filter … --- ## [Convert unix to standard date/time with milliseconds](https://discuss.elastic.co/t/convert-unix-to-standard-date-time-with-milliseconds/103411)
**Author:** [@tgdesrochers](https://discuss.elastic.co/u/tgdesrochers)\ **Replies:** 2\ **Last updated:** [October 10, 2017, 10:10pm UTC](https://discuss.elastic.co/t/convert-unix-to-standard-date-time-with-milliseconds/103411 "2017-10-10T22:10:07Z")
Elasticsearch 5.6 Logstash 5.6 I have a log that is created with the below field: unixTime: 1507655093 I am using the below logstash config to parse the date into the @timestamp field: date { match =\> \["unixTime", "… --- ## [Logstash 5.6.3: TCP input - sslsubject no longer getting set](https://discuss.elastic.co/t/logstash-5-6-3-tcp-input-sslsubject-no-longer-getting-set/103423)
**Author:** [@utilitynerd](https://discuss.elastic.co/u/utilitynerd)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 10:02pm UTC](https://discuss.elastic.co/t/logstash-5-6-3-tcp-input-sslsubject-no-longer-getting-set/103423 "2017-10-10T22:02:52Z")
I've run into an issue after upgrading to logstash 5.6.3 It seems that my TCP input, configured for TLS, is no longer setting the sslsubject field. If i downgrade back to 5.5.2, then sslsubject get correctly set again. … --- ## [Kibana isn't reciving Logstash + Elasticsearch logs](https://discuss.elastic.co/t/kibana-isnt-reciving-logstash-elasticsearch-logs/103217)
**Author:** [@Mickey](https://discuss.elastic.co/u/Mickey)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 9:59pm UTC](https://discuss.elastic.co/t/kibana-isnt-reciving-logstash-elasticsearch-logs/103217 "2017-10-10T21:59:15Z")
i am trying to monitor some servers in my small-mid network using ELK. my configuration file: Blockquote input{ file{ path =\> "/opt/experis-cyber/sys-monitor/log/monitor-sys-1507188302096.json" start\_position =\>"… --- ## [Logstash Encountered an unexpected error submitting a bulk request! "undefined method \`sanitized'](https://discuss.elastic.co/t/logstash-encountered-an-unexpected-error-submitting-a-bulk-request-undefined-method-sanitized/103360)
**Author:** [@Orsius](https://discuss.elastic.co/u/Orsius)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 9:55pm UTC](https://discuss.elastic.co/t/logstash-encountered-an-unexpected-error-submitting-a-bulk-request-undefined-method-sanitized/103360 "2017-10-10T21:55:56Z")
Dear all, I would like to change my actual flow: FROM metricbeat \> elasticsearch TO metricbeat \> logstash \> elasticsearch The following configuration works on my laptop but not on a linux production machine :confu… --- ## [Elastic Stack as a syslog relay?](https://discuss.elastic.co/t/elastic-stack-as-a-syslog-relay/103030)
**Author:** [@s.essa](https://discuss.elastic.co/u/s.essa)\ **Replies:** 3\ **Last updated:** [October 10, 2017, 9:18pm UTC](https://discuss.elastic.co/t/elastic-stack-as-a-syslog-relay/103030 "2017-10-10T21:18:33Z")
Hello, I am completely new to the Elastic Stack and was wondering if the following was possible: I want to use the Elastic stack for log collection, forensics and as a relay to LogRhythm for specific use cases. I was w… --- ## [New to elk and need assistance with a ruby re-write](https://discuss.elastic.co/t/new-to-elk-and-need-assistance-with-a-ruby-re-write/103433)
**Author:** [@tallan](https://discuss.elastic.co/u/tallan)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 8:36pm UTC](https://discuss.elastic.co/t/new-to-elk-and-need-assistance-with-a-ruby-re-write/103433 "2017-10-10T20:36:19Z")
Hello, I am pretty new to logstash but am getting the hang of it. I am a total noob with ruby however but I know I am close and just need a little advice from someone. I can't seem to get my code to drop the decimals an… --- ## [How can we store input\>file\>csv filename into log file?](https://discuss.elastic.co/t/how-can-we-store-input-file-csv-filename-into-log-file/103432)
**Author:** [@surajbs](https://discuss.elastic.co/u/surajbs)\ **Replies:** 0\ **Last updated:** [October 10, 2017, 7:58pm UTC](https://discuss.elastic.co/t/how-can-we-store-input-file-csv-filename-into-log-file/103432 "2017-10-10T19:58:33Z")
Hi, How can we write or store input filename into log file? Below is my config for the input: input { file { path =\> "path to file/\*.csv" type =\> "csv" start\_position =\> beginning } file { path =\> "path to log/… --- ## [Is there any example of logstash-filter with loading \*.jar](https://discuss.elastic.co/t/is-there-any-example-of-logstash-filter-with-loading-jar/103424)
**Author:** [@kmaci3k](https://discuss.elastic.co/u/kmaci3k)\ **Replies:** 0\ **Last updated:** [October 10, 2017, 7:14pm UTC](https://discuss.elastic.co/t/is-there-any-example-of-logstash-filter-with-loading-jar/103424 "2017-10-10T19:14:02Z")
I'm trying to use in my custom logstash filter a Java code bundled in .jar. I tried to play with lock\_jar which downloads all dependencies of .jar. The .jar and all its dependencies are then added to lib/java and bundled… --- ## [How can I hide the ssl\_key\_passphrase?](https://discuss.elastic.co/t/how-can-i-hide-the-ssl-key-passphrase/103214)
**Author:** [@Ihjaz](https://discuss.elastic.co/u/Ihjaz)\ **Replies:** 2\ **Last updated:** [October 10, 2017, 7:10pm UTC](https://discuss.elastic.co/t/how-can-i-hide-the-ssl-key-passphrase/103214 "2017-10-10T19:10:06Z")
Hi All, I have the following configuration input { beats { host =\> "127.0.0.1" port =\> 5044 ssl =\> true ssl\_certificate\_authorities =\> \["/etc/logstash/certs/trust.pem"\] ssl\_certificate =\> "/etc/lo… --- ## [Date parsing \[5.6.2\]](https://discuss.elastic.co/t/date-parsing-5-6-2/103368)
**Author:** [@cperriot](https://discuss.elastic.co/u/cperriot)\ **Replies:** 3\ **Last updated:** [October 10, 2017, 7:04pm UTC](https://discuss.elastic.co/t/date-parsing-5-6-2/103368 "2017-10-10T19:04:00Z")
trying to parse: 2016-03-27 01:45:00 2016-02-20 02:15:00 2017-10-10 09:57:46 with the following config: input { file { sincedb\_path=\>"sincedb" path =\> "/xxxxxxx/export\_test.txt" start\_position =\> "beginning" } … --- ## [How monitor whether filebeat is receiving log from specified path?](https://discuss.elastic.co/t/how-monitor-whether-filebeat-is-receiving-log-from-specified-path/103106)
**Author:** [@armita\_88](https://discuss.elastic.co/u/armita_88)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 6:32pm UTC](https://discuss.elastic.co/t/how-monitor-whether-filebeat-is-receiving-log-from-specified-path/103106 "2017-10-10T18:32:03Z")
hello I have installed logstash on a server for gathering logs from a number of client servers by installing filebeat on all of client servers and every thing works well. now I need to receive alarm if each of client se… --- ## [Found more number of data columns - after passing through logstash and visualizing in kibana](https://discuss.elastic.co/t/found-more-number-of-data-columns-after-passing-through-logstash-and-visualizing-in-kibana/103104)
**Author:** [@Gurram\_Vinay](https://discuss.elastic.co/u/Gurram_Vinay)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 6:30pm UTC](https://discuss.elastic.co/t/found-more-number-of-data-columns-after-passing-through-logstash-and-visualizing-in-kibana/103104 "2017-10-10T18:30:47Z")
Hello guys, I have sent data(416 columns of data ) through logstash and visualizing with kibana. I found more number of data rather than existed(I was watching 423, but initially I had sent only 416). Could any one plz … --- ## [Logstash grok matches on two different fields log\_type and message](https://discuss.elastic.co/t/logstash-grok-matches-on-two-different-fields-log-type-and-message/103077)
**Author:** [@ajstark123](https://discuss.elastic.co/u/ajstark123)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 6:29pm UTC](https://discuss.elastic.co/t/logstash-grok-matches-on-two-different-fields-log-type-and-message/103077 "2017-10-10T18:29:24Z")
We are using file beat to read multiple logs and we are send the output to logstash. We want a logstash grok filter to activate when (log\_type == syslog) and (matches on syslog message) grok { match =\> {"log\_type"… --- ## [Logstash 5.6.2 aborts with “warning: ignoring JAVA\_OPTS=-Xmx1024m; pass JVM parameters via LS\_JAVA\_OPTS”](https://discuss.elastic.co/t/logstash-5-6-2-aborts-with-warning-ignoring-java-opts-xmx1024m-pass-jvm-parameters-via-ls-java-opts/102753)
**Author:** [@srini1](https://discuss.elastic.co/u/srini1)\ **Replies:** 2\ **Last updated:** [October 10, 2017, 3:16pm UTC](https://discuss.elastic.co/t/logstash-5-6-2-aborts-with-warning-ignoring-java-opts-xmx1024m-pass-jvm-parameters-via-ls-java-opts/102753 "2017-10-10T15:16:06Z")
I just downloaded latest versions of ELK 5.6.2 to upgrade so that I can use X-pack 5.6.2. Elasticsearch and kibana started with no issues but Logstash is aborting with the below message. "warning: ignoring JAVA\_OPTS=-Xmx… --- ## [geoIP failure](https://discuss.elastic.co/t/geoip-failure/102487)
**Author:** [@Jose\_Campos](https://discuss.elastic.co/u/Jose_Campos)\ **Replies:** 2\ **Last updated:** [October 10, 2017, 3:06pm UTC](https://discuss.elastic.co/t/geoip-failure/102487 "2017-10-10T15:06:10Z")
Hello A few days ago, I updated my elastic cloud to version 5.5.3, and started to give me this error. "error" =\> {"type" =\> "mapper\_parsing\_exception", "reason" =\> "failed to parse", "caused\_by" =\> {"type" =\> "illegal\_… --- ## [S3 bucket data not processing correctly](https://discuss.elastic.co/t/s3-bucket-data-not-processing-correctly/103356)
**Author:** [@Sunil\_M](https://discuss.elastic.co/u/Sunil_M)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 3:03pm UTC](https://discuss.elastic.co/t/s3-bucket-data-not-processing-correctly/103356 "2017-10-10T15:03:00Z")
Hi All, I have logstash code which reads the S3 bucket data. The data in the bucket was like {"Request":{"test":"BLABLA"},"timestamp":"1503404629272"}} {"Request":{"test":"BLABLA"},"timestamp":"1503404629272"}} I us… --- ## [Grok a custom log](https://discuss.elastic.co/t/grok-a-custom-log/102848)
**Author:** [@RichardLaing](https://discuss.elastic.co/u/RichardLaing)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 2:12pm UTC](https://discuss.elastic.co/t/grok-a-custom-log/102848 "2017-10-10T14:12:17Z")
Hi There I am looking at how to grok a custom log type. At present that is gathering logs in the syslog format with a couple of extra labels. The following is a log sample. 2017-10-05T14:21:58+01:00 192.168.60.13 rando… --- ## [An unexpected error occurred! {:error=\>](https://discuss.elastic.co/t/an-unexpected-error-occurred-error/103382)
**Author:** [@mbvelo](https://discuss.elastic.co/u/mbvelo)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 2:02pm UTC](https://discuss.elastic.co/t/an-unexpected-error-occurred-error/103382 "2017-10-10T14:02:32Z")
An unexpected error occurred! {:error=\> #\, :backtrace=\>\["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:443:in… --- ## [Flatten json array to combine fields (right ruby code?))](https://discuss.elastic.co/t/flatten-json-array-to-combine-fields-right-ruby-code/103235)
**Author:** [@malhotras](https://discuss.elastic.co/u/malhotras)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 1:54pm UTC](https://discuss.elastic.co/t/flatten-json-array-to-combine-fields-right-ruby-code/103235 "2017-10-10T13:54:12Z")
Here is how my input data looks like: { "details": \[ { "place": "abc", "group": 3, "year": 2006, "id": 1304, "street": "xyz 14", "lf\_number": "0118", "code": 4433, … --- ## [Csv filtering misses 2 first columns](https://discuss.elastic.co/t/csv-filtering-misses-2-first-columns/103367)
**Author:** [@f1ccb489d33b33a0f12a](https://discuss.elastic.co/u/f1ccb489d33b33a0f12a)\ **Replies:** 0\ **Last updated:** [October 10, 2017, 1:04pm UTC](https://discuss.elastic.co/t/csv-filtering-misses-2-first-columns/103367 "2017-10-10T13:04:38Z")
i'm having a csv file with n columns in the format of: 1Col,2Col,.........,n-1Col, nCol all my data is parsed fine with the right headers names. all except the 2 first columns. the 2 first columns are shown in kibana… --- ## [Try to use aggregate filter](https://discuss.elastic.co/t/try-to-use-aggregate-filter/100078)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 9\ **Last updated:** [October 10, 2017, 11:55am UTC](https://discuss.elastic.co/t/try-to-use-aggregate-filter/100078 "2017-10-10T11:55:40Z")
I am trying to use the aggregate filter and that what I am getting in the log: \[2017-09-11T17:53:35,256\]\[ERROR\]\[logstash.plugins.registry\] Problems loading a plugin with {:type=\>"filter", :name=\>"aggregate", :path=\>"log… --- ## [How to configure logstash in our machine without affecting performance?](https://discuss.elastic.co/t/how-to-configure-logstash-in-our-machine-without-affecting-performance/103339)
**Author:** [@LTAravindh](https://discuss.elastic.co/u/LTAravindh)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 11:44am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-in-our-machine-without-affecting-performance/103339 "2017-10-10T11:44:04Z")
I need to use logstash to read some log information from my own system. I'm using Elasticsearch as a output of the logstash. Elastic search will be configured in another server. I read some articles and discussion forum… --- ## [Strict schema when using CSV plugin](https://discuss.elastic.co/t/strict-schema-when-using-csv-plugin/103358)
**Author:** [@Israel\_Elmekies](https://discuss.elastic.co/u/Israel_Elmekies)\ **Replies:** 0\ **Last updated:** [October 10, 2017, 11:41am UTC](https://discuss.elastic.co/t/strict-schema-when-using-csv-plugin/103358 "2017-10-10T11:41:47Z")
Hi All, I'm relatively new to Logstash. I'm using the CSV filter plugin to read a csv log. I wanted to add the option that if the line that is being read does not have the same schema as defined, then an error will be… --- ## [Restarting logstash inside docker](https://discuss.elastic.co/t/restarting-logstash-inside-docker/103180)
**Author:** [@Zama\_Las](https://discuss.elastic.co/u/Zama_Las)\ **Replies:** 7\ **Last updated:** [October 10, 2017, 11:40am UTC](https://discuss.elastic.co/t/restarting-logstash-inside-docker/103180 "2017-10-10T11:40:47Z")
How to restart logstash process inside docker ? Right now I am restarting the container containing logstash but that does not look to be the ideal process to do it. docker restart container-id --- ## [Sizing](https://discuss.elastic.co/t/sizing/103337)
**Author:** [@omprakasha](https://discuss.elastic.co/u/omprakasha)\ **Replies:** 0\ **Last updated:** [October 10, 2017, 9:50am UTC](https://discuss.elastic.co/t/sizing/103337 "2017-10-10T09:50:35Z")
What should be bare minimum RAM and Hard disk requirement while setting up Elastic components in testing or production environments. We are using Filebeat, Logstash and Elasticsearch and Kibana components. Daily log siz… --- ## [Extract domain with grok](https://discuss.elastic.co/t/extract-domain-with-grok/103156)
**Author:** [@Shoham\_Ben\_Shitrit](https://discuss.elastic.co/u/Shoham_Ben_Shitrit)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 8:54am UTC](https://discuss.elastic.co/t/extract-domain-with-grok/103156 "2017-10-10T08:54:46Z")
I'd like to make the simplest grok filter, just to extract domain from url For example, for the url https://discuss.elastic.co/top/all I'd like to get the result discuss.elastic.co I tried to do so with the filter %… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=893) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=895)