# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=237 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 238 --- ## [S3SNSSQS Plugin input](https://discuss.elastic.co/t/s3snssqs-plugin-input/270696)
**Author:** [@mjuras](https://discuss.elastic.co/u/mjuras)\ **Replies:** 0\ **Last updated:** [April 20, 2021, 11:38am UTC](https://discuss.elastic.co/t/s3snssqs-plugin-input/270696 "2021-04-20T11:38:31Z")
Hi I've switched the input for my logs to use the logstash-input-s3-sns-sqs plugin instead of the S3 plugin, but seem to be having an issue with the input with one of the logs. Input: input { s3snssqs { reg… --- ## [Processor for windows logs](https://discuss.elastic.co/t/processor-for-windows-logs/270695)
**Author:** [@dddddddddddddddd](https://discuss.elastic.co/u/dddddddddddddddd)\ **Replies:** 0\ **Last updated:** [April 20, 2021, 11:37am UTC](https://discuss.elastic.co/t/processor-for-windows-logs/270695 "2021-04-20T11:37:38Z")
hi there if somebody can help me to find a filter for windows logs with ECS 1.7.0 thanks --- ## [Search and filter part of message](https://discuss.elastic.co/t/search-and-filter-part-of-message/270660)
**Author:** [@ELK\_gj](https://discuss.elastic.co/u/ELK_gj)\ **Replies:** 0\ **Last updated:** [April 20, 2021, 6:55am UTC](https://discuss.elastic.co/t/search-and-filter-part-of-message/270660 "2021-04-20T06:55:24Z")
Hi, I've two separate strings like NAS-EM ATTACH\_REQUEST (0x41) to be searched in the log file and filter all the messages which has the above strings in the same line and send that data to kibana. Can you please hel… --- ## [Getting Error on logstash log file](https://discuss.elastic.co/t/getting-error-on-logstash-log-file/270482)
**Author:** [@varun1992](https://discuss.elastic.co/u/varun1992)\ **Replies:** 1\ **Last updated:** [April 20, 2021, 5:30am UTC](https://discuss.elastic.co/t/getting-error-on-logstash-log-file/270482 "2021-04-20T05:30:01Z")
\[2021-04-17T21:44:28,123\]\[ERROR\]\[logstash.javapipeline \] A plugin had an unrecoverable error. Will restart this plugin. Pipeline\_id:group\_index Plugin: \"beginning", … --- ## [If Statements Logstash Pipeline](https://discuss.elastic.co/t/if-statements-logstash-pipeline/270637)
**Author:** [@caitlyn.carlisle](https://discuss.elastic.co/u/caitlyn.carlisle)\ **Replies:** 2\ **Last updated:** [April 19, 2021, 11:55pm UTC](https://discuss.elastic.co/t/if-statements-logstash-pipeline/270637 "2021-04-19T23:55:51Z")
filter { if \["location\_lat"\] == “34.802075” { mutate { add\_field =\> { “Country” =\> “Syria” } } } if \["location\_lat"\] == “44.016521” { mutate { add\_field =\> { “Country” =\> “Serbia” } } } if \["locat… --- ## [Logstash: unrecognized service](https://discuss.elastic.co/t/logstash-unrecognized-service/270394)
**Author:** [@gisellecarballo](https://discuss.elastic.co/u/gisellecarballo)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 11:35pm UTC](https://discuss.elastic.co/t/logstash-unrecognized-service/270394 "2021-04-19T23:35:27Z")
Hi, I have my logstash actively running but when I executed "sudo service logstash configtest" result is "logstash: unrecognized service". I am running on Ubuntu 18.04 Many Thanks! --- ## [Looking for updated geoip database](https://discuss.elastic.co/t/looking-for-updated-geoip-database/270450)
**Author:** [@dddddddddddddddd](https://discuss.elastic.co/u/dddddddddddddddd)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 11:14pm UTC](https://discuss.elastic.co/t/looking-for-updated-geoip-database/270450 "2021-04-19T23:14:41Z")
hi if somebody can help me to find a free database GeoIP updated thanks. --- ## [Logstash setting field to date without any options set](https://discuss.elastic.co/t/logstash-setting-field-to-date-without-any-options-set/270621)
**Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 3\ **Last updated:** [April 19, 2021, 10:17pm UTC](https://discuss.elastic.co/t/logstash-setting-field-to-date-without-any-options-set/270621 "2021-04-19T22:17:56Z")
I am ingesting a csv file. The only options set in the filter are a gsub meant to remove double quotes; a csv filter which skips the header, the separator and the column/field names set; and a mutate to add one field an… --- ## [Ruby exception occurred: undefined method \`size' for nil:NilClass](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-size-for-nil-nilclass/270409)
**Author:** [@Eduard\_Abril](https://discuss.elastic.co/u/Eduard_Abril)\ **Replies:** 2\ **Last updated:** [April 19, 2021, 8:24pm UTC](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-size-for-nil-nilclass/270409 "2021-04-19T20:24:00Z")
Hi guys, I have the next error on a ruby filter. I'd really appreciate any suggest \[2021-04-16T11:28:06,202\]\[ERROR\]\[logstash.filters.ruby \]\[main-postilion-pipeline\]\[87afa373e1d0191b4f869238d2908162c237ad31fe6074ec0a… --- ## [Ingest JSON data into elasticsearch using logstash](https://discuss.elastic.co/t/ingest-json-data-into-elasticsearch-using-logstash/270558)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 4\ **Last updated:** [April 19, 2021, 6:33pm UTC](https://discuss.elastic.co/t/ingest-json-data-into-elasticsearch-using-logstash/270558 "2021-04-19T18:33:59Z")
HI All, I have a complicated json data to ingest into elasticsearch, need your suggestion to achieve this. Here is my data format, \ \ \/api/device/13766\ \ **Author:** [@tennaen](https://discuss.elastic.co/u/tennaen)\ **Replies:** 2\ **Last updated:** [April 19, 2021, 2:58pm UTC](https://discuss.elastic.co/t/how-to-make-it-work/270534 "2021-04-19T14:58:43Z") Dears, i have a problem and i cannot deal with it. In my pipeline configuration i have this code: if "%{\[process\]\[thread\]\[id\]}" =~ "\[a-z0-9\]+/\[a-z0-9\]+" { mutate { split =\> { "\[process\]\[thr… --- ## [%{\[Logs\]\[NET\] meaning](https://discuss.elastic.co/t/logs-net-meaning/270569)
**Author:** [@dddddddddddddddd](https://discuss.elastic.co/u/dddddddddddddddd)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 1:26pm UTC](https://discuss.elastic.co/t/logs-net-meaning/270569 "2021-04-19T13:26:51Z")
what is the meaning of index =\> "%{\[Logs\]\[NET\]-%{+YYYY.MM.dd}}" thanks --- ## [Snmp mib yamlmibdir](https://discuss.elastic.co/t/snmp-mib-yamlmibdir/270554)
**Author:** [@elastic\_study](https://discuss.elastic.co/u/elastic_study)\ **Replies:** 0\ **Last updated:** [April 19, 2021, 9:02am UTC](https://discuss.elastic.co/t/snmp-mib-yamlmibdir/270554 "2021-04-19T09:02:07Z")
How to convert a vendor mib file to an yaml mib ？ Is there any other tool？ --- ## [Mutate - using the value of one field to set the name of another](https://discuss.elastic.co/t/mutate-using-the-value-of-one-field-to-set-the-name-of-another/270462)
**Author:** [@timshaw](https://discuss.elastic.co/u/timshaw)\ **Replies:** 3\ **Last updated:** [April 18, 2021, 8:45pm UTC](https://discuss.elastic.co/t/mutate-using-the-value-of-one-field-to-set-the-name-of-another/270462 "2021-04-18T20:45:22Z")
The following fields come from an extract of a CEF format event in which is is possible to create custom fields and include labels for those fields relevant to a specific event. Labels may be different for different eve… --- ## [No config files found in path error Logstash Ubuntu](https://discuss.elastic.co/t/no-config-files-found-in-path-error-logstash-ubuntu/267318)
**Author:** [@varun1992](https://discuss.elastic.co/u/varun1992)\ **Replies:** 4\ **Last updated:** [April 18, 2021, 6:06am UTC](https://discuss.elastic.co/t/no-config-files-found-in-path-error-logstash-ubuntu/267318 "2021-04-18T06:06:01Z")
MLinux1@MLinux1:~/shared$ l LogStash-Beats-MTenant.conf\* MLinux1@MLinux1:~/shared$ My pipeline.yml as below - pipeline.id: beat\_input-shared path.config: "/home/MLinux1/shared/\*.conf" pipeline.workers: 2 queue.… --- ## [Issue using http\_poller to connect to streaming http api](https://discuss.elastic.co/t/issue-using-http-poller-to-connect-to-streaming-http-api/270434)
**Author:** [@metalshanked](https://discuss.elastic.co/u/metalshanked)\ **Replies:** 4\ **Last updated:** [April 17, 2021, 4:12pm UTC](https://discuss.elastic.co/t/issue-using-http-poller-to-connect-to-streaming-http-api/270434 "2021-04-17T16:12:22Z")
Hi Is there a way to implement a setting similar to Python requests "stream=True" setting in the Logstash http\_poller plugin. I am trying to pull streaming data from Shodan using logstash and get "no upstream" errors. … --- ## [Why is the logstash time difference of 5 minutes and 43 seconds before January 1, 1901？](https://discuss.elastic.co/t/why-is-the-logstash-time-difference-of-5-minutes-and-43-seconds-before-january-1-1901/270439)
**Author:** [@Fei1](https://discuss.elastic.co/u/Fei1)\ **Replies:** 0\ **Last updated:** [April 17, 2021, 6:39am UTC](https://discuss.elastic.co/t/why-is-the-logstash-time-difference-of-5-minutes-and-43-seconds-before-january-1-1901/270439 "2021-04-17T06:39:27Z")
Hello everyone, I am an ELK beginner, Background: Area: GMT+ 8 Time Zone Linux: CentOS Linux release 8.1.1911 (Core) Database：mysql Ver 15.1 Distrib 10.3.27-MariaDB, for Linux (x86\_64) using readline 5.1 logstash && … --- ## [Can ruby code be a speed bottleneck in logstash config file?](https://discuss.elastic.co/t/can-ruby-code-be-a-speed-bottleneck-in-logstash-config-file/270395)
**Author:** [@pk.241011](https://discuss.elastic.co/u/pk.241011)\ **Replies:** 4\ **Last updated:** [April 17, 2021, 3:21am UTC](https://discuss.elastic.co/t/can-ruby-code-be-a-speed-bottleneck-in-logstash-config-file/270395 "2021-04-17T03:21:09Z")
I m trying to get as much throughput as possible from my setup. I am using the http input. The client sometimes sends a collection of events instead of a single event for efficiency sake. Here is the relevant thread with… --- ## [Problem in importing csv with logstrash](https://discuss.elastic.co/t/problem-in-importing-csv-with-logstrash/270332)
**Author:** [@Fady\_Hany](https://discuss.elastic.co/u/Fady_Hany)\ **Replies:** 8\ **Last updated:** [April 16, 2021, 9:51pm UTC](https://discuss.elastic.co/t/problem-in-importing-csv-with-logstrash/270332 "2021-04-16T21:51:38Z")
i have read a lot of similar problems in this nice forum , but non of them fit my situation my .conf file : input { file { path =\> "E:/test/\*.csv" start\_position =\> "beginning" sincedb\_pat… --- ## [Convert Packetbeat Field Contents to Lowercase](https://discuss.elastic.co/t/convert-packetbeat-field-contents-to-lowercase/270402)
**Author:** [@John\_Collins](https://discuss.elastic.co/u/John_Collins)\ **Replies:** 8\ **Last updated:** [April 16, 2021, 8:31pm UTC](https://discuss.elastic.co/t/convert-packetbeat-field-contents-to-lowercase/270402 "2021-04-16T20:31:58Z")
I am sending DNS queries from a windows DNS server to Logstash via Packetbeat. When I view the contents of the field dns.question.name in Kibana, some entries contain capitalizations. I have tried the following in a Lo… --- ## [Logstash monitoring with Metricbeat and cluster\_uuid](https://discuss.elastic.co/t/logstash-monitoring-with-metricbeat-and-cluster-uuid/270410)
**Author:** [@jasenj1](https://discuss.elastic.co/u/jasenj1)\ **Replies:** 0\ **Last updated:** [April 16, 2021, 5:34pm UTC](https://discuss.elastic.co/t/logstash-monitoring-with-metricbeat-and-cluster-uuid/270410 "2021-04-16T17:34:04Z")
When configuring Logstash monitoring, the Elastic recommended best practice is to create a cluster dedicated to log and metrics collection. One then configures Metricbeat to collect the monitoring data from Logstash us… --- ## [How to exclude XML & json key-value if key length is greater than 15 char and value length is greater than 100 char](https://discuss.elastic.co/t/how-to-exclude-xml-json-key-value-if-key-length-is-greater-than-15-char-and-value-length-is-greater-than-100-char/270248)
**Author:** [@mangeshmj1992](https://discuss.elastic.co/u/mangeshmj1992)\ **Replies:** 14\ **Last updated:** [April 16, 2021, 4:19pm UTC](https://discuss.elastic.co/t/how-to-exclude-xml-json-key-value-if-key-length-is-greater-than-15-char-and-value-length-is-greater-than-100-char/270248 "2021-04-16T16:19:18Z")
@Badger I want to exclude XML & json key-value if key length is greater than 15 char and value length is greater than 100 char. Is this possible can you please suggest way for it. --- ## [How to convert extract month from date](https://discuss.elastic.co/t/how-to-convert-extract-month-from-date/270035)
**Author:** [@Pallavibhushan](https://discuss.elastic.co/u/Pallavibhushan)\ **Replies:** 5\ **Last updated:** [April 16, 2021, 2:35pm UTC](https://discuss.elastic.co/t/how-to-convert-extract-month-from-date/270035 "2021-04-16T14:35:16Z")
Hi, I have below date format . First\_date : Feb-21 (MMM-yy) Second\_date: 12/02/2021 (MM/dd/yyyy) first, I want new field like below: from first date I need month . so output should be first\_date\_month : 02 second … --- ## [Date parse failure](https://discuss.elastic.co/t/date-parse-failure/270276)
**Author:** [@Wilfried](https://discuss.elastic.co/u/Wilfried)\ **Replies:** 2\ **Last updated:** [April 16, 2021, 1:42pm UTC](https://discuss.elastic.co/t/date-parse-failure/270276 "2021-04-16T13:42:12Z")
Hello, I struggle to understand why my date matching is failing. We trying to build a reporting application using elasticsearch for our Storage. I've that kind of data in an XML: \ \ \0\ **Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 0\ **Last updated:** [April 16, 2021, 10:58am UTC](https://discuss.elastic.co/t/logstash-data-load-error-for-a-different-format-data/270375 "2021-04-16T10:58:58Z") Hi All, We are trying to ingest few data into logstash, the data is pretty different like mentioned below, How do i map the collection\_time with 1618556220 and d\_check to 1 I gave a split filter it didnt work. FYI. … --- ## [Provided Grok patterns do not match data in the input](https://discuss.elastic.co/t/provided-grok-patterns-do-not-match-data-in-the-input/270265)
**Author:** [@Cris\_R](https://discuss.elastic.co/u/Cris_R)\ **Replies:** 4\ **Last updated:** [April 16, 2021, 6:57am UTC](https://discuss.elastic.co/t/provided-grok-patterns-do-not-match-data-in-the-input/270265 "2021-04-16T06:57:17Z")
Hi, Trying to create my own grok patterns, I'm using the following with the Grok Debugger : Sample data : \[ 4812 6032\]\[15 Feb 18:23:15\]\[LdManInit\] Loading Library in Load mode Grok Pattern : %{IDLOG:whom} %{TIMEST:… --- ## [Upgrarding from logstash 5.6.16 to 6.8 results in errors for logstash tcp input plugin](https://discuss.elastic.co/t/upgrarding-from-logstash-5-6-16-to-6-8-results-in-errors-for-logstash-tcp-input-plugin/270316)
**Author:** [@mchoudhary](https://discuss.elastic.co/u/mchoudhary)\ **Replies:** 0\ **Last updated:** [April 16, 2021, 1:19am UTC](https://discuss.elastic.co/t/upgrarding-from-logstash-5-6-16-to-6-8-results-in-errors-for-logstash-tcp-input-plugin/270316 "2021-04-16T01:19:43Z")
After upgrading the logstash version from 5.6.16 to 6.8, the logstash tcp plugin from 4.2.2 to 5.2.2 After upgrade, we do see the following errors in the logstash logs: \[2021-04-16T01:14:50,392\]\[ERROR\]\[logstash.inputs.… --- ## [Using Fortinet Integration in Logstash](https://discuss.elastic.co/t/using-fortinet-integration-in-logstash/270304)
**Author:** [@b0r1s](https://discuss.elastic.co/u/b0r1s)\ **Replies:** 2\ **Last updated:** [April 15, 2021, 10:17pm UTC](https://discuss.elastic.co/t/using-fortinet-integration-in-logstash/270304 "2021-04-15T22:17:28Z")
Hey everyone o/ I'm using that conf I found on this topic for parsing logs from FortigateFW with Logstash: input { udp { port =\> 514 type =\> "forti\_log" tags =\> \[""\] } } #THE BEGINNING DOESN'T FORMAT TO CODE … --- ## [Weird test configuration problem || log\_file --\> Syslog-ng --\> Logstash --\> another\_file](https://discuss.elastic.co/t/weird-test-configuration-problem-log-file-syslog-ng-logstash-another-file/270253)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 1\ **Last updated:** [April 15, 2021, 6:58pm UTC](https://discuss.elastic.co/t/weird-test-configuration-problem-log-file-syslog-ng-logstash-another-file/270253 "2021-04-15T18:58:42Z")
Hi All, please ignore the stupid use case as I only use this config for testing purposes. I have an Ubuntu server with installed syslog-ng and Logstash. I'd like to do the following: I have a cronjob that on every min… --- ## [Logstash - elasticsarch and double quotes](https://discuss.elastic.co/t/logstash-elasticsarch-and-double-quotes/270133)
**Author:** [@cibernicola](https://discuss.elastic.co/u/cibernicola)\ **Replies:** 11\ **Last updated:** [April 15, 2021, 4:49pm UTC](https://discuss.elastic.co/t/logstash-elasticsarch-and-double-quotes/270133 "2021-04-15T16:49:21Z")
Hello I'm trying to ingest logs wich have " in some of their fields as part of a string, I'm trying to scape them with \\ or \\ or \\\\ but no luck, so, is there any way to ingest double quotes from logs into elasticsearch … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=236) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=238)