# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=418 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 419 --- ## [Differentiate Linux syslog log from Windows event log](https://discuss.elastic.co/t/differentiate-linux-syslog-log-from-windows-event-log/215449)
**Author:** [@claud81](https://discuss.elastic.co/u/claud81)\ **Replies:** 5\ **Last updated:** [January 17, 2020, 3:43pm UTC](https://discuss.elastic.co/t/differentiate-linux-syslog-log-from-windows-event-log/215449 "2020-01-17T15:43:32Z")
Hi everyone, we have 2 conf file in Logstash (version 6.8.6) conf.d directory to differentiate Microsoft log from Linux log, the first: input { tcp { port =\> 3515 codec =\> json … --- ## [Logstash Date parsing error from Kinesis](https://discuss.elastic.co/t/logstash-date-parsing-error-from-kinesis/215310)
**Author:** [@Chris\_Broll](https://discuss.elastic.co/u/Chris_Broll)\ **Replies:** 1\ **Last updated:** [January 17, 2020, 3:22pm UTC](https://discuss.elastic.co/t/logstash-date-parsing-error-from-kinesis/215310 "2020-01-17T15:22:01Z")
I have picked up a Logstash server to look after and I can see some incoming events from a Kinesis input that is failing to be indexed: Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>n… --- ## [Using Logstash to correct timestamps](https://discuss.elastic.co/t/using-logstash-to-correct-timestamps/215379)
**Author:** [@Hyperion0](https://discuss.elastic.co/u/Hyperion0)\ **Replies:** 8\ **Last updated:** [January 17, 2020, 3:11pm UTC](https://discuss.elastic.co/t/using-logstash-to-correct-timestamps/215379 "2020-01-17T15:11:40Z")
I'm very new to the ELK stack. I have several logs that starts collecting data before the systems time is set. The result is I have several hundred log entries with in accurate time stamps. Is there a way I can edit th… --- ## [How to skip some lines from a log file without skipping the complete message](https://discuss.elastic.co/t/how-to-skip-some-lines-from-a-log-file-without-skipping-the-complete-message/215399)
**Author:** [@Daniel\_Gonzalez1](https://discuss.elastic.co/u/Daniel_Gonzalez1)\ **Replies:** 1\ **Last updated:** [January 17, 2020, 3:04pm UTC](https://discuss.elastic.co/t/how-to-skip-some-lines-from-a-log-file-without-skipping-the-complete-message/215399 "2020-01-17T15:04:06Z")
Hello Everyone! I'm working with a file that has some lines that I want to skip when they come to logstash. These are a sample of the log lines: \> 15Jan20 11:34:10.39 PRHTGRCE MXP1 RACF CONNECT success for PRHTGRCE: C… --- ## [Logstash is not pulling data from s3 bucket](https://discuss.elastic.co/t/logstash-is-not-pulling-data-from-s3-bucket/215470)
**Author:** [@Sreekanth3](https://discuss.elastic.co/u/Sreekanth3)\ **Replies:** 0\ **Last updated:** [January 17, 2020, 1:47pm UTC](https://discuss.elastic.co/t/logstash-is-not-pulling-data-from-s3-bucket/215470 "2020-01-17T13:47:38Z")
Hi , This is my logstash conf input { s3 { access\_key\_id =\> "xxxx" secret\_access\_key =\> "xxx" #bucket =\> "mybucketname/2020/01/16" bucket =\> "sftlcloudtrail" sincedb\_path =\> "/tmp/last-s3-file-s3-access-logs" ad… --- ## [Not all logs are forwarded to elasticsearch](https://discuss.elastic.co/t/not-all-logs-are-forwarded-to-elasticsearch/215282)
**Author:** [@tennaen](https://discuss.elastic.co/u/tennaen)\ **Replies:** 3\ **Last updated:** [January 17, 2020, 2:47pm UTC](https://discuss.elastic.co/t/not-all-logs-are-forwarded-to-elasticsearch/215282 "2020-01-17T14:47:57Z")
Good morning, i have an issue with logstash. I use jdbc input to collect data from postgres and then forward them to elastic. I've noticed that there is a difference between what is forwarded to elastic (i use kibana to… --- ## [Mutate is not working as expected](https://discuss.elastic.co/t/mutate-is-not-working-as-expected/215423)
**Author:** [@Saravana37](https://discuss.elastic.co/u/Saravana37)\ **Replies:** 2\ **Last updated:** [January 17, 2020, 10:54am UTC](https://discuss.elastic.co/t/mutate-is-not-working-as-expected/215423 "2020-01-17T10:54:46Z")
Hello All , I am trying to convert MaxTasks field to Interger , I tried two ways here . Specifying the Number format in Grok filter also converting it to Integer using Mutate filter. But of no use. Anything wrong here ?… --- ## [Drop event if condition](https://discuss.elastic.co/t/drop-event-if-condition/215370)
**Author:** [@dennys\_84](https://discuss.elastic.co/u/dennys_84)\ **Replies:** 6\ **Last updated:** [January 17, 2020, 10:41am UTC](https://discuss.elastic.co/t/drop-event-if-condition/215370 "2020-01-17T10:41:35Z")
Hi all. We are trying to accomplish such task: Log auth events from windows security logs. We set winlogbeat to export data to logstash. But problem is that there are too much incoming events with the same id. So we ne… --- ## [Use Logstash for filtering out "not importen data "and as output use Filebeat](https://discuss.elastic.co/t/use-logstash-for-filtering-out-not-importen-data-and-as-output-use-filebeat/215429)
**Author:** [@juuuhuuu](https://discuss.elastic.co/u/juuuhuuu)\ **Replies:** 2\ **Last updated:** [January 17, 2020, 10:27am UTC](https://discuss.elastic.co/t/use-logstash-for-filtering-out-not-importen-data-and-as-output-use-filebeat/215429 "2020-01-17T10:27:52Z")
Hello everyone, I would like to know if its possible to have logstash for filtering data out and in output of logstash thanks filebeats send this logs to an other node where are elasticsearch and Kibana .. node a (… --- ## [Logstash or re-index API to chose for re-indexing?](https://discuss.elastic.co/t/logstash-or-re-index-api-to-chose-for-re-indexing/215435)
**Author:** [@arunpmohan](https://discuss.elastic.co/u/arunpmohan)\ **Replies:** 0\ **Last updated:** [January 17, 2020, 10:18am UTC](https://discuss.elastic.co/t/logstash-or-re-index-api-to-chose-for-re-indexing/215435 "2020-01-17T10:18:59Z")
We have a cluster with the following configurations 12 nodes Elasticsearch 5.x 24 TB data I need to upgrade this cluster to Elasticsearch 7.x So we have come up with a plan to provision a new cluster with exact same … --- ## [Extract all jsons from log to one field](https://discuss.elastic.co/t/extract-all-jsons-from-log-to-one-field/215286)
**Author:** [@dardev](https://discuss.elastic.co/u/dardev)\ **Replies:** 9\ **Last updated:** [January 17, 2020, 9:29am UTC](https://discuss.elastic.co/t/extract-all-jsons-from-log-to-one-field/215286 "2020-01-17T09:29:27Z")
Hello, I'm trying to extract few possible jsons from log and put them to one field into an array. This is the example of log I want to parse: This is one of example logs: \[{"lat":12.33,"lng":55.44}\] that I should corr… --- ## [Multiple Grok pattern filters arent filtering multiple logs in one logstash file](https://discuss.elastic.co/t/multiple-grok-pattern-filters-arent-filtering-multiple-logs-in-one-logstash-file/214654)
**Author:** [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)\ **Replies:** 7\ **Last updated:** [January 16, 2020, 11:33pm UTC](https://discuss.elastic.co/t/multiple-grok-pattern-filters-arent-filtering-multiple-logs-in-one-logstash-file/214654 "2020-01-16T23:33:57Z")
Below is my pipeline.conf where I want the filter block to apply three separate grok patterns on three different log files. But currently, it isnt working. Should I add multiple pipelines to it? or create three different… --- ## [Error 400 during parsing logs from logstash](https://discuss.elastic.co/t/error-400-during-parsing-logs-from-logstash/214352)
**Author:** [@akshay\_singh2](https://discuss.elastic.co/u/akshay_singh2)\ **Replies:** 7\ **Last updated:** [January 16, 2020, 11:06pm UTC](https://discuss.elastic.co/t/error-400-during-parsing-logs-from-logstash/214352 "2020-01-16T23:06:24Z")
Hello Team, I have recently moved ES from 6.x to 7.x and faced a issue in one of pipeline which was working fine in ES 6.x version, but in ES 7.x I'm getting 400 error in logstash. Logstash is flooded with this error: … --- ## [General OpenSslEngine problem](https://discuss.elastic.co/t/general-opensslengine-problem/215358)
**Author:** [@lsambolino](https://discuss.elastic.co/u/lsambolino)\ **Replies:** 0\ **Last updated:** [January 16, 2020, 4:36pm UTC](https://discuss.elastic.co/t/general-opensslengine-problem/215358 "2020-01-16T16:36:02Z")
Hi, In our infrastructure we have deployed logstash on Debian nodes. At the moment, it appears logstash is not producing logs anymore. Log timestamp is stucked at 10 hours before the moment of writing. After checking /… --- ## [Aggregate logs if value of field is same](https://discuss.elastic.co/t/aggregate-logs-if-value-of-field-is-same/215348)
**Author:** [@mouloudjean](https://discuss.elastic.co/u/mouloudjean)\ **Replies:** 0\ **Last updated:** [January 16, 2020, 3:54pm UTC](https://discuss.elastic.co/t/aggregate-logs-if-value-of-field-is-same/215348 "2020-01-16T15:54:45Z")
Hello, I'm working on a project in my company, The goal is to implement the ELK stack to have an tracability of our data, and be able to aggregate some logs that containt the same value of a field. I'm trying to aggreg… --- ## [Install aggregate plugin (Offline) Logstash](https://discuss.elastic.co/t/install-aggregate-plugin-offline-logstash/215299)
**Author:** [@Khalil\_Diouri](https://discuss.elastic.co/u/Khalil_Diouri)\ **Replies:** 1\ **Last updated:** [January 16, 2020, 3:36pm UTC](https://discuss.elastic.co/t/install-aggregate-plugin-offline-logstash/215299 "2020-01-16T15:36:32Z")
Hello, Generally I need to install a plugin aggregate for logstash in a remote server to use it in my file .config to inject data in elasticsearch. the problem is that I can't download directly the aggregate plugin usi… --- ## [Logstash 7.5.1 flood syslog with messages](https://discuss.elastic.co/t/logstash-7-5-1-flood-syslog-with-messages/214366)
**Author:** [@wga](https://discuss.elastic.co/u/wga)\ **Replies:** 2\ **Last updated:** [January 16, 2020, 3:23pm UTC](https://discuss.elastic.co/t/logstash-7-5-1-flood-syslog-with-messages/214366 "2020-01-16T15:23:18Z")
After upgrade from 7.5 to 7.5.1 except that all messages which receive from filebeat nodes going to elastic, for some reason start bombing /var/log/syslog with the same information without even to have any of output-sysl… --- ## [Could not find jruby in C:\\logstash-7.5.1\\vendor\\jruby](https://discuss.elastic.co/t/could-not-find-jruby-in-c-logstash-7-5-1-vendor-jruby/215319)
**Author:** [@martinm1](https://discuss.elastic.co/u/martinm1)\ **Replies:** 0\ **Last updated:** [January 16, 2020, 1:47pm UTC](https://discuss.elastic.co/t/could-not-find-jruby-in-c-logstash-7-5-1-vendor-jruby/215319 "2020-01-16T13:47:01Z")
logstash-7.5.1 Win10/64 jdk1.8.0\_241 I'm following the windows installation instructions and have setup the Java paths. I also installed JRuby in case it was needed but when I still get PS C:\\logstash-7.5.1\> ./bin/lo… --- ## [Different index for different logs](https://discuss.elastic.co/t/different-index-for-different-logs/215317)
**Author:** [@srisum0728](https://discuss.elastic.co/u/srisum0728)\ **Replies:** 0\ **Last updated:** [January 16, 2020, 1:33pm UTC](https://discuss.elastic.co/t/different-index-for-different-logs/215317 "2020-01-16T13:33:52Z")
Need different index for each log file path. 1\> access log in access.conf file -\> same server 2\> csv file in task\_engine.csv -\> same server Need to have task\_engine.csv file in different index so that I can pull it in… --- ## [Can't load installed filter plugin](https://discuss.elastic.co/t/cant-load-installed-filter-plugin/215303)
**Author:** [@toms130](https://discuss.elastic.co/u/toms130)\ **Replies:** 0\ **Last updated:** [January 16, 2020, 11:27am UTC](https://discuss.elastic.co/t/cant-load-installed-filter-plugin/215303 "2020-01-16T11:27:44Z")
Hi all, I found logstash-filter-jwt-decode plugin to decode jwt tokens, but I am not able to load it despite successfull installation. Below is what i've done. bash-4.2$ logstash-plugin install logstash-filter-jwt-deco… --- ## [Logstash as SNMP Trap to GELF Converter, how to split message into seperate fields?](https://discuss.elastic.co/t/logstash-as-snmp-trap-to-gelf-converter-how-to-split-message-into-seperate-fields/214988)
**Author:** [@Salve](https://discuss.elastic.co/u/Salve)\ **Replies:** 2\ **Last updated:** [January 16, 2020, 5:37am UTC](https://discuss.elastic.co/t/logstash-as-snmp-trap-to-gelf-converter-how-to-split-message-into-seperate-fields/214988 "2020-01-16T05:37:25Z")
Hello, I am new to logstash and I want to use it to convert SNMP Traps to GELF. This is my very first (simple) pipeline. input { snmptrap { id =\> "snmptrap" type =\> "snmptrap" port =\> 1162 } } output {… --- ## [Add additional field (count of another field)](https://discuss.elastic.co/t/add-additional-field-count-of-another-field/215008)
**Author:** [@anon42972578](https://discuss.elastic.co/u/anon42972578)\ **Replies:** 8\ **Last updated:** [January 16, 2020, 10:43am UTC](https://discuss.elastic.co/t/add-additional-field-count-of-another-field/215008 "2020-01-16T10:43:37Z")
Hello, i have documents with multiline fields in my.field and need to count the number of received filenames. @Fabio-sama suggested me in another thread (which was in the wrong section) something like that: { "my… --- ## [Error: Unable to find driver class via URLClassLoader in given driver jars](https://discuss.elastic.co/t/error-unable-to-find-driver-class-via-urlclassloader-in-given-driver-jars/215254)
**Author:** [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Replies:** 1\ **Last updated:** [January 16, 2020, 10:32am UTC](https://discuss.elastic.co/t/error-unable-to-find-driver-class-via-urlclassloader-in-given-driver-jars/215254 "2020-01-16T10:32:05Z")
Hello, I am trying run config file using logstash. It is giving below error: Error: Unable to find driver class via URLClassLoader in given driver jars: Java::oracle.jdbc.driver.OracleDriver and Java::oracle.jdbc.dri… --- ## [Field list](https://discuss.elastic.co/t/field-list/214863)
**Author:** [@volcano](https://discuss.elastic.co/u/volcano)\ **Replies:** 22\ **Last updated:** [January 16, 2020, 4:29am UTC](https://discuss.elastic.co/t/field-list/214863 "2020-01-16T04:29:03Z")
I have this entry in logstash.conf mutate { split =\> \["message","Employee"\] add\_field =\> {"part1" =\>"%{\[message\]\[0\]}"} add\_field =\> {"part2" =\>"%{\[message\]\[1\]}"} } mutate { split =\> \[… --- ## [Geo lookups using Canadian postal codes](https://discuss.elastic.co/t/geo-lookups-using-canadian-postal-codes/215230)
**Author:** [@Rob\_wylde](https://discuss.elastic.co/u/Rob_wylde)\ **Replies:** 2\ **Last updated:** [January 16, 2020, 1:30am UTC](https://discuss.elastic.co/t/geo-lookups-using-canadian-postal-codes/215230 "2020-01-16T01:30:22Z")
Has anyone come across a solution for this? I know maxmind mmdb's are not an option. So i think im stuck possibly writing a custom filter to query a google api and to be honest I do not even know where to begin with that… --- ## [Skip\_on\_invalid\_json skipping all filters](https://discuss.elastic.co/t/skip-on-invalid-json-skipping-all-filters/215195)
**Author:** [@jfcantu](https://discuss.elastic.co/u/jfcantu)\ **Replies:** 3\ **Last updated:** [January 16, 2020, 12:06am UTC](https://discuss.elastic.co/t/skip-on-invalid-json-skipping-all-filters/215195 "2020-01-16T00:06:54Z")
Hi, I ran across something unexpected yesterday, that doesn't seem to line up with the Logstash documentation, and I was wondering if this was intended behavior (or if I did something wrong.) TL;DR - if you have multip… --- ## [Ingesting latest logs from kafka with logstash](https://discuss.elastic.co/t/ingesting-latest-logs-from-kafka-with-logstash/215209)
**Author:** [@canaria](https://discuss.elastic.co/u/canaria)\ **Replies:** 2\ **Last updated:** [January 15, 2020, 10:20pm UTC](https://discuss.elastic.co/t/ingesting-latest-logs-from-kafka-with-logstash/215209 "2020-01-15T22:20:46Z")
Hi, I searched previous opened topics related that question but none of them have any reply. I’m ingesting logs from apache kafka with logstash, everything is fine but I noticed that whenever I started logstash instance… --- ## [Multiple nodes that get input from a scheduled JDBC database query?](https://discuss.elastic.co/t/multiple-nodes-that-get-input-from-a-scheduled-jdbc-database-query/215218)
**Author:** [@aidanoc15](https://discuss.elastic.co/u/aidanoc15)\ **Replies:** 1\ **Last updated:** [January 15, 2020, 10:17pm UTC](https://discuss.elastic.co/t/multiple-nodes-that-get-input-from-a-scheduled-jdbc-database-query/215218 "2020-01-15T22:17:36Z")
Right now we have one logstash node running in Docker that queries a database every 2 minutes using JDBC and sends it over to a collector. How does something like this work if you want to have this done by a swarm of nod… --- ## [Error FORBIDDEN/8/index write (api)\] and Elasticsearch stop to receiving logstash bulk requests](https://discuss.elastic.co/t/error-forbidden-8-index-write-api-and-elasticsearch-stop-to-receiving-logstash-bulk-requests/215028)
**Author:** [@ALEXANDRE\_BUNN1](https://discuss.elastic.co/u/ALEXANDRE_BUNN1)\ **Replies:** 7\ **Last updated:** [January 15, 2020, 10:16pm UTC](https://discuss.elastic.co/t/error-forbidden-8-index-write-api-and-elasticsearch-stop-to-receiving-logstash-bulk-requests/215028 "2020-01-15T22:16:55Z")
I have a server running Logstash, Kibana and ElasticSearch 6.8, all in the same server. I was setting up the ILM for the indexes, just to set a retention and executing force\_merge for the old indices (older than 1 day) … --- ## [Different charset in filter http](https://discuss.elastic.co/t/different-charset-in-filter-http/215219)
**Author:** [@Peter\_Li](https://discuss.elastic.co/u/Peter_Li)\ **Replies:** 0\ **Last updated:** [January 15, 2020, 9:56pm UTC](https://discuss.elastic.co/t/different-charset-in-filter-http/215219 "2020-01-15T21:56:26Z")
I have a filter http module, but http returns nonUTF8 character sequences. Got this error: invalid byte sequence in utf-8 If one use input http, one can add "codec plain" to deal with this. But no such option exist fo… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=417) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=419)