# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=427 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 428 --- ## [How can we put multiline filter for below custom log](https://discuss.elastic.co/t/how-can-we-put-multiline-filter-for-below-custom-log/213161)
**Author:** [@KashinathPatil](https://discuss.elastic.co/u/KashinathPatil)\ **Replies:** 5\ **Last updated:** [December 27, 2019, 1:36pm UTC](https://discuss.elastic.co/t/how-can-we-put-multiline-filter-for-below-custom-log/213161 "2019-12-27T13:36:09Z")
MachineName: PC00287 TimeStamp: 12/19/2019 6:18:33 PM FullName: POA.ReaserchSolution.Infrastructure.Logging, Version=2.51.7292.26907, Culture=neutral, PublicKeyToken=null AppDomainName: /RM/W3SVC/4/ROOT-2-132212323240… --- ## [Parse logs generated by Ksh script](https://discuss.elastic.co/t/parse-logs-generated-by-ksh-script/213166)
**Author:** [@simo123](https://discuss.elastic.co/u/simo123)\ **Replies:** 1\ **Last updated:** [December 27, 2019, 11:50am UTC](https://discuss.elastic.co/t/parse-logs-generated-by-ksh-script/213166 "2019-12-27T11:50:20Z")
Hello everyone, I'm just beginning with ELK . I have a custom logs generated by Ksh Script (see Below) and I want is possible to exctact specific lines from it like (Comment,Start Date,End Date ) Perhaps should I use G… --- ## [Azure nsg flow logs](https://discuss.elastic.co/t/azure-nsg-flow-logs/213147)
**Author:** [@sourabhlangoo](https://discuss.elastic.co/u/sourabhlangoo)\ **Replies:** 0\ **Last updated:** [December 27, 2019, 7:16am UTC](https://discuss.elastic.co/t/azure-nsg-flow-logs/213147 "2019-12-27T07:16:05Z")
Hi, I need to parse azure nsg flow log using logstash version 2.4 but i am unable to find the solution following is the log Input Log : {"records":\[{"time":"2019-12-06T10:37:14.5140721Z","systemId":"2c002c16-72f3-4d… --- ## [Disable "amazon\_es" output plugin console logs](https://discuss.elastic.co/t/disable-amazon-es-output-plugin-console-logs/209995)
**Author:** [@ArunANayagam](https://discuss.elastic.co/u/ArunANayagam)\ **Replies:** 1\ **Last updated:** [December 27, 2019, 9:21am UTC](https://discuss.elastic.co/t/disable-amazon-es-output-plugin-console-logs/209995 "2019-12-27T09:21:01Z")
Hi, I use a docker logstash (7.1.1) and write to AWS ES via amazon\_es output plugin. The plugin as such works fine, but then all the messages are printed on the console and that generates a lot of I/O on disk, making t… --- ## [Split - Only String and Array types are splittable. field:\[value\] is of type = NilClass](https://discuss.elastic.co/t/split-only-string-and-array-types-are-splittable-field-value-is-of-type-nilclass/213154)
**Author:** [@exilemirror](https://discuss.elastic.co/u/exilemirror)\ **Replies:** 0\ **Last updated:** [December 27, 2019, 8:11am UTC](https://discuss.elastic.co/t/split-only-string-and-array-types-are-splittable-field-value-is-of-type-nilclass/213154 "2019-12-27T08:11:08Z")
Hi I'm trying to filter a JSON file extracted from Microsoft Graph API. JSON file content below. { "@odata.context": "https://microsoft.com", "value": \[ { "currentScore": 203.0, "maxScore": 802.0, "createdDateTim… --- ## [\[logstash.outputs.kafka \] Sending batch to Kafka failed](https://discuss.elastic.co/t/logstash-outputs-kafka-sending-batch-to-kafka-failed/212942)
**Author:** [@gloria\_xie](https://discuss.elastic.co/u/gloria_xie)\ **Replies:** 2\ **Last updated:** [December 27, 2019, 7:15am UTC](https://discuss.elastic.co/t/logstash-outputs-kafka-sending-batch-to-kafka-failed/212942 "2019-12-27T07:15:45Z")
hi everyone, i am using logstash to sink data from one kafka topic A to another kafka topic B; whenever A produced data,B will receive it immediately; everything works fine,but i keep getting this message in the log … --- ## [Logstash aggregate filter is inconsistent](https://discuss.elastic.co/t/logstash-aggregate-filter-is-inconsistent/212925)
**Author:** [@karfei00](https://discuss.elastic.co/u/karfei00)\ **Replies:** 2\ **Last updated:** [December 27, 2019, 6:02am UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-is-inconsistent/212925 "2019-12-27T06:02:10Z")
I'm trying to group multiple cities into the same country using data from jdbc. I'm making use of the config push\_previous\_map\_as\_event, but result is inconsistent even though SQL is ordered correctly by country code. T… --- ## [Filtering in LogStash using the if..in syntax](https://discuss.elastic.co/t/filtering-in-logstash-using-the-if-in-syntax/213090)
**Author:** [@RunningSmurf](https://discuss.elastic.co/u/RunningSmurf)\ **Replies:** 2\ **Last updated:** [December 26, 2019, 7:42pm UTC](https://discuss.elastic.co/t/filtering-in-logstash-using-the-if-in-syntax/213090 "2019-12-26T19:42:53Z")
Hello, I want to update the "tags" filed to a certain value when my "message" contains the string "Error: 18456" in it. This is the \[message\] value that ends up in Kibana: message 2019-12-26 15:07:09.53 Logon Er… --- ## [Can I import data from Oracle adavanced queue table to rabbitmq or kafka using logstash](https://discuss.elastic.co/t/can-i-import-data-from-oracle-adavanced-queue-table-to-rabbitmq-or-kafka-using-logstash/213092)
**Author:** [@Pramod\_Thakur](https://discuss.elastic.co/u/Pramod_Thakur)\ **Replies:** 0\ **Last updated:** [December 26, 2019, 3:48pm UTC](https://discuss.elastic.co/t/can-i-import-data-from-oracle-adavanced-queue-table-to-rabbitmq-or-kafka-using-logstash/213092 "2019-12-26T15:48:22Z")
Hi Folks, Can I import data from Oracle advanced queue table to rabbitmq or kafka using logstash. Is logstash work with other platforms or only works with elastic search... --- ## [Logstash not reading data from text file in windows](https://discuss.elastic.co/t/logstash-not-reading-data-from-text-file-in-windows/213013)
**Author:** [@sayyadsays](https://discuss.elastic.co/u/sayyadsays)\ **Replies:** 4\ **Last updated:** [December 26, 2019, 2:04pm UTC](https://discuss.elastic.co/t/logstash-not-reading-data-from-text-file-in-windows/213013 "2019-12-26T14:04:08Z")
Hi when i tried with stdin{} it works fine. but reading from file not getting any data in elasticsearch in windows. Please find the below conf details. Conf file: input{ # stdin{} file{ type =\>"syslog" path =\> \["… --- ## [How to handle message receive multiline in logstash](https://discuss.elastic.co/t/how-to-handle-message-receive-multiline-in-logstash/213083)
**Author:** [@Bob94](https://discuss.elastic.co/u/Bob94)\ **Replies:** 0\ **Last updated:** [December 26, 2019, 1:35pm UTC](https://discuss.elastic.co/t/how-to-handle-message-receive-multiline-in-logstash/213083 "2019-12-26T13:35:21Z")
Hi all, I create a grok pattern and execute some bash script..Unfortunately this script produce two line of output. The question is how to properly index the data into elasticsearch as it contains 2 line of text in mess… --- ## [Getting "Pipeline aborted due to error"](https://discuss.elastic.co/t/getting-pipeline-aborted-due-to-error/213077)
**Author:** [@saravananveera](https://discuss.elastic.co/u/saravananveera)\ **Replies:** 0\ **Last updated:** [December 26, 2019, 12:42pm UTC](https://discuss.elastic.co/t/getting-pipeline-aborted-due-to-error/213077 "2019-12-26T12:42:36Z")
Hi I have checked the logstash config test. it's show there is no issue in config file but i have received the logstash log "Pipeline aborted due to error" Config test log: /usr/share/logstash/bin/logstash -f /etc/log… --- ## [Logstash jdbc limited import](https://discuss.elastic.co/t/logstash-jdbc-limited-import/213056)
**Author:** [@Zeyni\_Adan](https://discuss.elastic.co/u/Zeyni_Adan)\ **Replies:** 3\ **Last updated:** [December 26, 2019, 10:45am UTC](https://discuss.elastic.co/t/logstash-jdbc-limited-import/213056 "2019-12-26T10:45:49Z")
Hi. am sending data from MySQL to elasticsearch using logstash. i have 10469 records but only 132 is getting imported. kindly advise. --- ## [Logstash is not working when i am importing data](https://discuss.elastic.co/t/logstash-is-not-working-when-i-am-importing-data/213052)
**Author:** [@Pramod\_Thakur](https://discuss.elastic.co/u/Pramod_Thakur)\ **Replies:** 0\ **Last updated:** [December 26, 2019, 7:39am UTC](https://discuss.elastic.co/t/logstash-is-not-working-when-i-am-importing-data/213052 "2019-12-26T07:39:26Z")
Hi Folks, I am having the following setup. OS: Window 10 Enterprise 64 bit Elasticsearch, Logstash, Kibana version: 7.2.0 My elastic search and kibana is working fine. when I am trying to configure logstash its throw… --- ## [Fetching Data from Databses](https://discuss.elastic.co/t/fetching-data-from-databses/213041)
**Author:** [@Sagar\_Mandal](https://discuss.elastic.co/u/Sagar_Mandal)\ **Replies:** 0\ **Last updated:** [December 26, 2019, 5:09am UTC](https://discuss.elastic.co/t/fetching-data-from-databses/213041 "2019-12-26T05:09:57Z")
Hi Team, Okay so I have this configuration file which fetches data from oracle DB, the problem is that in this config file there are more than 50 queries, which are also scheduled, the problem is that they all are runni… --- ## [Logstash IndexOutOfBoundsException](https://discuss.elastic.co/t/logstash-indexoutofboundsexception/213020)
**Author:** [@rsmoorthy](https://discuss.elastic.co/u/rsmoorthy)\ **Replies:** 0\ **Last updated:** [December 25, 2019, 4:49pm UTC](https://discuss.elastic.co/t/logstash-indexoutofboundsexception/213020 "2019-12-25T16:49:56Z")
Logstash version is 6.4.0, Filebeat is 6.4.2. Today, we are receiving the following exceptions continuously on Logstash (several of this appear): \[2019-12-25T14:17:07,113\]\[INFO \]\[org.logstash.beats.BeatsHandler\] \[local… --- ## [How to concatenate time fields and generate new timestamp](https://discuss.elastic.co/t/how-to-concatenate-time-fields-and-generate-new-timestamp/213012)
**Author:** [@Bhaskar3184](https://discuss.elastic.co/u/Bhaskar3184)\ **Replies:** 0\ **Last updated:** [December 25, 2019, 10:23am UTC](https://discuss.elastic.co/t/how-to-concatenate-time-fields-and-generate-new-timestamp/213012 "2019-12-25T10:23:20Z")
Hi, I am new to elk, can someone help me in adding new timestamp by concatenating multiple time fields. Expected Output : Time totalfiles totalclean notscanned mailid deleted cleaned Dec 25, 2019 @ 15:42:23.815 333 … --- ## [Logstash to logstash communication without ssl?](https://discuss.elastic.co/t/logstash-to-logstash-communication-without-ssl/212947)
**Author:** [@Riddhesh](https://discuss.elastic.co/u/Riddhesh)\ **Replies:** 0\ **Last updated:** [December 24, 2019, 11:47am UTC](https://discuss.elastic.co/t/logstash-to-logstash-communication-without-ssl/212947 "2019-12-24T11:47:58Z")
Is there a way I can setup logstash to logstash communication without configuring ssl? I already have logstash deployed without any ssl configuration with beats input. If possible I would like to deploy client LS with l… --- ## [Scientific notation convert](https://discuss.elastic.co/t/scientific-notation-convert/213002)
**Author:** [@CCH0124](https://discuss.elastic.co/u/CCH0124)\ **Replies:** 0\ **Last updated:** [December 25, 2019, 8:40am UTC](https://discuss.elastic.co/t/scientific-notation-convert/213002 "2019-12-25T08:40:08Z")
How to convert scientific notation to float in CSV file? --- ## [Logstash with Redis Input plugin using Redis Streams](https://discuss.elastic.co/t/logstash-with-redis-input-plugin-using-redis-streams/212982)
**Author:** [@yha](https://discuss.elastic.co/u/yha)\ **Replies:** 0\ **Last updated:** [December 24, 2019, 8:57pm UTC](https://discuss.elastic.co/t/logstash-with-redis-input-plugin-using-redis-streams/212982 "2019-12-24T20:57:51Z")
It seems like LogStash Redis input plugin doesn't support Redis Streams (Only Redis pubsub is supported ?), is there way to read message from a stream of Redis Streams? tried like below but did not work input { red… --- ## [Logstash - can I log file owner?](https://discuss.elastic.co/t/logstash-can-i-log-file-owner/212951)
**Author:** [@danielp](https://discuss.elastic.co/u/danielp)\ **Replies:** 0\ **Last updated:** [December 24, 2019, 12:00pm UTC](https://discuss.elastic.co/t/logstash-can-i-log-file-owner/212951 "2019-12-24T12:00:03Z")
Hi, I'd like to add file owner id to my logs. Is it possible to read it from file properties with Logstash? Thanks, Daniel --- ## [Problem in filebeat-\>logstash-\>elasticsearch](https://discuss.elastic.co/t/problem-in-filebeat-logstash-elasticsearch/212938)
**Author:** [@andishe](https://discuss.elastic.co/u/andishe)\ **Replies:** 0\ **Last updated:** [December 24, 2019, 10:02am UTC](https://discuss.elastic.co/t/problem-in-filebeat-logstash-elasticsearch/212938 "2019-12-24T10:02:06Z")
Hi, I have set filebeat to read a text file(inside the file i have typed "hello world") from a location, then sends it to output.logstash and then i set the logstash to read from filebeat and without defining any filter… --- ## [Logstash 7.5 with SSL giving SSLV3\_ALERT\_BAD\_CERTIFICATE](https://discuss.elastic.co/t/logstash-7-5-with-ssl-giving-sslv3-alert-bad-certificate/212512)
**Author:** [@alesanchez](https://discuss.elastic.co/u/alesanchez)\ **Replies:** 3\ **Last updated:** [December 24, 2019, 8:32am UTC](https://discuss.elastic.co/t/logstash-7-5-with-ssl-giving-sslv3-alert-bad-certificate/212512 "2019-12-24T08:32:50Z")
Hi everyone, I know that there are a couple of topics regarding this problem. I reviewed them but there are for older logstash versions or the solution doesn't work for me. I have the following logstash configuration: … --- ## [Snow API not working with logstash](https://discuss.elastic.co/t/snow-api-not-working-with-logstash/212910)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 0\ **Last updated:** [December 24, 2019, 4:00am UTC](https://discuss.elastic.co/t/snow-api-not-working-with-logstash/212910 "2019-12-24T04:00:23Z")
Hi, Im trying to query a snow API with the below configuration in Logstash 7.2.0, input { http\_poller { urls =\> { snowinc =\> { url =\> "https://l.service-now.com/api/now/table/incident?sysparm\_limit=10" … --- ## [Incremental value in document IDs](https://discuss.elastic.co/t/incremental-value-in-document-ids/212810)
**Author:** [@psteh](https://discuss.elastic.co/u/psteh)\ **Replies:** 2\ **Last updated:** [December 24, 2019, 2:24am UTC](https://discuss.elastic.co/t/incremental-value-in-document-ids/212810 "2019-12-24T02:24:07Z")
Greetings, How do I have an incrementing field (counter) on document\_id if I am using JDBC input? The reason is that my statement is using left join on multiple tables and have duplicate of IDs. Using IDs for document\_… --- ## [Parsing firewall logs in logstash](https://discuss.elastic.co/t/parsing-firewall-logs-in-logstash/212786)
**Author:** [@heinrich](https://discuss.elastic.co/u/heinrich)\ **Replies:** 6\ **Last updated:** [December 23, 2019, 9:45pm UTC](https://discuss.elastic.co/t/parsing-firewall-logs-in-logstash/212786 "2019-12-23T21:45:44Z")
Hi all, I'm struggling parsing sophos utm firewall logs to extract data from the message field from filebeat. I'm new to grok and kv filters and require some help please. After some digging i have come across suggestion… --- ## [EPS by single tcp session](https://discuss.elastic.co/t/eps-by-single-tcp-session/212754)
**Author:** [@artem.n](https://discuss.elastic.co/u/artem.n)\ **Replies:** 1\ **Last updated:** [December 23, 2019, 9:24pm UTC](https://discuss.elastic.co/t/eps-by-single-tcp-session/212754 "2019-12-23T21:24:37Z")
I send logs to logstash using input syslog tcp. I can not overcome the EPS limit for one TCP session (2500 eps). Each TCP session provides a linear increase in EPS. The load on the processor is minimal, the sysctl setti… --- ## [Logstash version for multiple pipelines](https://discuss.elastic.co/t/logstash-version-for-multiple-pipelines/212875)
**Author:** [@mtudisco](https://discuss.elastic.co/u/mtudisco)\ **Replies:** 1\ **Last updated:** [December 23, 2019, 6:41pm UTC](https://discuss.elastic.co/t/logstash-version-for-multiple-pipelines/212875 "2019-12-23T18:41:52Z")
Hi, I'd like to know if the option to have multiple pipelines has always existed or in what logstash version it started. I have a logstash 2.2.4 in a production machine and i'd like to know if i can configure multiple … --- ## [Best way to parse lap data to make it sortable and filterable](https://discuss.elastic.co/t/best-way-to-parse-lap-data-to-make-it-sortable-and-filterable/211201)
**Author:** [@562uned](https://discuss.elastic.co/u/562uned)\ **Replies:** 1\ **Last updated:** [December 23, 2019, 3:46pm UTC](https://discuss.elastic.co/t/best-way-to-parse-lap-data-to-make-it-sortable-and-filterable/211201 "2019-12-23T15:46:47Z")
Hello, So i have my whole pipeline working great but there is one data field that im trying to figure out the best way to parse to make it sort able / aggregatable. The data comes in like 20/5:10.080. I had thought abou… --- ## [Logstash & VirusTotal lookup](https://discuss.elastic.co/t/logstash-virustotal-lookup/212858)
**Author:** [@gamarue](https://discuss.elastic.co/u/gamarue)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 3:05pm UTC](https://discuss.elastic.co/t/logstash-virustotal-lookup/212858 "2019-12-23T15:05:09Z")
Hi, We're currently building an ELK instance which has the purpose to nicely show forensic level data by reading data from a file and outputting to an elasticsearch instance. In that data which we have, there's a subse… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=426) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=428)