# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=465 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 466 --- ## [Add the target field](https://discuss.elastic.co/t/add-the-target-field/202869)
**Author:** [@Vijayakumar\_Kannan](https://discuss.elastic.co/u/Vijayakumar_Kannan)\ **Replies:** 0\ **Last updated:** [October 9, 2019, 3:44pm UTC](https://discuss.elastic.co/t/add-the-target-field/202869 "2019-10-09T15:44:04Z")
Dear All, My event is parsed against multiple patterns and then it generates fields . number of fields change against matched pattern. My aim is to add the target for all those random fields. Current src: 127.0.0.1 ds… --- ## [Logstash grok problem with elastic](https://discuss.elastic.co/t/logstash-grok-problem-with-elastic/202854)
**Author:** [@Marta\_Zagrajek](https://discuss.elastic.co/u/Marta_Zagrajek)\ **Replies:** 2\ **Last updated:** [October 9, 2019, 3:43pm UTC](https://discuss.elastic.co/t/logstash-grok-problem-with-elastic/202854 "2019-10-09T15:43:55Z")
I have apache logs like that: 188.122.20.100 \[09/Oct/2019:14:00:02 +0200\] "POST /something/template/ HTTP/1.1" 200 286 "url.html" "Mozilla/5.0 (iPhone; CPU iPhone OS 13\_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like… --- ## [Rewriting the IP Header before Sending](https://discuss.elastic.co/t/rewriting-the-ip-header-before-sending/202852)
**Author:** [@bnason](https://discuss.elastic.co/u/bnason)\ **Replies:** 2\ **Last updated:** [October 9, 2019, 3:37pm UTC](https://discuss.elastic.co/t/rewriting-the-ip-header-before-sending/202852 "2019-10-09T15:37:42Z")
I have a situation where I want to send syslog data (i.e. from a firewall) to two different logging environments (Elasticsearch and QRadar). If I send the original syslog data (from the firewall) to Logstash, can it rewr… --- ## [Dynamic input Http\_poller url](https://discuss.elastic.co/t/dynamic-input-http-poller-url/201928)
**Author:** [@fernrguez](https://discuss.elastic.co/u/fernrguez)\ **Replies:** 4\ **Last updated:** [October 9, 2019, 3:07pm UTC](https://discuss.elastic.co/t/dynamic-input-http-poller-url/201928 "2019-10-09T15:07:03Z")
Hi all, I have a pipeline that executes every minute, at this moment I'm using the http\_poller with a Get operation that, without filter, returns the first 100 in the last 24h. It is possible to add filters on the Url … --- ## [Loading data from a stored procedure](https://discuss.elastic.co/t/loading-data-from-a-stored-procedure/202860)
**Author:** [@DottorP](https://discuss.elastic.co/u/DottorP)\ **Replies:** 0\ **Last updated:** [October 9, 2019, 2:55pm UTC](https://discuss.elastic.co/t/loading-data-from-a-stored-procedure/202860 "2019-10-09T14:55:38Z")
I'm trying to load data from Oracle into my index. I need to use a stored procedure, which uses a cursor to iterate over data. My SQL cript begins with a declaration section, in which I declare the cursor and the tempo… --- ## [How can i send multiple json messages to logstash using http plugin?](https://discuss.elastic.co/t/how-can-i-send-multiple-json-messages-to-logstash-using-http-plugin/202592)
**Author:** [@rajkumar.m](https://discuss.elastic.co/u/rajkumar.m)\ **Replies:** 1\ **Last updated:** [October 9, 2019, 2:44pm UTC](https://discuss.elastic.co/t/how-can-i-send-multiple-json-messages-to-logstash-using-http-plugin/202592 "2019-10-09T14:44:01Z")
I am trying to send multiple json messages to logstash using http plugin, but i could see only the first message on kibana. please help. below is the my plugin. input { http { port =\> XXXX } } I am trying to pus… --- ## [Jdbc\_streaming named parameters](https://discuss.elastic.co/t/jdbc-streaming-named-parameters/202831)
**Author:** [@cerutidavide](https://discuss.elastic.co/u/cerutidavide)\ **Replies:** 0\ **Last updated:** [October 9, 2019, 12:46pm UTC](https://discuss.elastic.co/t/jdbc-streaming-named-parameters/202831 "2019-10-09T12:46:22Z")
I am pretty new to ELK, I have a problem with jdbc\_streaming filter plugin. It seems that parameters is not properly set and my query doesn't work. filter { jdbc\_streaming { jdbc\_user =\> "" jdbc\_password =\> "" jdbc… --- ## [Logstash stopped unexpectedly](https://discuss.elastic.co/t/logstash-stopped-unexpectedly/202823)
**Author:** [@Suresh\_Pal](https://discuss.elastic.co/u/Suresh_Pal)\ **Replies:** 0\ **Last updated:** [October 9, 2019, 11:41am UTC](https://discuss.elastic.co/t/logstash-stopped-unexpectedly/202823 "2019-10-09T11:41:21Z")
Hi team, My logstash service is stopping again and again after few time, giving below error. warning: thread "\[main\]\>worker1" terminated with exception (report\_on\_exception is true): java.lang.InterruptedException at… --- ## [Removing Logstash logs from syslog](https://discuss.elastic.co/t/removing-logstash-logs-from-syslog/202466)
**Author:** [@sud0](https://discuss.elastic.co/u/sud0)\ **Replies:** 5\ **Last updated:** [October 8, 2019, 11:35pm UTC](https://discuss.elastic.co/t/removing-logstash-logs-from-syslog/202466 "2019-10-08T23:35:20Z")
Hi. My logstash is sending logs to /var/log/syslog, which is causing the disk to fill up pretty quick. How can I disable it? My logstash.yml: log.level: info path.logs: /var/log/logstash syslog: root@elk:/etc/logs… --- ## [Logstash running and failing w/o apparent errors in logs on Docker/Fargate](https://discuss.elastic.co/t/logstash-running-and-failing-w-o-apparent-errors-in-logs-on-docker-fargate/202726)
**Author:** [@davewolf](https://discuss.elastic.co/u/davewolf)\ **Replies:** 1\ **Last updated:** [October 8, 2019, 11:24pm UTC](https://discuss.elastic.co/t/logstash-running-and-failing-w-o-apparent-errors-in-logs-on-docker-fargate/202726 "2019-10-08T23:24:36Z")
After resolving my previous errors the logstash application is starting successfully, connecting with Elasticsearch on Elastic Cloud, validating the licenses successfully, starting the pipelines, and metrics, and success… --- ## [Logstash read all CSV](https://discuss.elastic.co/t/logstash-read-all-csv/202728)
**Author:** [@SirJune](https://discuss.elastic.co/u/SirJune)\ **Replies:** 1\ **Last updated:** [October 8, 2019, 9:33pm UTC](https://discuss.elastic.co/t/logstash-read-all-csv/202728 "2019-10-08T21:33:17Z")
this one kind of confuses me. I have 4 logfiles, all CSV format. logfile13.csv logfile14.csv logfile15.csv logfile16.csv myconfig states: path =\> "/var/log/csv/logfile\*" start\_position =\> "beginning" sinced… --- ## [File updates in read mode](https://discuss.elastic.co/t/file-updates-in-read-mode/202730)
**Author:** [@kurbar](https://discuss.elastic.co/u/kurbar)\ **Replies:** 2\ **Last updated:** [October 8, 2019, 9:19pm UTC](https://discuss.elastic.co/t/file-updates-in-read-mode/202730 "2019-10-08T21:19:29Z")
I have input file plugin configured in read mode where I read the entire file and use multiline to concat it into a single event. input { file { path =\> "/absolute/path/to/xmls/\*\*/\*.xml" start\_position =\> "beg… --- ## [Logstash on Fargate - Cannot assign requested address](https://discuss.elastic.co/t/logstash-on-fargate-cannot-assign-requested-address/202589)
**Author:** [@davewolf](https://discuss.elastic.co/u/davewolf)\ **Replies:** 1\ **Last updated:** [October 8, 2019, 7:12pm UTC](https://discuss.elastic.co/t/logstash-on-fargate-cannot-assign-requested-address/202589 "2019-10-08T19:12:57Z")
Version: 7.4.0 Deployment: AWS ECS Fargate I'm getting this error: EADDRNOTAVAIL: Cannot assign requested address - bind - Cannot assign requested address Since I'm deploying to Fargate, I can't in advance know (or e… --- ## [Is there a way to report the datatype mapping from my data generator to Logstash?](https://discuss.elastic.co/t/is-there-a-way-to-report-the-datatype-mapping-from-my-data-generator-to-logstash/202710)
**Author:** [@Bob2](https://discuss.elastic.co/u/Bob2)\ **Replies:** 0\ **Last updated:** [October 8, 2019, 6:05pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-report-the-datatype-mapping-from-my-data-generator-to-logstash/202710 "2019-10-08T18:05:14Z")
Hello, Short question: Is there a way to report the datatype mapping from my data generator to Logstash? I don't want to use grok on Logstash since my data generator already knows the datatype and I don't want the need… --- ## [Quick question on scheduler for Logstash](https://discuss.elastic.co/t/quick-question-on-scheduler-for-logstash/202708)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 0\ **Last updated:** [October 8, 2019, 5:43pm UTC](https://discuss.elastic.co/t/quick-question-on-scheduler-for-logstash/202708 "2019-10-08T17:43:17Z")
When using a scheduler on logstash input plugins (elasticsearch, or http\_poller), if you run every 2 weeks, or every 20 mins, when does the first one run? Does it run in 2 weeks, or it runs now and then in 2 weeks? --- ## [Unable to get XPath values in XML filter](https://discuss.elastic.co/t/unable-to-get-xpath-values-in-xml-filter/202675)
**Author:** [@kurbar](https://discuss.elastic.co/u/kurbar)\ **Replies:** 1\ **Last updated:** [October 8, 2019, 5:20pm UTC](https://discuss.elastic.co/t/unable-to-get-xpath-values-in-xml-filter/202675 "2019-10-08T17:20:49Z")
Hello, I started working on Logstash and Elasticsearch and my first task was trying to get XML files indexed by Elasticsearch. For that I have a File input plugin that reads whole XML files XML filter plugin which cr… --- ## [How to access third party rest api using http\_poller,http filter and ruby filter using basic authentication,header and body](https://discuss.elastic.co/t/how-to-access-third-party-rest-api-using-http-poller-http-filter-and-ruby-filter-using-basic-authentication-header-and-body/202013)
**Author:** [@saroja](https://discuss.elastic.co/u/saroja)\ **Replies:** 3\ **Last updated:** [October 8, 2019, 3:45pm UTC](https://discuss.elastic.co/t/how-to-access-third-party-rest-api-using-http-poller-http-filter-and-ruby-filter-using-basic-authentication-header-and-body/202013 "2019-10-08T15:45:22Z")
logstash version : 6.3.1 curl -k -v -i -u 'API Key for V3':'API Secret for V3' 'https://api.com/v3/token' -d 'grant\_type=password&username=&password=\< Password\>&scope=\< Area UUID\>' Using above curl command I am able to… --- ## [Logstash syslog UDP output](https://discuss.elastic.co/t/logstash-syslog-udp-output/202698)
**Author:** [@toosaman](https://discuss.elastic.co/u/toosaman)\ **Replies:** 0\ **Last updated:** [October 8, 2019, 3:39pm UTC](https://discuss.elastic.co/t/logstash-syslog-udp-output/202698 "2019-10-08T15:39:50Z")
Hello. I'm trying to forward syslog messages to my ES cluster and UDP output to external server at the same time. But, for UDP output I don't want to change its contents. For example: now, logstash insert timestamp field… --- ## ["Error: com.mysql.jdbc.Driver not loaded. Are you sure you've included the correct jdbc driver in :jdbc\_driver\_library?"](https://discuss.elastic.co/t/error-com-mysql-jdbc-driver-not-loaded-are-you-sure-youve-included-the-correct-jdbc-driver-in-jdbc-driver-library/202567)
**Author:** [@happycampo](https://discuss.elastic.co/u/happycampo)\ **Replies:** 3\ **Last updated:** [October 8, 2019, 2:50pm UTC](https://discuss.elastic.co/t/error-com-mysql-jdbc-driver-not-loaded-are-you-sure-youve-included-the-correct-jdbc-driver-in-jdbc-driver-library/202567 "2019-10-08T14:50:00Z")
Hello, I'm going in circles and I'm having no luck getting the mysql-connector to work with 7.3.1. Now I'm using two separate Docker code bases but the method should be the same I have a working model with 6.5.1 using … --- ## [Logstash cloud central pipeline fails with configuration issue](https://discuss.elastic.co/t/logstash-cloud-central-pipeline-fails-with-configuration-issue/202490)
**Author:** [@benoit1](https://discuss.elastic.co/u/benoit1)\ **Replies:** 1\ **Last updated:** [October 8, 2019, 1:31pm UTC](https://discuss.elastic.co/t/logstash-cloud-central-pipeline-fails-with-configuration-issue/202490 "2019-10-08T13:31:39Z")
Hi all, I'm testing an Azure Cloud deployement (Western Europe) and try to use the Kibana centralized Pipelines feature. I have a local Logstash instance that I want to register on my Cloud deployment. Local Logstash … --- ## [When run logstash using the service command it fail to send data to aws elasticsearch](https://discuss.elastic.co/t/when-run-logstash-using-the-service-command-it-fail-to-send-data-to-aws-elasticsearch/202387)
**Author:** [@Marcos\_Silva](https://discuss.elastic.co/u/Marcos_Silva)\ **Replies:** 1\ **Last updated:** [October 8, 2019, 1:17pm UTC](https://discuss.elastic.co/t/when-run-logstash-using-the-service-command-it-fail-to-send-data-to-aws-elasticsearch/202387 "2019-10-08T13:17:11Z")
Hello, I'm having this issue: When I run: sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/apache.conf works perfectly and logstash send logs to AWS ES. When I run as a service using sudo initctl logsta… --- ## [Merge and transform more than one json files to elasticsearch from logstash](https://discuss.elastic.co/t/merge-and-transform-more-than-one-json-files-to-elasticsearch-from-logstash/202660)
**Author:** [@namali](https://discuss.elastic.co/u/namali)\ **Replies:** 0\ **Last updated:** [October 8, 2019, 12:01pm UTC](https://discuss.elastic.co/t/merge-and-transform-more-than-one-json-files-to-elasticsearch-from-logstash/202660 "2019-10-08T12:01:27Z")
I try from translate plugin, but didnt work. translate { destination =\> "\[@metadata\]\[city\]" dictionary\_path =\> "/home/namalie/elk/hotelspro\_v2/city.json" field =\> "citycode" } dissect { mapping =\> { "\[@metadata\]\[de… --- ## [Having multiple parsing (dissect) conditions chained](https://discuss.elastic.co/t/having-multiple-parsing-dissect-conditions-chained/202570)
**Author:** [@Vincent\_Maury](https://discuss.elastic.co/u/Vincent_Maury)\ **Replies:** 2\ **Last updated:** [October 8, 2019, 6:47am UTC](https://discuss.elastic.co/t/having-multiple-parsing-dissect-conditions-chained/202570 "2019-10-08T06:47:15Z")
Hi I'm using Logstash 7.4. I would like to try dissecting my logs (in this example apache logs) with multiple patterns, each having possibly its own mutations (renaming, formatting, etc). I thought about the configura… --- ## [SQS Plugin](https://discuss.elastic.co/t/sqs-plugin/202604)
**Author:** [@Rahul\_Kumar4](https://discuss.elastic.co/u/Rahul_Kumar4)\ **Replies:** 0\ **Last updated:** [October 8, 2019, 12:14am UTC](https://discuss.elastic.co/t/sqs-plugin/202604 "2019-10-08T00:14:35Z")
Hi, Does the Logstash SQS Plugin automatically deletes the message in the queue after consuming it? Where can this behavior (if it exits) be changed? Thanks Rahul --- ## [Shipping Postfix Logs to Elasticsearch V7.3](https://discuss.elastic.co/t/shipping-postfix-logs-to-elasticsearch-v7-3/202602)
**Author:** [@a.gavric](https://discuss.elastic.co/u/a.gavric)\ **Replies:** 0\ **Last updated:** [October 7, 2019, 11:23pm UTC](https://discuss.elastic.co/t/shipping-postfix-logs-to-elasticsearch-v7-3/202602 "2019-10-07T23:23:25Z")
Hello Everyone, we have been trying to find a solution to ship and parse postfix logs with Elastic stack 7.3, and have attempted several solutions in the last few days and have ran out of options. Solutions attempted: … --- ## [S3 output uploading failed BadDigest using gzip encoding](https://discuss.elastic.co/t/s3-output-uploading-failed-baddigest-using-gzip-encoding/201433)
**Author:** [@tomleb](https://discuss.elastic.co/u/tomleb)\ **Replies:** 1\ **Last updated:** [October 7, 2019, 6:21pm UTC](https://discuss.elastic.co/t/s3-output-uploading-failed-baddigest-using-gzip-encoding/201433 "2019-10-07T18:21:06Z")
Hi, I am trying to setup a logstash output using the s3 output plugin. I am getting the following error: \[2019-09-27T13:55:57,651\]\[DEBUG\]\[logstash.outputs.s3 \] Periodic check for stale files \[2019-09-27T13:56:05,… --- ## [Replace special characters in csv header](https://discuss.elastic.co/t/replace-special-characters-in-csv-header/202571)
**Author:** [@jarvis5181](https://discuss.elastic.co/u/jarvis5181)\ **Replies:** 1\ **Last updated:** [October 7, 2019, 5:05pm UTC](https://discuss.elastic.co/t/replace-special-characters-in-csv-header/202571 "2019-10-07T17:05:19Z")
Hello, I have a csv file with a header row which contains the special characters "\[" and "\]" in few of the column names. Logstash issues the exception "#\ **Author:** [@jsj](https://discuss.elastic.co/u/jsj)\ **Replies:** 3\ **Last updated:** [October 7, 2019, 4:12pm UTC](https://discuss.elastic.co/t/parsing-nested-xml-with-logstash/202530 "2019-10-07T16:12:19Z") Hi, I'm parsing xml-data from endpoint system with logstash and i'm not sure how parse nested data from xml; Here's the sample data & logstash config \ \2… --- ## [Failed to get Json parse failure count, stats, pipeline, plugin](https://discuss.elastic.co/t/failed-to-get-json-parse-failure-count-stats-pipeline-plugin/200166)
**Author:** [@sunfriendli](https://discuss.elastic.co/u/sunfriendli)\ **Replies:** 1\ **Last updated:** [October 7, 2019, 3:50pm UTC](https://discuss.elastic.co/t/failed-to-get-json-parse-failure-count-stats-pipeline-plugin/200166 "2019-10-07T15:50:36Z")
Version: 7.3.2 Operating System: Ubuntu 18.04.2 LTS Hi all, I have a problem with failed to get json parse failure count in stats/pipelines. In my env, input comes from filebeat, when json parsing data failed due to i… --- ## [Logstash doesn't want to split my CEF events](https://discuss.elastic.co/t/logstash-doesnt-want-to-split-my-cef-events/202562)
**Author:** [@nico88](https://discuss.elastic.co/u/nico88)\ **Replies:** 0\ **Last updated:** [October 7, 2019, 3:50pm UTC](https://discuss.elastic.co/t/logstash-doesnt-want-to-split-my-cef-events/202562 "2019-10-07T15:50:05Z")
Hello, I try to parse a CEF event with logstash with this mutate: mutate { split =\> {"message" =\> "|"} add\_field =\> { "cef\_sig\_name" =\> "%{message\[5\]}" } add\_field =\> { "cef\_extension" =\> "%{message\[7\]}"} } I get … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=464) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=466)