# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=260 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 261 --- ## [Logstash doesn't work on windows](https://discuss.elastic.co/t/logstash-doesnt-work-on-windows/262506)
**Author:** [@l418](https://discuss.elastic.co/u/l418)\ **Replies:** 4\ **Last updated:** [January 28, 2021, 3:03pm UTC](https://discuss.elastic.co/t/logstash-doesnt-work-on-windows/262506 "2021-01-28T15:03:52Z")
Hello. Trying to populate simple elastic cluster from demo using windows Your logstash windows implementation seems broken completely file { path =\> "/var/log/apache2/access.log" start\_position =\> "beginning" since… --- ## [Trying to pass mongodb id as elasticsearch id](https://discuss.elastic.co/t/trying-to-pass-mongodb-id-as-elasticsearch-id/262449)
**Author:** [@fidsamurai](https://discuss.elastic.co/u/fidsamurai)\ **Replies:** 1\ **Last updated:** [January 28, 2021, 6:40am UTC](https://discuss.elastic.co/t/trying-to-pass-mongodb-id-as-elasticsearch-id/262449 "2021-01-28T06:40:37Z")
Logstash 7.8 We have a case where we collect logs from a mobile app using parse-sdk. The data is sent to a MongoDB replicaset. Logstash is configured using the mongoschema jdbc driver to pull the data from Mongo and p… --- ## [\_jsonparsefailure on reading json](https://discuss.elastic.co/t/jsonparsefailure-on-reading-json/262335)
**Author:** [@logstash123](https://discuss.elastic.co/u/logstash123)\ **Replies:** 2\ **Last updated:** [January 27, 2021, 7:03pm UTC](https://discuss.elastic.co/t/jsonparsefailure-on-reading-json/262335 "2021-01-27T19:03:45Z")
Dear community, I need to parse a JSON of the following format: { "commitId": "xxxxxxx", "author": { "name": "", "email": "", "date": "" }, "committer": { "name": "", "email": "", "date"… --- ## [How to configure ELK to receive logs from multiple servers](https://discuss.elastic.co/t/how-to-configure-elk-to-receive-logs-from-multiple-servers/262383)
**Author:** [@huzaifa224](https://discuss.elastic.co/u/huzaifa224)\ **Replies:** 3\ **Last updated:** [January 27, 2021, 7:01pm UTC](https://discuss.elastic.co/t/how-to-configure-elk-to-receive-logs-from-multiple-servers/262383 "2021-01-27T19:01:13Z")
I have multiple servers in which filebeat is installed and sending their logs to ELK. In kibana multiple server's logs are showing on single logs page which is every complicated to identify which log is coming from whic… --- ## [Throttle filter is not throttling correctly](https://discuss.elastic.co/t/throttle-filter-is-not-throttling-correctly/262402)
**Author:** [@holobolo0815](https://discuss.elastic.co/u/holobolo0815)\ **Replies:** 0\ **Last updated:** [January 27, 2021, 5:32pm UTC](https://discuss.elastic.co/t/throttle-filter-is-not-throttling-correctly/262402 "2021-01-27T17:32:07Z")
With reference to https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861 When doing this: if \[srcip\] and \[dstip\] and \[dstport\] { throttle { # decorates event if OUTSIDE bounds key =\> "%{%filter\_ma… --- ## [Date Match is setting incorrect timestamp / not actually setting timestamp](https://discuss.elastic.co/t/date-match-is-setting-incorrect-timestamp-not-actually-setting-timestamp/262363)
**Author:** [@beirtipol](https://discuss.elastic.co/u/beirtipol)\ **Replies:** 5\ **Last updated:** [January 27, 2021, 5:57pm UTC](https://discuss.elastic.co/t/date-match-is-setting-incorrect-timestamp-not-actually-setting-timestamp/262363 "2021-01-27T17:57:24Z")
I have the following filter in my logstash pipeline: date { match =\> \[ "\[fields\]\[serverrequest\]\[starttime\]", "UNIX\_MS" \] tag\_on\_failure =\> \[ "serverrequest\_logstyle\_dateparsefailure" \] } However, th… --- ## [Issue with HTTP Poller : ConfigurationError: Invalid URL](https://discuss.elastic.co/t/issue-with-http-poller-configurationerror-invalid-url/262328)
**Author:** [@AmarKolhapuri](https://discuss.elastic.co/u/AmarKolhapuri)\ **Replies:** 2\ **Last updated:** [January 27, 2021, 5:51pm UTC](https://discuss.elastic.co/t/issue-with-http-poller-configurationerror-invalid-url/262328 "2021-01-27T17:51:52Z")
I am facing error with invalid URL for below http\_poller code. input { http\_poller{ urls =\> { test2 =\> { method =\> get user =\> "weblogic" password =\> "welcome1" urls =\> "http://weblogic:7001/management/wls/lates… --- ## [\[ Error parsing CSV \] : :exception=\>#\](https://discuss.elastic.co/t/error-parsing-csv-exception-csv-illegal-quoting-in-line-1/262370)
**Author:** [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\ **Replies:** 2\ **Last updated:** [January 27, 2021, 4:55pm UTC](https://discuss.elastic.co/t/error-parsing-csv-exception-csv-illegal-quoting-in-line-1/262370 "2021-01-27T16:55:12Z")
Hello, I am trying to parse some csv logs using logstash and I am getting this error: \[WARN \] 2021-01-27 14:05:31.217 \[\[main\]\>worker2\] csv - Error parsing csv {:field=\>"message", :source=\>"\\"2020-12-24 00:01:23\\", \\"29… --- ## [Grok pattern to extract the fields from the log file](https://discuss.elastic.co/t/grok-pattern-to-extract-the-fields-from-the-log-file/262326)
**Author:** [@shiv97](https://discuss.elastic.co/u/shiv97)\ **Replies:** 1\ **Last updated:** [January 27, 2021, 4:54pm UTC](https://discuss.elastic.co/t/grok-pattern-to-extract-the-fields-from-the-log-file/262326 "2021-01-27T16:54:43Z")
Hi Everyone, I am having one log file and I need to parse the message field into multiple subfields for that I need grok parser please look at the log file below:- message:{"level":"info","ts":1611311978.7658348,"calle… --- ## [Array values into single field](https://discuss.elastic.co/t/array-values-into-single-field/262348)
**Author:** [@Stevek](https://discuss.elastic.co/u/Stevek)\ **Replies:** 1\ **Last updated:** [January 27, 2021, 4:50pm UTC](https://discuss.elastic.co/t/array-values-into-single-field/262348 "2021-01-27T16:50:36Z")
Hello, I struggle to extract IPs from JSON, I need to extract all IPs from array/s(regardless of count) into one field. Below the part of JSON I want to extract data from: "source": { "explicit": { … --- ## [Logstash writer permissions](https://discuss.elastic.co/t/logstash-writer-permissions/262397)
**Author:** [@iccMe](https://discuss.elastic.co/u/iccMe)\ **Replies:** 0\ **Last updated:** [January 27, 2021, 4:49pm UTC](https://discuss.elastic.co/t/logstash-writer-permissions/262397 "2021-01-27T16:49:23Z")
Hi, Looking for advice. I have created a new logstash writer user and created the appropriate role with permissions as per: https://www.elastic.co/guide/en/logstash/7.3/ls-security.html for a managed ILM index. The onl… --- ## [@timestamp not match with the log date](https://discuss.elastic.co/t/timestamp-not-match-with-the-log-date/262272)
**Author:** [@Riyanka](https://discuss.elastic.co/u/Riyanka)\ **Replies:** 3\ **Last updated:** [January 27, 2021, 4:46pm UTC](https://discuss.elastic.co/t/timestamp-not-match-with-the-log-date/262272 "2021-01-27T16:46:34Z")
The timestamp does not match with the actual log time in kibana @timestamp January 26th 2021, 23:38:10.426 message \[2021-01-22 09:53:09\] php.CRITICAL: Uncaught Error: Call to a member function getName() on null… --- ## [Convert string to number with decimal](https://discuss.elastic.co/t/convert-string-to-number-with-decimal/262163)
**Author:** [@jayv](https://discuss.elastic.co/u/jayv)\ **Replies:** 2\ **Last updated:** [January 27, 2021, 4:42pm UTC](https://discuss.elastic.co/t/convert-string-to-number-with-decimal/262163 "2021-01-27T16:42:26Z")
Hi there, can anyone help me mutate a string into a decimal-formatted number for geo-points? For example, I have a CSV with fields containing: -115168598 ....which needs to be: -115.168598 My first attempt just gave m… --- ## [Oracle to Elastic search date conversion #logstash](https://discuss.elastic.co/t/oracle-to-elastic-search-date-conversion-logstash/262368)
**Author:** [@onkarborade](https://discuss.elastic.co/u/onkarborade)\ **Replies:** 1\ **Last updated:** [January 27, 2021, 4:29pm UTC](https://discuss.elastic.co/t/oracle-to-elastic-search-date-conversion-logstash/262368 "2021-01-27T16:29:28Z")
Hi, Trying to insert records from Oracle db to Elastic search using logstash I am getting default date format of field from oracle database is 2021-01-27T12:39:01.867Z and I want to save a field in elastic search in … --- ## [Logstash randomly skipping files](https://discuss.elastic.co/t/logstash-randomly-skipping-files/262228)
**Author:** [@Rutger2000](https://discuss.elastic.co/u/Rutger2000)\ **Replies:** 3\ **Last updated:** [January 27, 2021, 4:25pm UTC](https://discuss.elastic.co/t/logstash-randomly-skipping-files/262228 "2021-01-27T16:25:38Z")
I'm using logstash to ingest data into Elasticsearch and view it in Kibana. Every hour I receive multiple files in a folder which then are processed by logstash. We notice that logstash sometimes skips a file, as far as … --- ## [Logstash JOSN parsing faile](https://discuss.elastic.co/t/logstash-josn-parsing-faile/262395)
**Author:** [@motyha](https://discuss.elastic.co/u/motyha)\ **Replies:** 0\ **Last updated:** [January 27, 2021, 4:15pm UTC](https://discuss.elastic.co/t/logstash-josn-parsing-faile/262395 "2021-01-27T16:15:24Z")
Hi i have some issue with JSON parsing that i create by Python the JSON file and CONF file uploaded the problem that the json file split the raws in the message {...} how do i make logstash parse the JSON that eve… --- ## [Boost throughput of Kafka input](https://discuss.elastic.co/t/boost-throughput-of-kafka-input/261958)
**Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 5\ **Last updated:** [January 27, 2021, 2:19pm UTC](https://discuss.elastic.co/t/boost-throughput-of-kafka-input/261958 "2021-01-27T14:19:38Z")
Hi, I've tried to look up every available documentation, example, blog post that's available on the net but I couldn't find anything useful on how to speed up ingesting from Kafka. Logstash barely uses any resources (10… --- ## [Not able to connect elastic search from logstash](https://discuss.elastic.co/t/not-able-to-connect-elastic-search-from-logstash/262309)
**Author:** [@Madhu05281](https://discuss.elastic.co/u/Madhu05281)\ **Replies:** 2\ **Last updated:** [January 27, 2021, 6:20am UTC](https://discuss.elastic.co/t/not-able-to-connect-elastic-search-from-logstash/262309 "2021-01-27T06:20:50Z")
kind: ConfigMap metadata: name: logstash-config namespace: ns-erp data: logstash.conf: |- input { beats { port =\> "9600" } } output { # You can uncomment this line to inves… --- ## [Logstash import two columns](https://discuss.elastic.co/t/logstash-import-two-columns/262236)
**Author:** [@Farid\_N](https://discuss.elastic.co/u/Farid_N)\ **Replies:** 3\ **Last updated:** [January 27, 2021, 6:08am UTC](https://discuss.elastic.co/t/logstash-import-two-columns/262236 "2021-01-27T06:08:46Z")
Hi I use JDBC driver to connect Logstash with a data base and run a SP in statement. the result of SP is one number(integer) and two columns. I want to send these into one log in Elasticsearch Could you please tell m… --- ## [Repo for zip/tar logstash plugins](https://discuss.elastic.co/t/repo-for-zip-tar-logstash-plugins/262166)
**Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 3\ **Last updated:** [January 26, 2021, 6:16pm UTC](https://discuss.elastic.co/t/repo-for-zip-tar-logstash-plugins/262166 "2021-01-26T18:16:30Z")
Hello, I am looking for a link to a repo or some location that can provide me logstash plugin zip/tar files. My servers do not have internet connection/access. So I am looking to install logstash plugins offline. However… --- ## [No error in logstash. But no data in kibana](https://discuss.elastic.co/t/no-error-in-logstash-but-no-data-in-kibana/262143)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 4\ **Last updated:** [January 26, 2021, 5:08pm UTC](https://discuss.elastic.co/t/no-error-in-logstash-but-no-data-in-kibana/262143 "2021-01-26T17:08:35Z")
I don't see any indices created in amazon ES. architecture filebeat-\> logstash-\> amazon ES. filebeat conf filebeat.inputs: - type: log paths: - /var/test.log fields: tags: test environment: uflek … --- ## [Input command not executing](https://discuss.elastic.co/t/input-command-not-executing/262137)
**Author:** [@Christophe\_Dumont](https://discuss.elastic.co/u/Christophe_Dumont)\ **Replies:** 3\ **Last updated:** [January 26, 2021, 4:48pm UTC](https://discuss.elastic.co/t/input-command-not-executing/262137 "2021-01-26T16:48:12Z")
Hello, I'm trying to run a command every 10 seconds but it's not working. Here's my conf file : input { exec { command =\> "/home/xxx/CICFlow/bin/cfm \> '/tmp/logstash.log'" interval =\> 10 } } --- ## [Logstah tcp input plugin](https://discuss.elastic.co/t/logstah-tcp-input-plugin/262116)
**Author:** [@gongon](https://discuss.elastic.co/u/gongon)\ **Replies:** 3\ **Last updated:** [January 26, 2021, 1:18pm UTC](https://discuss.elastic.co/t/logstah-tcp-input-plugin/262116 "2021-01-26T13:18:20Z")
Hello, When I use the tcp input plugin in mode server, it works. But it does not work in mode client.It can connect to the server,but does not receive data.It's about a webserver : I try to extract data from a database … --- ## [Logstash pipeline service failed with X-Pack](https://discuss.elastic.co/t/logstash-pipeline-service-failed-with-x-pack/262119)
**Author:** [@esijati](https://discuss.elastic.co/u/esijati)\ **Replies:** 1\ **Last updated:** [January 26, 2021, 12:58pm UTC](https://discuss.elastic.co/t/logstash-pipeline-service-failed-with-x-pack/262119 "2021-01-26T12:58:29Z")
Hi, I have created a logstash service (tarball) and enabled the x-pack management. But getting the below error while running the service. \[ERROR\]\[logstash.config.sourceloader\] Could not fetch all the sources {:exceptio… --- ## [S3 output event per file](https://discuss.elastic.co/t/s3-output-event-per-file/262037)
**Author:** [@Sunflower](https://discuss.elastic.co/u/Sunflower)\ **Replies:** 1\ **Last updated:** [January 26, 2021, 12:23pm UTC](https://discuss.elastic.co/t/s3-output-event-per-file/262037 "2021-01-26T12:23:56Z")
Hi, I'm using the S3 output plugin and I noticed that I'm receiving files that include multiple events per file. How can I create a file for each event? I saw the option to limit the file size but I'm not sure it is g… --- ## [Create a view mysql with logstash](https://discuss.elastic.co/t/create-a-view-mysql-with-logstash/262223)
**Author:** [@MatteoM](https://discuss.elastic.co/u/MatteoM)\ **Replies:** 0\ **Last updated:** [January 26, 2021, 11:01am UTC](https://discuss.elastic.co/t/create-a-view-mysql-with-logstash/262223 "2021-01-26T11:01:02Z")
Hi there!, I was wondering if exists a way to create a view mysql through by the statement in the pipeline input jdbc. I need to create some views because these are involved in a query that I need it to save in a index… --- ## [Docker services stop working when loosing connection from elastic-logging-plugin to ElasticDB](https://discuss.elastic.co/t/docker-services-stop-working-when-loosing-connection-from-elastic-logging-plugin-to-elasticdb/262196)
**Author:** [@LeeAn](https://discuss.elastic.co/u/LeeAn)\ **Replies:** 0\ **Last updated:** [January 26, 2021, 6:38am UTC](https://discuss.elastic.co/t/docker-services-stop-working-when-loosing-connection-from-elastic-logging-plugin-to-elasticdb/262196 "2021-01-26T06:38:03Z")
Hi, Im using Elastic-logging-plugin to collect logs from my docker containers and send them to Kibana but sometime the connection to ElasticDB got lost for reasons like internal error from ElasticDB server or network p… --- ## [If condition Statement](https://discuss.elastic.co/t/if-condition-statement/262167)
**Author:** [@hzarrabi](https://discuss.elastic.co/u/hzarrabi)\ **Replies:** 2\ **Last updated:** [January 26, 2021, 2:53am UTC](https://discuss.elastic.co/t/if-condition-statement/262167 "2021-01-26T02:53:35Z")
Hi All, Could you please tell me what is wrong with the following IF condition if \[ DOCUMENT\_MODEL \] == "10000" { mutate { replace =\> { "DOCUMENT\_MODEL" =\> "INVOICE" } } } Even though the DOCUM… --- ## [Can't change timestamp of logstash to timestamp of log](https://discuss.elastic.co/t/cant-change-timestamp-of-logstash-to-timestamp-of-log/262134)
**Author:** [@vladik22](https://discuss.elastic.co/u/vladik22)\ **Replies:** 2\ **Last updated:** [January 25, 2021, 9:20pm UTC](https://discuss.elastic.co/t/cant-change-timestamp-of-logstash-to-timestamp-of-log/262134 "2021-01-25T21:20:33Z")
Can't replace timestamp of logstash with timestamp of log file csv and send to elasticsearch. in debugger all work. When I change output to elasticsearch it's write that is work but after I create index patern in kiban… --- ## [Logstash: Persistent Queue Behaviour](https://discuss.elastic.co/t/logstash-persistent-queue-behaviour/262141)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 3\ **Last updated:** [January 25, 2021, 5:03pm UTC](https://discuss.elastic.co/t/logstash-persistent-queue-behaviour/262141 "2021-01-25T17:03:15Z")
Hi, I've seen some behaviour with persistent queues that surprises me.I noticed this when throughput was capped on a logstash node when the EBS Burst Balance was exceeded for the volume of the logstash node in question.… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=259) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=261)