# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=897 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 898 --- ## [Logstach Starting issue!](https://discuss.elastic.co/t/logstach-starting-issue/102830)
**Author:** [@SJN8](https://discuss.elastic.co/u/SJN8)\ **Replies:** 14\ **Last updated:** [October 6, 2017, 10:14am UTC](https://discuss.elastic.co/t/logstach-starting-issue/102830 "2017-10-06T10:14:34Z")
Hi All, I have configured File beat and logstach for log management . I am able to start Filebeat but while starting getting below error , ##################################################### root@logstach\_host bin\]… --- ## [How to convert string to JSON?](https://discuss.elastic.co/t/how-to-convert-string-to-json/102845)
**Author:** [@iStepich](https://discuss.elastic.co/u/iStepich)\ **Replies:** 9\ **Last updated:** [October 6, 2017, 9:31am UTC](https://discuss.elastic.co/t/how-to-convert-string-to-json/102845 "2017-10-06T09:31:36Z")
My logstash configuration is: input { tcp{ codec =\> json\_lines {charset =\> "CP1251"} ... } } output { elasticsearch{...}} But there is a problem that I can recieve string to already mapped as object field. In t… --- ## [SQLSyntaxErrorException](https://discuss.elastic.co/t/sqlsyntaxerrorexception/102967)
**Author:** [@Harald\_Hodl](https://discuss.elastic.co/u/Harald_Hodl)\ **Replies:** 1\ **Last updated:** [October 6, 2017, 9:11am UTC](https://discuss.elastic.co/t/sqlsyntaxerrorexception/102967 "2017-10-06T09:11:46Z")
Hi, im facing an issue while running logstash in combination with mariadb. I have already figured out that the statement without quotation marks is working. logstash 5.6.2 Now my Question how can i remove the "", m… --- ## [Exploding http\_poller json line](https://discuss.elastic.co/t/exploding-http-poller-json-line/102975)
**Author:** [@Callahan](https://discuss.elastic.co/u/Callahan)\ **Replies:** 0\ **Last updated:** [October 6, 2017, 8:41am UTC](https://discuss.elastic.co/t/exploding-http-poller-json-line/102975 "2017-10-06T08:41:12Z")
Hi, So a provider has updated their REST API and changed the way the JSON is displayed/returned. The newer version now returns a line beginning with: {"tickets":\[{"cc\_emails":\[\],"fwd\_emails":\[\], This means all the dat… --- ## [Duplication data in Zabbix](https://discuss.elastic.co/t/duplication-data-in-zabbix/102922)
**Author:** [@vl.zemtsov](https://discuss.elastic.co/u/vl.zemtsov)\ **Replies:** 0\ **Last updated:** [October 5, 2017, 9:47pm UTC](https://discuss.elastic.co/t/duplication-data-in-zabbix/102922 "2017-10-05T21:47:01Z")
Hello. I have dir '/tmp/dataForZabbix/' In that dir i have many files, for example consider 1 file: '/tmp/dataForZabbix/testFile' In this file written lines, simultaneously, every 15 sec: ZabbixHost: \ \ **Author:** [@paul1243](https://discuss.elastic.co/u/paul1243)\ **Replies:** 1\ **Last updated:** [October 6, 2017, 7:06am UTC](https://discuss.elastic.co/t/logstash-offline-plugin-installation/102536 "2017-10-06T07:06:43Z") Hi - Am using Logstash-5.3.0, tried to create a offline plugin package and moved the ZIP file to my Virtual machine, which doesn't have any connection to the rubygems.org, and when I tried to install the package, but it… --- ## [Logstash-codec-netflow does not accept netflow packets coming from a different IP subnet](https://discuss.elastic.co/t/logstash-codec-netflow-does-not-accept-netflow-packets-coming-from-a-different-ip-subnet/102950)
**Author:** [@blueren](https://discuss.elastic.co/u/blueren)\ **Replies:** 0\ **Last updated:** [October 6, 2017, 6:43am UTC](https://discuss.elastic.co/t/logstash-codec-netflow-does-not-accept-netflow-packets-coming-from-a-different-ip-subnet/102950 "2017-10-06T06:43:21Z")
Hello. logstash-codec-netflow version: 3.6.0 Operating System: Ubuntu 16.04 Config File: All files pertaining to logstash config is located here. Issue: If the netflow collector and the machine running logstash are … --- ## [Logstash dying out during startup](https://discuss.elastic.co/t/logstash-dying-out-during-startup/102850)
**Author:** [@Zama\_Las](https://discuss.elastic.co/u/Zama_Las)\ **Replies:** 2\ **Last updated:** [October 6, 2017, 6:05am UTC](https://discuss.elastic.co/t/logstash-dying-out-during-startup/102850 "2017-10-06T06:05:28Z")
My logstash is dying out during startup Can see the following in log,....No errors obserbed. \[2017-10-05T14:18:33,230\]\[WARN \]\[logstash.outputs.elasticsearch\] Restored connection to ES instance {:url=\>"http://logstash\_s… --- ## [Extract document id into another variable in logstash](https://discuss.elastic.co/t/extract-document-id-into-another-variable-in-logstash/102643)
**Author:** [@raja\_gopal](https://discuss.elastic.co/u/raja_gopal)\ **Replies:** 6\ **Last updated:** [October 6, 2017, 5:56am UTC](https://discuss.elastic.co/t/extract-document-id-into-another-variable-in-logstash/102643 "2017-10-06T05:56:32Z")
I have a need to get document id for a particular condition. Get the document\_id from the document matching the query in elasticsearch input and assign it to a variable and later use that document id to delete a document… --- ## [Attempted to resurrect connection to dead ES instance, but got an error](https://discuss.elastic.co/t/attempted-to-resurrect-connection-to-dead-es-instance-but-got-an-error/102632)
**Author:** [@Adam\_Schappell](https://discuss.elastic.co/u/Adam_Schappell)\ **Replies:** 1\ **Last updated:** [October 6, 2017, 5:39am UTC](https://discuss.elastic.co/t/attempted-to-resurrect-connection-to-dead-es-instance-but-got-an-error/102632 "2017-10-06T05:39:11Z")
Hey all can someone help me please. I just upgraded to XPack and am having trouble getting Logstash back up and running. Here is the errors I am seeing in logstash-plain.log \[2017-10-03T19:58:14,092\]\[WARN \]\[logstash.out… --- ## [File input Path variable](https://discuss.elastic.co/t/file-input-path-variable/102907)
**Author:** [@CDR](https://discuss.elastic.co/u/CDR)\ **Replies:** 5\ **Last updated:** [October 6, 2017, 5:34am UTC](https://discuss.elastic.co/t/file-input-path-variable/102907 "2017-10-06T05:34:33Z")
My logstash.conf file is using the file input as such: file{ path =\> "${PWD}/data/\*\*/log.file" start\_position =\> beginning ignore\_older =\> 0 sincedb\_path =\> "${PWD}/software\_files/logstash-5.6.1/logstash\_use/null" … --- ## [Elapsed questions](https://discuss.elastic.co/t/elapsed-questions/102708)
**Author:** [@kiwi0217](https://discuss.elastic.co/u/kiwi0217)\ **Replies:** 1\ **Last updated:** [October 6, 2017, 5:33am UTC](https://discuss.elastic.co/t/elapsed-questions/102708 "2017-10-06T05:33:46Z")
i can count the elapsed time. but in kibana it just display the time count. i want to count time with difference user the log is: |2017-09-29-15:13:57|INFO|JupyterHub|User logged in: test| |2017-09-29-15:14:21|INFO|Ju… --- ## [Files not being pushed - even though the pipeline has been successfully created](https://discuss.elastic.co/t/files-not-being-pushed-even-though-the-pipeline-has-been-successfully-created/102750)
**Author:** [@naresh.komma](https://discuss.elastic.co/u/naresh.komma)\ **Replies:** 0\ **Last updated:** [October 4, 2017, 7:06pm UTC](https://discuss.elastic.co/t/files-not-being-pushed-even-though-the-pipeline-has-been-successfully-created/102750 "2017-10-04T19:06:08Z")
Hi, I am using logstash to push files from the ec2 instance (/media/ephemeral0/logs/yarn/\*.gz) to S3 folders. This instance is part of a spark cluster. However the same configuration (for logstash) works fine on the mas… --- ## [Logstash And InfluxDB configuration](https://discuss.elastic.co/t/logstash-and-influxdb-configuration/102769)
**Author:** [@Vishal\_Shah](https://discuss.elastic.co/u/Vishal_Shah)\ **Replies:** 1\ **Last updated:** [October 6, 2017, 4:44am UTC](https://discuss.elastic.co/t/logstash-and-influxdb-configuration/102769 "2017-10-06T04:44:53Z")
Hi I am trying to ingest the data collected by logstash into InfluxDB. Following are rows from the Log file: 2017-10-04 22:21:31 +0000 UTC | SourceNode | LogLevel | Module | ProcessStatus | ProcessName | DestinationNode … --- ## [Event count per second and](https://discuss.elastic.co/t/event-count-per-second-and/102903)
**Author:** [@mr\_chang7](https://discuss.elastic.co/u/mr_chang7)\ **Replies:** 2\ **Last updated:** [October 6, 2017, 2:39am UTC](https://discuss.elastic.co/t/event-count-per-second-and/102903 "2017-10-06T02:39:18Z")
Hi, I'd like to count the events coming in per second, and then want to decide something based on the count. How can I achieve this? I tired to count the events by using aggregate plugin per second, but as you could im… --- ## [Pipeline starts but data does not ingest - json or xml files](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538)
**Author:** [@cxbarbosa](https://discuss.elastic.co/u/cxbarbosa)\ **Replies:** 8\ **Last updated:** [October 6, 2017, 12:07am UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538 "2017-10-06T00:07:36Z")
I'm using API call to pull data into json docs and ingesting through logstash, but when I go to run my config file, I get this in log stash \[2017-09-15T16:08:02,579\]\[INFO \]\[logstash.pipeline \] Pipeline main start… --- ## [\[Logstash 5.6.2\] Attempted to resurrect connection to dead ES instance](https://discuss.elastic.co/t/logstash-5-6-2-attempted-to-resurrect-connection-to-dead-es-instance/102911)
**Author:** [@fernandofot](https://discuss.elastic.co/u/fernandofot)\ **Replies:** 1\ **Last updated:** [October 5, 2017, 11:30pm UTC](https://discuss.elastic.co/t/logstash-5-6-2-attempted-to-resurrect-connection-to-dead-es-instance/102911 "2017-10-05T23:30:11Z")
Hi Experts, I'm newbie in Elasticsearch. I installed ElasticSearch, Kibana and even Logstash. And there is a problem. I am using all components in version 5.6.2. I know that there are many topics about my issue, and bel… --- ## [Best way to parse the KV values in logs coming from winlog beat](https://discuss.elastic.co/t/best-way-to-parse-the-kv-values-in-logs-coming-from-winlog-beat/102918)
**Author:** [@shroh](https://discuss.elastic.co/u/shroh)\ **Replies:** 1\ **Last updated:** [October 5, 2017, 11:28pm UTC](https://discuss.elastic.co/t/best-way-to-parse-the-kv-values-in-logs-coming-from-winlog-beat/102918 "2017-10-05T23:28:08Z")
Hi experts, My windows event logs are coming through win log beat and have some key value pairs when i see the xml format of the logs. Is there a way i can parse these fields as key value pairs? --- ## [Unable to ingest events from ArcSight Smart Connector to Logstash ArcSight Module](https://discuss.elastic.co/t/unable-to-ingest-events-from-arcsight-smart-connector-to-logstash-arcsight-module/102919)
**Author:** [@descobar](https://discuss.elastic.co/u/descobar)\ **Replies:** 0\ **Last updated:** [October 5, 2017, 9:21pm UTC](https://discuss.elastic.co/t/unable-to-ingest-events-from-arcsight-smart-connector-to-logstash-arcsight-module/102919 "2017-10-05T21:21:11Z")
We have a 5 node ES (version 5.6.0) cluster running on RHEL 7. In addition, 3 of the nodes have Logstash and the remaining 2 nodes have Kibana. The X-pack plugin for Elasticsearch, Logstash and Kibana has been installe… --- ## [Not able to input data from github and create an index](https://discuss.elastic.co/t/not-able-to-input-data-from-github-and-create-an-index/102841)
**Author:** [@Aynkaran](https://discuss.elastic.co/u/Aynkaran)\ **Replies:** 1\ **Last updated:** [October 5, 2017, 7:46pm UTC](https://discuss.elastic.co/t/not-able-to-input-data-from-github-and-create-an-index/102841 "2017-10-05T19:46:18Z")
Using the "enable forwarding log feature in github" to forward syslog to my logstash server. And my task is to recieve the input from github and create index for the same in Kibana. Has anyone worked on this before. Ri… --- ## [Winlogbeat to Logstash](https://discuss.elastic.co/t/winlogbeat-to-logstash/102890)
**Author:** [@p3y](https://discuss.elastic.co/u/p3y)\ **Replies:** 2\ **Last updated:** [October 5, 2017, 7:06pm UTC](https://discuss.elastic.co/t/winlogbeat-to-logstash/102890 "2017-10-05T19:06:26Z")
Hello, I am very new to ELK technology. I am trying to push windows Event logs to the Logstash server but i am unable to do it. I am using this documentation: https://www.elastic.co/guide/en/beats/winlogbeat/5.0/winlog… --- ## [Is it possible to Grok out part of a string and then the whole string?](https://discuss.elastic.co/t/is-it-possible-to-grok-out-part-of-a-string-and-then-the-whole-string/102725)
**Author:** [@cchooks2](https://discuss.elastic.co/u/cchooks2)\ **Replies:** 2\ **Last updated:** [October 5, 2017, 6:41pm UTC](https://discuss.elastic.co/t/is-it-possible-to-grok-out-part-of-a-string-and-then-the-whole-string/102725 "2017-10-05T18:41:29Z")
Hello, I have the following request type: "GET /rest/api/latest/search?jql=project+%3D+%22THP%22+AND+type+%3D+%22Defect%22+order+by+updated+desc&startAt=0&fields=summary,updated HTTP/1.1" Is it possible to GROK out "G… --- ## [Avoiding duplicate records](https://discuss.elastic.co/t/avoiding-duplicate-records/102471)
**Author:** [@bgoldowsky](https://discuss.elastic.co/u/bgoldowsky)\ **Replies:** 3\ **Last updated:** [October 5, 2017, 6:40pm UTC](https://discuss.elastic.co/t/avoiding-duplicate-records/102471 "2017-10-05T18:40:41Z")
I am using the logstash JDBC input plugin to push new rows from a database query into Elasticsearch, updating any old items that have changed. I'd like to avoid duplicates, so I tried to use an "upsert" pattern: output… --- ## [Logstash v6.0 multi pipleline faining to run](https://discuss.elastic.co/t/logstash-v6-0-multi-pipleline-faining-to-run/102689)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 3\ **Last updated:** [October 5, 2017, 5:17pm UTC](https://discuss.elastic.co/t/logstash-v6-0-multi-pipleline-faining-to-run/102689 "2017-10-05T17:17:35Z")
Thanks for reading this! I am getting the following error when running logstash v6.0 with multi pipleline on Centos 7.3: sudo /usr/share/logstash/bin/logstash Sending Logstash's logs to /var/log/logstash which is now … --- ## [Logstash: retrying failed action with response code: 429](https://discuss.elastic.co/t/logstash-retrying-failed-action-with-response-code-429/102824)
**Author:** [@hasan\_sharekhan](https://discuss.elastic.co/u/hasan_sharekhan)\ **Replies:** 4\ **Last updated:** [October 5, 2017, 5:07pm UTC](https://discuss.elastic.co/t/logstash-retrying-failed-action-with-response-code-429/102824 "2017-10-05T17:07:14Z")
I am get the below error continuously in Logstash (v5.5) Pushing logs using FileBeat (v5.5) with 2-3 logs files then it works fine. But when using logs using Filebeat with 150+ logs files then I get below error. (since… --- ## [\[SOLVED\] Always see "\_dateparsefailure" in the logstash-plain.log](https://discuss.elastic.co/t/solved-always-see-dateparsefailure-in-the-logstash-plain-log/102488)
**Author:** [@praveenmak](https://discuss.elastic.co/u/praveenmak)\ **Replies:** 7\ **Last updated:** [October 5, 2017, 3:37pm UTC](https://discuss.elastic.co/t/solved-always-see-dateparsefailure-in-the-logstash-plain-log/102488 "2017-10-05T15:37:25Z")
Hello folks, Here is my script , I always see this line in the logstash-plain.log file. PLEASE HELP! \[2017-10-02T16:43:59,750\]\[DEBUG\]\[logstash.filters.date \] config LogStash::Filters::Date/@tag\_on\_failure = \["\_date… --- ## [Working with subfields in logstash](https://discuss.elastic.co/t/working-with-subfields-in-logstash/102844)
**Author:** [@maxf](https://discuss.elastic.co/u/maxf)\ **Replies:** 4\ **Last updated:** [October 5, 2017, 2:39pm UTC](https://discuss.elastic.co/t/working-with-subfields-in-logstash/102844 "2017-10-05T14:39:11Z")
Hello. Im using logstash to parse sysmon events. So i cant correctly use condition if and change subfields. What am i doing wrong? my config: filter { if \[log\_name\] == "Microsoft-Windows-Sysmon/Operational" and \[eve… --- ## [Setup logstash with multinodes](https://discuss.elastic.co/t/setup-logstash-with-multinodes/102828)
**Author:** [@moabbas](https://discuss.elastic.co/u/moabbas)\ **Replies:** 7\ **Last updated:** [October 5, 2017, 2:31pm UTC](https://discuss.elastic.co/t/setup-logstash-with-multinodes/102828 "2017-10-05T14:31:52Z")
Hello All, I need to setup Logstash with multinodes. Thanks, --- ## [Elasticsearch filter plugin query question](https://discuss.elastic.co/t/elasticsearch-filter-plugin-query-question/101998)
**Author:** [@iStepich](https://discuss.elastic.co/u/iStepich)\ **Replies:** 2\ **Last updated:** [October 5, 2017, 12:29pm UTC](https://discuss.elastic.co/t/elasticsearch-filter-plugin-query-question/101998 "2017-10-05T12:29:51Z")
Hello! I need to join some responses and requests to have request time in response JSON. For each request-response pair I have unique trace\_id that looks like "dd4b46b0-44da-473e-8775-d90cce583b0f" Solution for that w… --- ## [XML parsing](https://discuss.elastic.co/t/xml-parsing/102836)
**Author:** [@artobstrel](https://discuss.elastic.co/u/artobstrel)\ **Replies:** 0\ **Last updated:** [October 5, 2017, 12:16pm UTC](https://discuss.elastic.co/t/xml-parsing/102836 "2017-10-05T12:16:38Z")
I will use the following XML document: \ \ \ \\ \ \ \ \ \\ … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=896) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=898)