# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=369 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 370 --- ## [CSV output file row separator](https://discuss.elastic.co/t/csv-output-file-row-separator/229375)
**Author:** [@karnamonkster](https://discuss.elastic.co/u/karnamonkster)\ **Replies:** 0\ **Last updated:** [April 23, 2020, 4:36am UTC](https://discuss.elastic.co/t/csv-output-file-row-separator/229375 "2020-04-23T04:36:16Z")
Hi, LS Version - 7.5 I am trying to get the data out using CSV output plugin, I could see there is are separate rows for header and values on my system (centOS) but when being pushed using curl X POST to another syst… --- ## [Logstash | Azure blob input plugin error](https://discuss.elastic.co/t/logstash-azure-blob-input-plugin-error/225344)
**Author:** [@iniyan.g](https://discuss.elastic.co/u/iniyan.g)\ **Replies:** 2\ **Last updated:** [April 23, 2020, 3:20am UTC](https://discuss.elastic.co/t/logstash-azure-blob-input-plugin-error/225344 "2020-04-23T03:20:47Z")
Hi I'm trying to ingest "Azure App Gateway Performance Logs" into logstash using "azureblob" input plugin. When I run the following command, I see the version mismatch error. Can someone tell me what is the exact issue… --- ## [Jmx plugin jmx conf file pass variable](https://discuss.elastic.co/t/jmx-plugin-jmx-conf-file-pass-variable/229371)
**Author:** [@yigene](https://discuss.elastic.co/u/yigene)\ **Replies:** 0\ **Last updated:** [April 23, 2020, 2:31am UTC](https://discuss.elastic.co/t/jmx-plugin-jmx-conf-file-pass-variable/229371 "2020-04-23T02:31:28Z")
i am trying to use jmx to load the metrics into ES, the jmx metrics are name only with different server host and port, is it possible to user variable in host and port instead of hardcode them? reason is that if i chang… --- ## [Parsing multiline events with logstash](https://discuss.elastic.co/t/parsing-multiline-events-with-logstash/229368)
**Author:** [@olatunde.tokun](https://discuss.elastic.co/u/olatunde.tokun)\ **Replies:** 0\ **Last updated:** [April 23, 2020, 1:48am UTC](https://discuss.elastic.co/t/parsing-multiline-events-with-logstash/229368 "2020-04-23T01:48:11Z")
Hello all, I'm trying to parse this multiline log from kafka, its a mix of multilines, json and kv. Using the multiline codec and grok or KV doesnt seem to produce the expected results. Any thoughts on how I can parse … --- ## [ERROR: Failed to read pipelines yaml file. Location: /usr/share/logstash/config/pipelines.yml](https://discuss.elastic.co/t/error-failed-to-read-pipelines-yaml-file-location-usr-share-logstash-config-pipelines-yml/229362)
**Author:** [@myrsecurity](https://discuss.elastic.co/u/myrsecurity)\ **Replies:** 0\ **Last updated:** [April 22, 2020, 11:04pm UTC](https://discuss.elastic.co/t/error-failed-to-read-pipelines-yaml-file-location-usr-share-logstash-config-pipelines-yml/229362 "2020-04-22T23:04:09Z")
private proprietary enterprise professional background developing on Open Source.. what a pain in the neck! new into this nightmare, ive spent around 36 hrs fighting new ELK install ( from ubuntu ISO to application ) Se… --- ## [Elasticsearch output plugin cloud\_id property](https://discuss.elastic.co/t/elasticsearch-output-plugin-cloud-id-property/229275)
**Author:** [@Alex\_Nikolis](https://discuss.elastic.co/u/Alex_Nikolis)\ **Replies:** 2\ **Last updated:** [April 22, 2020, 10:09pm UTC](https://discuss.elastic.co/t/elasticsearch-output-plugin-cloud-id-property/229275 "2020-04-22T22:09:47Z")
A cloud\_id property is documented on the elastisearch output plugin, but when trying to set it, I receive an error that the property is invalid. Am I missing something? https://www.elastic.co/guide/en/logstash/current/p… --- ## [Help transforming json document for output to InfluxDB](https://discuss.elastic.co/t/help-transforming-json-document-for-output-to-influxdb/229351)
**Author:** [@john.houghton](https://discuss.elastic.co/u/john.houghton)\ **Replies:** 0\ **Last updated:** [April 22, 2020, 8:50pm UTC](https://discuss.elastic.co/t/help-transforming-json-document-for-output-to-influxdb/229351 "2020-04-22T20:50:05Z")
From an application, we generate well-formed json responses into a text file (read: structured output from Telegraf for InfluxDB, formatted in json), e.g. {"tags":{"tag\_1":"tag\_1\_value","tag\_2":"tag\_2\_value"},"fields":… --- ## [Http Input Plugin - basic question](https://discuss.elastic.co/t/http-input-plugin-basic-question/229108)
**Author:** [@Karanbir\_Mann](https://discuss.elastic.co/u/Karanbir_Mann)\ **Replies:** 4\ **Last updated:** [April 22, 2020, 7:39pm UTC](https://discuss.elastic.co/t/http-input-plugin-basic-question/229108 "2020-04-22T19:39:21Z")
Hello team, I am very new to to ELK stack and this is my first task which I am trying to do: I am not understand a part of Http Input Plugin documentation here: https://www.elastic.co/blog/introducing-logstash-input-ht… --- ## [Read mode vs tail mode](https://discuss.elastic.co/t/read-mode-vs-tail-mode/229343)
**Author:** [@Claudio\_Ract\_Costa](https://discuss.elastic.co/u/Claudio_Ract_Costa)\ **Replies:** 0\ **Last updated:** [April 22, 2020, 7:22pm UTC](https://discuss.elastic.co/t/read-mode-vs-tail-mode/229343 "2020-04-22T19:22:10Z")
Hi, I'm processing files that comes from an external server each minute; so once copied, files are static and no new content will be added into the file. I am reading the files in tail mode but I've experienced some is… --- ## [Logstash slow](https://discuss.elastic.co/t/logstash-slow/228858)
**Author:** [@adwaitjoshi](https://discuss.elastic.co/u/adwaitjoshi)\ **Replies:** 6\ **Last updated:** [April 22, 2020, 6:37pm UTC](https://discuss.elastic.co/t/logstash-slow/228858 "2020-04-22T18:37:59Z")
I am missing a setting or two on logstash I am sure but don't know which. My elastic cluster typically indexes at about 12000 docs/sec. I have Logstash pulling from kafka and sensing to Elastic. Seeing speeds of 250/sec. … --- ## [Time format 30.12.2019 14:06:00 GMT+00:00](https://discuss.elastic.co/t/time-format-30-12-2019-1400-gmt-00-00/229256)
**Author:** [@Yaniv\_Nuriel](https://discuss.elastic.co/u/Yaniv_Nuriel)\ **Replies:** 3\ **Last updated:** [April 22, 2020, 4:42pm UTC](https://discuss.elastic.co/t/time-format-30-12-2019-1400-gmt-00-00/229256 "2020-04-22T16:42:36Z")
Dear all, Any idea how to match the following date format? 30.12.2019 14:06:00 GMT+00:100 This is not working dd.MM.yyy HH:mm:ss zzz --- ## [Calculate time difference between 2 lines in log file if there are multiple sets](https://discuss.elastic.co/t/calculate-time-difference-between-2-lines-in-log-file-if-there-are-multiple-sets/228856)
**Author:** [@airan](https://discuss.elastic.co/u/airan)\ **Replies:** 3\ **Last updated:** [April 22, 2020, 4:38pm UTC](https://discuss.elastic.co/t/calculate-time-difference-between-2-lines-in-log-file-if-there-are-multiple-sets/228856 "2020-04-22T16:38:56Z")
Hi everyone, I retrieved a log file that look like this : 2019-03-07 04:35:57.421 19EC | INFO TIMEDATA.DLL: Capacity allocated: Hashtable entries 20648881, Heap mem 805306368 2019-03-07 04:35:57.421 … --- ## [How can I send logs to Logstash with no authentication?](https://discuss.elastic.co/t/how-can-i-send-logs-to-logstash-with-no-authentication/229297)
**Author:** [@FREDDIE2020](https://discuss.elastic.co/u/FREDDIE2020)\ **Replies:** 2\ **Last updated:** [April 22, 2020, 4:13pm UTC](https://discuss.elastic.co/t/how-can-i-send-logs-to-logstash-with-no-authentication/229297 "2020-04-22T16:13:22Z")
Hello All, Need to know how I can send logs to Logstash with no authentication? Is this possible and if so, what is process? Any help appreciated. Thanks, FREDDIE2020 --- ## [Logstash grok ignore field if unavailable](https://discuss.elastic.co/t/logstash-grok-ignore-field-if-unavailable/229159)
**Author:** [@Chandra12](https://discuss.elastic.co/u/Chandra12)\ **Replies:** 1\ **Last updated:** [April 22, 2020, 3:38pm UTC](https://discuss.elastic.co/t/logstash-grok-ignore-field-if-unavailable/229159 "2020-04-22T15:38:37Z")
Hi, I'm trying to skip field if any of below field is missing I couldn't to parse VD=73,IP=10.4.1.1,M1=1586,M2=158,TB=Mobile, Grok expression for above VD=%{NUMBER:VD},IP=%{IPV4:ipaddress},M1=%{NUMBER:m1},M2=%{NUMBE… --- ## [Logstash conf](https://discuss.elastic.co/t/logstash-conf/228972)
**Author:** [@Cheruku125](https://discuss.elastic.co/u/Cheruku125)\ **Replies:** 5\ **Last updated:** [April 22, 2020, 3:36pm UTC](https://discuss.elastic.co/t/logstash-conf/228972 "2020-04-22T15:36:53Z")
Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, {, } at line 8, column 97 (byte 388) after input {\\n jdbc {… --- ## [Is it possible to impose order for the input files?](https://discuss.elastic.co/t/is-it-possible-to-impose-order-for-the-input-files/229255)
**Author:** [@mnrdammu](https://discuss.elastic.co/u/mnrdammu)\ **Replies:** 1\ **Last updated:** [April 22, 2020, 3:33pm UTC](https://discuss.elastic.co/t/is-it-possible-to-impose-order-for-the-input-files/229255 "2020-04-22T15:33:44Z")
Hi All, I have a usecase, where I have to read two files (csv and txt) , using two file inputs. Then in the filter section, I need to do some filtering on the contents of csv file, based on the field derived from txt fi… --- ## [Logstash filter based on first line as identifier](https://discuss.elastic.co/t/logstash-filter-based-on-first-line-as-identifier/229282)
**Author:** [@prkd](https://discuss.elastic.co/u/prkd)\ **Replies:** 1\ **Last updated:** [April 22, 2020, 3:16pm UTC](https://discuss.elastic.co/t/logstash-filter-based-on-first-line-as-identifier/229282 "2020-04-22T15:16:37Z")
I am using a light weight python based forwarder (github) instead of filebeat due to an old Solaris version dependency on the server sending the files. I am unable to add fields like we can do in filebeat using this forw… --- ## [Logstash connecting to ES instance on localhost instead of physical IP](https://discuss.elastic.co/t/logstash-connecting-to-es-instance-on-localhost-instead-of-physical-ip/229232)
**Author:** [@cnvramana](https://discuss.elastic.co/u/cnvramana)\ **Replies:** 2\ **Last updated:** [April 22, 2020, 12:44pm UTC](https://discuss.elastic.co/t/logstash-connecting-to-es-instance-on-localhost-instead-of-physical-ip/229232 "2020-04-22T12:44:19Z")
I have installed ELK stack 7.6 on a single server. This is on a trial run I set up ElasticSearch, Logstash, and Kibana to run on physical IP instead of locahost. I am able to reach all three components with the IP add… --- ## [\[logstash.outputs.elasticsearch\] Marking url as dead. Elasticsearch Unreachable: \[Manticore::SocketTimeout\] Read timed out. LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-elasticsearch-unreachable-manticore-sockettimeout-read-timed-out-logstash-hostunreachableerror/227059)
**Author:** [@m\_vibin](https://discuss.elastic.co/u/m_vibin)\ **Replies:** 2\ **Last updated:** [April 22, 2020, 7:39am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-elasticsearch-unreachable-manticore-sockettimeout-read-timed-out-logstash-hostunreachableerror/227059 "2020-04-22T07:39:27Z")
Hello Team, We built a new Elastic cluster with 3 masters and 2 data nodes with 2 logstash nodes. On the logstash nodes, we are seeing these errors when pushing the logs. It disconnects from the Elastic cluster and con… --- ## [Logstash Monitoring API with HTTPS](https://discuss.elastic.co/t/logstash-monitoring-api-with-https/229023)
**Author:** [@jgato](https://discuss.elastic.co/u/jgato)\ **Replies:** 2\ **Last updated:** [April 22, 2020, 7:07am UTC](https://discuss.elastic.co/t/logstash-monitoring-api-with-https/229023 "2020-04-22T07:07:21Z")
Hi there, I am trying to enable HTTPS for monitoring API. To ency this kind of requests curl -XGET 'localhost:9600/\_node/pipelines?pretty' Other option, would be to disable this API? which APIS exposes Logstash by def… --- ## [Elasticsearch input query with sorting](https://discuss.elastic.co/t/elasticsearch-input-query-with-sorting/229178)
**Author:** [@karnamonkster](https://discuss.elastic.co/u/karnamonkster)\ **Replies:** 0\ **Last updated:** [April 22, 2020, 5:53am UTC](https://discuss.elastic.co/t/elasticsearch-input-query-with-sorting/229178 "2020-04-22T05:53:41Z")
Hi , On dev tools i am able to see the query working and fetching results as expected. GET index/\_search { "query": { "query\_string": { "query": "FT52402" } }, "sort": \[ { "@timestamp": { … --- ## [Logstash keeps regenerating last event](https://discuss.elastic.co/t/logstash-keeps-regenerating-last-event/229051)
**Author:** [@ranadion](https://discuss.elastic.co/u/ranadion)\ **Replies:** 0\ **Last updated:** [April 21, 2020, 1:27pm UTC](https://discuss.elastic.co/t/logstash-keeps-regenerating-last-event/229051 "2020-04-21T13:27:27Z")
Hello everyone. I'm new to Elastic stak and I'm having some issues with my logstash pipeline. I have the following code: input { beats { id =\> "data-collector-filebeat-input" port =\> 5044… --- ## [Scalability of Logstash when using path =\> "/path/to/file/\*/\*/\*/\*/\*/test.log"](https://discuss.elastic.co/t/scalability-of-logstash-when-using-path-path-to-file-test-log/229143)
**Author:** [@david4](https://discuss.elastic.co/u/david4)\ **Replies:** 1\ **Last updated:** [April 22, 2020, 2:37am UTC](https://discuss.elastic.co/t/scalability-of-logstash-when-using-path-path-to-file-test-log/229143 "2020-04-22T02:37:57Z")
Hello, I'm new to the ELK stack and trying to write a Logstash configuration file to monitor a dynamic file structure. I already have two approaches that work, but really want to learn which is a better way. Approach 1… --- ## [Logstash service using almost 200% CPU](https://discuss.elastic.co/t/logstash-service-using-almost-200-cpu/229154)
**Author:** [@jai.waghela](https://discuss.elastic.co/u/jai.waghela)\ **Replies:** 7\ **Last updated:** [April 22, 2020, 2:00am UTC](https://discuss.elastic.co/t/logstash-service-using-almost-200-cpu/229154 "2020-04-22T02:00:57Z")
Hi, The logstash service is constantly consuming high CPU above 150%. Below is the error but not sure what to do. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 32381 logstash 39 19 6650… --- ## [Tried to parse field as object, but found a concrete value](https://discuss.elastic.co/t/tried-to-parse-field-as-object-but-found-a-concrete-value/229134)
**Author:** [@Sirius](https://discuss.elastic.co/u/Sirius)\ **Replies:** 2\ **Last updated:** [April 22, 2020, 12:07am UTC](https://discuss.elastic.co/t/tried-to-parse-field-as-object-but-found-a-concrete-value/229134 "2020-04-22T00:07:44Z")
Hi, I'm running filebeat 7.6.2 with logstash 7.5.1 having missing lines of logs in ES due this warning message ... \[2020-04-21T21:25:21,105\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearc… --- ## [Grok filter is not working properly](https://discuss.elastic.co/t/grok-filter-is-not-working-properly/229073)
**Author:** [@Ankit-github-26](https://discuss.elastic.co/u/Ankit-github-26)\ **Replies:** 9\ **Last updated:** [April 21, 2020, 9:43pm UTC](https://discuss.elastic.co/t/grok-filter-is-not-working-properly/229073 "2020-04-21T21:43:39Z")
Hello Guys, I have Filebeat-7.1 installed in a Debian server, this Filebeat send data from files in this Debian server to server with Logstash 7.6 , here are the files config Filebeat.yml: #===========================… --- ## [Installing Logstash plugin for Chef spec](https://discuss.elastic.co/t/installing-logstash-plugin-for-chef-spec/226541)
**Author:** [@meowtochondria](https://discuss.elastic.co/u/meowtochondria)\ **Replies:** 1\ **Last updated:** [April 21, 2020, 9:27pm UTC](https://discuss.elastic.co/t/installing-logstash-plugin-for-chef-spec/226541 "2020-04-21T21:27:29Z")
I am trying to setup tests for Logstash filters. We use Chef, and are having trouble installing plugins for our filters. Here's the spec file that sets up and run the tests. Tests are described in a YAML file, which I do… --- ## [Fingerprint to get the last out of user\_agent](https://discuss.elastic.co/t/fingerprint-to-get-the-last-out-of-user-agent/229120)
**Author:** [@hispeed](https://discuss.elastic.co/u/hispeed)\ **Replies:** 0\ **Last updated:** [April 21, 2020, 6:39pm UTC](https://discuss.elastic.co/t/fingerprint-to-get-the-last-out-of-user-agent/229120 "2020-04-21T18:39:45Z")
Hi i have now nearly finished the parsing of my nginx access logfile. This line is the "user\_agent" field. I parse with my filter already different stuff out of that. Chrome example: "Mozilla/5.0 (Windows NT 10.0; Wi… --- ## [Issue with CAST](https://discuss.elastic.co/t/issue-with-cast/229104)
**Author:** [@JeremyP](https://discuss.elastic.co/u/JeremyP)\ **Replies:** 1\ **Last updated:** [April 21, 2020, 5:47pm UTC](https://discuss.elastic.co/t/issue-with-cast/229104 "2020-04-21T17:47:48Z")
Hello, I have an IP address in my database which I'm trying to extract and ingest into Elastic. I originally did a CAST on this statement (required as logstash did not like the source format). input { jdbc { … --- ## [Sincedb\_clean\_after not working in logstash plugin](https://discuss.elastic.co/t/sincedb-clean-after-not-working-in-logstash-plugin/225411)
**Author:** [@Mohsin\_Alam](https://discuss.elastic.co/u/Mohsin_Alam)\ **Replies:** 1\ **Last updated:** [April 21, 2020, 4:39pm UTC](https://discuss.elastic.co/t/sincedb-clean-after-not-working-in-logstash-plugin/225411 "2020-04-21T16:39:32Z")
This is my config. file for logstash file input plugin: file { id =\> "my\_plugin\_id" type =\> "log" path =\> "/uploads//.log" mode =\> "read" start\_position =\> beginning sincedb\_clean\_after =\> "10 seconds" codec =\> … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=368) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=370)