# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=948 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 949 --- ## [Trying to fetch status of a web url to determine if its up/down](https://discuss.elastic.co/t/trying-to-fetch-status-of-a-web-url-to-determine-if-its-up-down/92163)
**Author:** [@aman\_yadav](https://discuss.elastic.co/u/aman_yadav)\ **Replies:** 4\ **Last updated:** [July 11, 2017, 6:55pm UTC](https://discuss.elastic.co/t/trying-to-fetch-status-of-a-web-url-to-determine-if-its-up-down/92163 "2017-07-11T18:55:44Z")
Hi, I have a url and I need to find out if this site is up or not. I was trying to do something like below. and then I thought I could get header's statuscode to determine its status. But this url returns 'text/html' r… --- ## [Logstash filter multiline not working](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885)
**Author:** [@tharu85](https://discuss.elastic.co/u/tharu85)\ **Replies:** 9\ **Last updated:** [July 11, 2017, 6:46pm UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885 "2017-07-11T18:46:58Z")
I have a java log which has multiline in the log entries. So I have used multiline in filter function in logstash as below. input { file { path =\> "/etc/logstash-5.2.2/sample7.log" start\_position =\> b… --- ## [Help! Problem output using jdbc input](https://discuss.elastic.co/t/help-problem-output-using-jdbc-input/92221)
**Author:** [@nano](https://discuss.elastic.co/u/nano)\ **Replies:** 6\ **Last updated:** [July 11, 2017, 6:29pm UTC](https://discuss.elastic.co/t/help-problem-output-using-jdbc-input/92221 "2017-07-11T18:29:52Z")
Hello! i'm using jdbc input to access my first Mysql data, my schedule is around 30 minutes, after that i'm using jdbc streaming to upload an other database , then i use simple output to see my data as stdout ruby , the… --- ## [Email on alert](https://discuss.elastic.co/t/email-on-alert/92353)
**Author:** [@Padmavathy](https://discuss.elastic.co/u/Padmavathy)\ **Replies:** 3\ **Last updated:** [July 11, 2017, 5:59pm UTC](https://discuss.elastic.co/t/email-on-alert/92353 "2017-07-11T17:59:36Z")
I am using logstash 5.3.I am taking records from oracle database as a cron job. I need to send a email to sort problem which arises when logstash is loading data or elasticsearch server is down Can someone help me with… --- ## [Logstash doesn't send any emails](https://discuss.elastic.co/t/logstash-doesnt-send-any-emails/91967)
**Author:** [@Bob04](https://discuss.elastic.co/u/Bob04)\ **Replies:** 14\ **Last updated:** [July 11, 2017, 5:52pm UTC](https://discuss.elastic.co/t/logstash-doesnt-send-any-emails/91967 "2017-07-11T17:52:54Z")
I am using Logstash-email output with the following configuration: email { to =\> "..." subject =\> "..." port =\> 25 via =\> "smtp" body =\> "%{message}" } But I didn't receive any email. telnet … --- ## [Parsing a mix of String and JSON fields with the kv() filter](https://discuss.elastic.co/t/parsing-a-mix-of-string-and-json-fields-with-the-kv-filter/92093)
**Author:** [@olivier1](https://discuss.elastic.co/u/olivier1)\ **Replies:** 2\ **Last updated:** [July 11, 2017, 5:29pm UTC](https://discuss.elastic.co/t/parsing-a-mix-of-string-and-json-fields-with-the-kv-filter/92093 "2017-07-11T17:29:05Z")
Hi, I'm trying to do something that looks fairly simple according to the doc: trimming start and end double quotes from a field using the kv{} filter. I've tried a whole bunch of combinations like '"', "\\"", '\\"' but… --- ## [Lostash grok parse restapi url issue](https://discuss.elastic.co/t/lostash-grok-parse-restapi-url-issue/92264)
**Author:** [@hemant25](https://discuss.elastic.co/u/hemant25)\ **Replies:** 1\ **Last updated:** [July 11, 2017, 5:16pm UTC](https://discuss.elastic.co/t/lostash-grok-parse-restapi-url-issue/92264 "2017-07-11T17:16:01Z")
Hi , I am trying to parse the following url 1) http://localhost:8001/restapi/user:id=ca00dfd1ccd5f95af329742a7600d8f0/org:id=3967844767629143/test from the above url i want to parse the userid and orgid can anyone… --- ## [Unable to parse xml message with logstash](https://discuss.elastic.co/t/unable-to-parse-xml-message-with-logstash/92568)
**Author:** [@Devesh\_Pandey](https://discuss.elastic.co/u/Devesh_Pandey)\ **Replies:** 1\ **Last updated:** [July 11, 2017, 4:15pm UTC](https://discuss.elastic.co/t/unable-to-parse-xml-message-with-logstash/92568 "2017-07-11T16:15:45Z")
Hi , I am newbie to this technology and trying to set up new infra with Elasticsearch , logstash, and Kibana. I am using logstash-input-jms plugin to get XML message from the MQ queues. I can see the XML message in ki… --- ## [Logstash parsing syslog to different indices depending on source hosts](https://discuss.elastic.co/t/logstash-parsing-syslog-to-different-indices-depending-on-source-hosts/92069)
**Author:** [@tias](https://discuss.elastic.co/u/tias)\ **Replies:** 5\ **Last updated:** [July 11, 2017, 3:20pm UTC](https://discuss.elastic.co/t/logstash-parsing-syslog-to-different-indices-depending-on-source-hosts/92069 "2017-07-11T15:20:35Z")
Hello everyone, I have the following logstash config: input { udp { type =\> "syslog" port =\> 5140 } } filter { if \[type\] == "syslog" { grok { match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslo… --- ## [Remove a field from logstash](https://discuss.elastic.co/t/remove-a-field-from-logstash/92677)
**Author:** [@baha1](https://discuss.elastic.co/u/baha1)\ **Replies:** 0\ **Last updated:** [July 11, 2017, 3:10pm UTC](https://discuss.elastic.co/t/remove-a-field-from-logstash/92677 "2017-07-11T15:10:08Z")
Hi there, I am using logStash to index log file in ElasticSearch.I get this result: { \_index: "errors", \_type: "error", \_id: "AV0sjSUzkDeT3TJ69hAm", \_score: 1, \_source: { date: "16:54:46,235", path: "c:/logstash.log", … --- ## [Arcsight Logs do not Ouput to Kibana](https://discuss.elastic.co/t/arcsight-logs-do-not-ouput-to-kibana/92327)
**Author:** [@vaz1108](https://discuss.elastic.co/u/vaz1108)\ **Replies:** 1\ **Last updated:** [July 11, 2017, 3:09pm UTC](https://discuss.elastic.co/t/arcsight-logs-do-not-ouput-to-kibana/92327 "2017-07-11T15:09:56Z")
Hi Experts, I need some help. I have configure a conf file for logstash to listen on port 8443 to received ArcSight logs on cef format. I can see the connection from the Archsight connector to the the elk server via … --- ## [How to input the dateType path about log file in logstash](https://discuss.elastic.co/t/how-to-input-the-datetype-path-about-log-file-in-logstash/92186)
**Author:** [@0314e7ff87c297426a9e](https://discuss.elastic.co/u/0314e7ff87c297426a9e)\ **Replies:** 1\ **Last updated:** [July 11, 2017, 2:59pm UTC](https://discuss.elastic.co/t/how-to-input-the-datetype-path-about-log-file-in-logstash/92186 "2017-07-11T14:59:30Z")
i want to get log data in ds\_20170707.log log file name contains the date... for example ) ds\_20170705.log ds\_20170706.log ds\_20170707.log .... so.. input { file { path =\> \["/~~~~~~~~/~~~~… --- ## [Replace timestamp by a other field](https://discuss.elastic.co/t/replace-timestamp-by-a-other-field/92488)
**Author:** [@FR-TheS](https://discuss.elastic.co/u/FR-TheS)\ **Replies:** 10\ **Last updated:** [July 11, 2017, 2:58pm UTC](https://discuss.elastic.co/t/replace-timestamp-by-a-other-field/92488 "2017-07-11T14:58:48Z")
Hello everybody, I know this subject is asked many time but after many try, I don't understand some things... Here is my data input "@timestamp" =\> 2017-07-10T14:12:01.442Z, "data\_completiondate" =\> 2017-07-01T02:02… --- ## [Logstash-syslog-output install issues](https://discuss.elastic.co/t/logstash-syslog-output-install-issues/92336)
**Author:** [@jyork](https://discuss.elastic.co/u/jyork)\ **Replies:** 1\ **Last updated:** [July 11, 2017, 1:34pm UTC](https://discuss.elastic.co/t/logstash-syslog-output-install-issues/92336 "2017-07-11T13:34:48Z")
I ran the following command: sudo gem install logstash-output-syslog Get the errors below: ERROR: Could not find a valid gem 'logstash-core-plugin-api' (\<= 2.99, \>= 1.60) in any repository ERROR: Possible alternativ… --- ## [Timeout Error in logstash](https://discuss.elastic.co/t/timeout-error-in-logstash/92656)
**Author:** [@swatititame](https://discuss.elastic.co/u/swatititame)\ **Replies:** 0\ **Last updated:** [July 11, 2017, 1:21pm UTC](https://discuss.elastic.co/t/timeout-error-in-logstash/92656 "2017-07-11T13:21:43Z")
", "id"=\>"AV0xCZcO0YIn9Z-Da2uz", "status"=\>400, "error"=\>{"type"=\>"illegalargu ment\_exception", "reason"=\>"Document contains at least one immense term in field =\\"prd\_ErrorMessage\\" (whose UTF8 encoding is longer than th… --- ## [Logstash file input : read a file only once](https://discuss.elastic.co/t/logstash-file-input-read-a-file-only-once/92637)
**Author:** [@kottapar](https://discuss.elastic.co/u/kottapar)\ **Replies:** 0\ **Last updated:** [July 11, 2017, 12:04pm UTC](https://discuss.elastic.co/t/logstash-file-input-read-a-file-only-once/92637 "2017-07-11T12:04:20Z")
I'm sort of confused, probably on how to use file input for a CSV file. I have a CSV file which is copied over to a dir once every day. I want it to be read only once in a day. So here's my conf. The issue is I restar… --- ## [Logstash 5.4.0 stuck/hang after processing certain amount of data (http-output plugin)](https://discuss.elastic.co/t/logstash-5-4-0-stuck-hang-after-processing-certain-amount-of-data-http-output-plugin/92252)
**Author:** [@singhashutosh111](https://discuss.elastic.co/u/singhashutosh111)\ **Replies:** 0\ **Last updated:** [July 7, 2017, 10:20am UTC](https://discuss.elastic.co/t/logstash-5-4-0-stuck-hang-after-processing-certain-amount-of-data-http-output-plugin/92252 "2017-07-07T10:20:06Z")
Hi All, I am using Logstash 5.4.0 for logs parsing and data ingestion. Issue is that the Logstash becomes unresponsive after processing around 2M log lines. Hardware Configuration is: • RHEL 7 • 16 Cores CPU, 64 GB… --- ## [How to extract and transform unstructured data into fields](https://discuss.elastic.co/t/how-to-extract-and-transform-unstructured-data-into-fields/91916)
**Author:** [@sentinel](https://discuss.elastic.co/u/sentinel)\ **Replies:** 5\ **Last updated:** [July 6, 2017, 11:37am UTC](https://discuss.elastic.co/t/how-to-extract-and-transform-unstructured-data-into-fields/91916 "2017-07-06T11:37:43Z")
My message index field has field1=A big brown wolf, field2=Yet another furry animal, field3=No guarantee how many fields we will see This is what is indexed: field1=A field2=Yet field3=No I'm missing the rest of the … --- ## [Geo code in logstash](https://discuss.elastic.co/t/geo-code-in-logstash/92609)
**Author:** [@DILIP\_SHARMA](https://discuss.elastic.co/u/DILIP_SHARMA)\ **Replies:** 5\ **Last updated:** [July 11, 2017, 9:39am UTC](https://discuss.elastic.co/t/geo-code-in-logstash/92609 "2017-07-11T09:39:12Z")
Hi i am trying to load geo data to elasticsearch using logstash , my .csv file look like this - "col1","col2","col3","col4","col5" "dilip","2017-05-28","45.5","12.56789,72.23456","43" my .config file look like this -… --- ## [Error 403 when log camin on logstah](https://discuss.elastic.co/t/error-403-when-log-camin-on-logstah/92359)
**Author:** [@egalizia](https://discuss.elastic.co/u/egalizia)\ **Replies:** 6\ **Last updated:** [July 11, 2017, 9:11am UTC](https://discuss.elastic.co/t/error-403-when-log-camin-on-logstah/92359 "2017-07-11T09:11:53Z")
HI I install and configure ELK and all work I install the x-pack needed to get authentication to see log and got same problem on logstash when log start caming from nxlog I got this error \[2017-07-08T13:09:13,837\]\[ERR… --- ## [Help with plugin error](https://discuss.elastic.co/t/help-with-plugin-error/92582)
**Author:** [@ahongyun](https://discuss.elastic.co/u/ahongyun)\ **Replies:** 1\ **Last updated:** [July 11, 2017, 7:48am UTC](https://discuss.elastic.co/t/help-with-plugin-error/92582 "2017-07-11T07:48:12Z")
Hi all, I have the following error when running filebeat on one server and logstash on another. A plugin had an unrecoverable error. Will restart this plugin. Plugin: \5044, host=\>"10.… --- ## [Use of \* in logstash config file](https://discuss.elastic.co/t/use-of-in-logstash-config-file/92417)
**Author:** [@ahongyun](https://discuss.elastic.co/u/ahongyun)\ **Replies:** 2\ **Last updated:** [July 11, 2017, 6:23am UTC](https://discuss.elastic.co/t/use-of-in-logstash-config-file/92417 "2017-07-11T06:23:30Z")
hi all, would like to check if the conf below works. if \[type\] == "coreservices-\*" { ... } where the types available are - coreservices-fs - coreservices-xmls thanks --- ## [GELF Output Problem missing short\_message](https://discuss.elastic.co/t/gelf-output-problem-missing-short-message/92578)
**Author:** [@philippkahr](https://discuss.elastic.co/u/philippkahr)\ **Replies:** 0\ **Last updated:** [July 11, 2017, 5:56am UTC](https://discuss.elastic.co/t/gelf-output-problem-missing-short-message/92578 "2017-07-11T05:56:03Z")
Hallo I face an interestinge problem with the GELF Output. I need to send all data that comes from the netflow plugin into the GELF Output, but on STDOUT i always get this Error \> WARN logstash.outputs.gelf - Trouble … --- ## [How to extract .zip files in Logstash?](https://discuss.elastic.co/t/how-to-extract-zip-files-in-logstash/92545)
**Author:** [@g\_dev](https://discuss.elastic.co/u/g_dev)\ **Replies:** 1\ **Last updated:** [July 11, 2017, 2:40am UTC](https://discuss.elastic.co/t/how-to-extract-zip-files-in-logstash/92545 "2017-07-11T02:40:01Z")
How to extract .zip files in Logstash? .zip files have 3 .xml files and I want to apply different filters to the different files. --- ## [Error start logstash after update from 5.4.3 to 5.5](https://discuss.elastic.co/t/error-start-logstash-after-update-from-5-4-3-to-5-5/92418)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 5\ **Last updated:** [July 11, 2017, 12:54am UTC](https://discuss.elastic.co/t/error-start-logstash-after-update-from-5-4-3-to-5-5/92418 "2017-07-11T00:54:25Z")
Hi, I updated my elastic stask from 5.4.3 to 5.5.0. When i start logstash, i got some error Data flow : Log -\> logstash collector(v5.5) -\> RabbitMQ (v3.6.1) -\> Logstash proccessor (filter)(v5.5) -\> ES(v5.5) This is er… --- ## [Logstash is only partially reading input files](https://discuss.elastic.co/t/logstash-is-only-partially-reading-input-files/91489)
**Author:** [@cpast](https://discuss.elastic.co/u/cpast)\ **Replies:** 4\ **Last updated:** [July 10, 2017, 8:15pm UTC](https://discuss.elastic.co/t/logstash-is-only-partially-reading-input-files/91489 "2017-07-10T20:15:43Z")
I'm trying to parse some very large input files (\>10,000 lines) with Logstash, pass them through the CSV, ruby, and mutate filters, and send them out via HTTP. My config is something like this: input { file { pat… --- ## [Logstash not accepting input from 2 filebeat agents](https://discuss.elastic.co/t/logstash-not-accepting-input-from-2-filebeat-agents/92503)
**Author:** [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)\ **Replies:** 1\ **Last updated:** [July 10, 2017, 5:37pm UTC](https://discuss.elastic.co/t/logstash-not-accepting-input-from-2-filebeat-agents/92503 "2017-07-10T17:37:48Z")
HI , i have ELK stack setup and its working fine with one filebeat agent but when i installed filebeat on another client with same filebeat.yml config ...elasticsearch is not seeing 2nd agent . Not sure if there is prob… --- ## [Grok failure in matching two different phrases in the same line, but the pattern is valid](https://discuss.elastic.co/t/grok-failure-in-matching-two-different-phrases-in-the-same-line-but-the-pattern-is-valid/92506)
**Author:** [@parthoiiitm](https://discuss.elastic.co/u/parthoiiitm)\ **Replies:** 0\ **Last updated:** [July 10, 2017, 4:39pm UTC](https://discuss.elastic.co/t/grok-failure-in-matching-two-different-phrases-in-the-same-line-but-the-pattern-is-valid/92506 "2017-07-10T16:39:00Z")
Hi, I am trying to match a line and separate it as two fields. The following is my line: {\\"event\\":\\"transcode\_start\\",\\"task\_uuid\\":\\"1a7ae3e4-434a-4d36-b6db-8ccfebadbbf6\\",\\"ding\_id\\":6439368420201886312,\\"raw\_key\\… --- ## [Pipeline breaks after 5.5.0 upgrade seemingly due to GeoIP filter changes](https://discuss.elastic.co/t/pipeline-breaks-after-5-5-0-upgrade-seemingly-due-to-geoip-filter-changes/92498)
**Author:** [@Myles](https://discuss.elastic.co/u/Myles)\ **Replies:** 0\ **Last updated:** [July 10, 2017, 3:29pm UTC](https://discuss.elastic.co/t/pipeline-breaks-after-5-5-0-upgrade-seemingly-due-to-geoip-filter-changes/92498 "2017-07-10T15:29:46Z")
I just went from 5.4.2 -\> 5.5.0 which resulted in a failed pipeline as per the error below: 2017-07-10T11:22:33,024 | ERROR | logstash.pipeline - Exception in pipelineworker, the pipeline stopped processing new event… --- ## [Box.com](https://discuss.elastic.co/t/box-com/92409)
**Author:** [@Jackson\_Pollock](https://discuss.elastic.co/u/Jackson_Pollock)\ **Replies:** 4\ **Last updated:** [July 10, 2017, 2:49pm UTC](https://discuss.elastic.co/t/box-com/92409 "2017-07-10T14:49:25Z")
Any suggestions on how to index box.com files into elastic? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=947) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=949)