# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=672 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 673 --- ## [Condition to check if two or three fields exist](https://discuss.elastic.co/t/condition-to-check-if-two-or-three-fields-exist/151577)
**Author:** [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Replies:** 1\ **Last updated:** [October 9, 2018, 11:24am UTC](https://discuss.elastic.co/t/condition-to-check-if-two-or-three-fields-exist/151577 "2018-10-09T11:24:53Z")
Hello, I have the following condition in the logstash filter section: if \[MD5\] { mutate { add\_field =\> { "md5\_field" =\> "MD5 Hash is: %{MD5}" for %{url} and %{location}}} } It is not solid enough, because if url o… --- ## [Timestamp replacing with format MMM dd HH:mm:ss yyyy](https://discuss.elastic.co/t/timestamp-replacing-with-format-mmm-dd-hhss-yyyy/151368)
**Author:** [@calebereis](https://discuss.elastic.co/u/calebereis)\ **Replies:** 6\ **Last updated:** [October 9, 2018, 11:12am UTC](https://discuss.elastic.co/t/timestamp-replacing-with-format-mmm-dd-hhss-yyyy/151368 "2018-10-09T11:12:26Z")
Hi, I'm Calebe and I'm trying to replace the timestamp with the timestamp from the log file. I tried so many things and nothing works. I have this kind of timestamp in my log file: Jun 24 19:34:54 2016 The problem is… --- ## [Calculate time difference between two logs with uniqueid](https://discuss.elastic.co/t/calculate-time-difference-between-two-logs-with-uniqueid/149041)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 4\ **Last updated:** [October 9, 2018, 10:39am UTC](https://discuss.elastic.co/t/calculate-time-difference-between-two-logs-with-uniqueid/149041 "2018-10-09T10:39:14Z")
hi there is there anyway to calculate time difference between two logs with uniqueid? eg i create fields log\_timestamp, log\_type, user\_id, mobile\_no, etc., below logs 2018-09-19 09:24:57,113 (null) 21 WARN : Test :: v… --- ## [Logstash on windows doesnt feed into ES](https://discuss.elastic.co/t/logstash-on-windows-doesnt-feed-into-es/151372)
**Author:** [@wheelq](https://discuss.elastic.co/u/wheelq)\ **Replies:** 1\ **Last updated:** [October 9, 2018, 10:22am UTC](https://discuss.elastic.co/t/logstash-on-windows-doesnt-feed-into-es/151372 "2018-10-09T10:22:23Z")
I have got my\_config.yml file which works on Unix. I have copied it into windows machine and changed the paths ( (input-\>file-\>path/sincedb\_path) accordingly. In the logs I can see logstash starts up correctly, using t… --- ## [Continuous time parse failures](https://discuss.elastic.co/t/continuous-time-parse-failures/151607)
**Author:** [@Nithani25](https://discuss.elastic.co/u/Nithani25)\ **Replies:** 0\ **Last updated:** [October 9, 2018, 10:06am UTC](https://discuss.elastic.co/t/continuous-time-parse-failures/151607 "2018-10-09T10:06:24Z")
Hi Team, I am working on a log which looks like below \ \ and my logstash conf … --- ## [Can we Create dynamic xpath in logstash](https://discuss.elastic.co/t/can-we-create-dynamic-xpath-in-logstash/151574)
**Author:** [@balumurari1](https://discuss.elastic.co/u/balumurari1)\ **Replies:** 0\ **Last updated:** [October 9, 2018, 8:45am UTC](https://discuss.elastic.co/t/can-we-create-dynamic-xpath-in-logstash/151574 "2018-10-09T08:45:41Z")
Here is my xml file My input: input { file { path =\> "D:/SampleELK/input.xml" start\_position =\> "beginning" sincedb\_path =\> "NUL" codec =\> multiline { pattern =\> "^" negate =\> true what =\> "previous" multilin… --- ## [Add new field based on future events](https://discuss.elastic.co/t/add-new-field-based-on-future-events/151601)
**Author:** [@icking](https://discuss.elastic.co/u/icking)\ **Replies:** 0\ **Last updated:** [October 9, 2018, 9:58am UTC](https://discuss.elastic.co/t/add-new-field-based-on-future-events/151601 "2018-10-09T09:58:33Z")
I have a log like this: id- API - logtype 1 - login - process 1 - logout - process 1 - init - entry I want to add a new field called parentAPI, based on the API with the same id and logtype "entry". The expected res… --- ## [Use Dissect filter on json input data](https://discuss.elastic.co/t/use-dissect-filter-on-json-input-data/151564)
**Author:** [@abhi\_latpate](https://discuss.elastic.co/u/abhi_latpate)\ **Replies:** 1\ **Last updated:** [October 9, 2018, 9:43am UTC](https://discuss.elastic.co/t/use-dissect-filter-on-json-input-data/151564 "2018-10-09T09:43:53Z")
I have inputted two json files from logstash, from a variable named file , i want to create few fields using dissect filter, but this is giving me error: this is my json input file: { "tags": \[ "jdkinstall", "class"… --- ## [Elasticsearch query in Logstash config file for aggregation and re-indexing not working](https://discuss.elastic.co/t/elasticsearch-query-in-logstash-config-file-for-aggregation-and-re-indexing-not-working/151591)
**Author:** [@dx123](https://discuss.elastic.co/u/dx123)\ **Replies:** 0\ **Last updated:** [October 9, 2018, 9:27am UTC](https://discuss.elastic.co/t/elasticsearch-query-in-logstash-config-file-for-aggregation-and-re-indexing-not-working/151591 "2018-10-09T09:27:21Z")
I have data that is already indexed and now I want to perform some querying and aggregating to re-index the data into smaller specific indices. However I have trouble writing the Elasticsearch query where I just want to … --- ## [Multiple Log Files With Different Output Fails](https://discuss.elastic.co/t/multiple-log-files-with-different-output-fails/151590)
**Author:** [@Sparsh\_Singhal](https://discuss.elastic.co/u/Sparsh_Singhal)\ **Replies:** 0\ **Last updated:** [October 9, 2018, 9:25am UTC](https://discuss.elastic.co/t/multiple-log-files-with-different-output-fails/151590 "2018-10-09T09:25:10Z")
I am getting 3 log files from the FileBeat on 4 nodes. Only 1 log from single node is getting indexed in elasticsearch. input { beats { port =\> 5044 ssl =\> true ssl\_certificate =\> "/etc/pki/tls/certs/logst… --- ## [Logstash https output with letsencrypt cert](https://discuss.elastic.co/t/logstash-https-output-with-letsencrypt-cert/151589)
**Author:** [@McElroy](https://discuss.elastic.co/u/McElroy)\ **Replies:** 0\ **Last updated:** [October 9, 2018, 9:25am UTC](https://discuss.elastic.co/t/logstash-https-output-with-letsencrypt-cert/151589 "2018-10-09T09:25:08Z")
Hi, I had troubles with logstash to send logs to elastic because my proxy uses letsencrypt as cert. I found an issue on github regarding such sort of problem. https://github.com/logstash-plugins/logstash-output-http/i… --- ## [Logstash filling daemon.log](https://discuss.elastic.co/t/logstash-filling-daemon-log/151154)
**Author:** [@McElroy](https://discuss.elastic.co/u/McElroy)\ **Replies:** 7\ **Last updated:** [October 9, 2018, 9:21am UTC](https://discuss.elastic.co/t/logstash-filling-daemon-log/151154 "2018-10-09T09:21:56Z")
Hi, Since two days I run into a huge problem with my logstash. It runs fine during the day, but at night it runs crazy like feeding a mogwai after midnight. and I always get the same messages in my daemon.log: Oct 5… --- ## [Communication Between S3 Bucket and logstash S3 Input Plugin](https://discuss.elastic.co/t/communication-between-s3-bucket-and-logstash-s3-input-plugin/151584)
**Author:** [@hell](https://discuss.elastic.co/u/hell)\ **Replies:** 0\ **Last updated:** [October 9, 2018, 9:17am UTC](https://discuss.elastic.co/t/communication-between-s3-bucket-and-logstash-s3-input-plugin/151584 "2018-10-09T09:17:32Z")
I've a question about S3 input plugin which I can't find any answer in the Logstash documentation site. Does the S3 input plugin download events over secure communication protocol? As the events I'm downloading from S… --- ## [Tracking column and Document ID should be same?](https://discuss.elastic.co/t/tracking-column-and-document-id-should-be-same/151583)
**Author:** [@moneisha](https://discuss.elastic.co/u/moneisha)\ **Replies:** 0\ **Last updated:** [October 9, 2018, 9:15am UTC](https://discuss.elastic.co/t/tracking-column-and-document-id-should-be-same/151583 "2018-10-09T09:15:03Z")
There is inserted\_time column and username column in a mysql table. I have to pick the new records daily from mysql. I want to use inserted\_time as a tracking column and use sql\_last\_value. but inserted\_time is not un… --- ## [Special characters in elasticsearch \_id from logstash](https://discuss.elastic.co/t/special-characters-in-elasticsearch-id-from-logstash/150968)
**Author:** [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Replies:** 4\ **Last updated:** [October 9, 2018, 8:48am UTC](https://discuss.elastic.co/t/special-characters-in-elasticsearch-id-from-logstash/150968 "2018-10-09T08:48:09Z")
Hello. I am using Logstash to load data into Elasticsearch. I have chosen to use default "\_id". One of my tools only accepts characters from a range a-z, A-Z, 0-9, -. Sometimes the \_id looks like 1alyPWYB9XD0VZRJa9… --- ## [Calculating day of week using log @timestamp filed instead of the logstash @timestamp](https://discuss.elastic.co/t/calculating-day-of-week-using-log-timestamp-filed-instead-of-the-logstash-timestamp/151346)
**Author:** [@Ira](https://discuss.elastic.co/u/Ira)\ **Replies:** 1\ **Last updated:** [October 9, 2018, 7:36am UTC](https://discuss.elastic.co/t/calculating-day-of-week-using-log-timestamp-filed-instead-of-the-logstash-timestamp/151346 "2018-10-09T07:36:43Z")
Hi, In my JSON log file I have a field called @timestamp: {"@timestamp":"2018-10-04T03:07:12.189-04:00","@version":1,"caller\_method\_name":"beforeBodyWrite","caller\_line\_number":119,"elapsedTime":"174"} When I query th… --- ## [Logstash http poller multiple URL's - output random from any 3 URL's](https://discuss.elastic.co/t/logstash-http-poller-multiple-urls-output-random-from-any-3-urls/151042)
**Author:** [@Harish\_Malu](https://discuss.elastic.co/u/Harish_Malu)\ **Replies:** 1\ **Last updated:** [October 9, 2018, 6:57am UTC](https://discuss.elastic.co/t/logstash-http-poller-multiple-urls-output-random-from-any-3-urls/151042 "2018-10-09T06:57:41Z")
I have an logstash pipeline with http poller urls. Each urls will send JSON data for a given server name. Below is my conf - input { http\_poller { urls =\> { servsb =\> "http://servsb:8003/servmetric/json/" servsd =\> … --- ## [Converting date dynamically using logstash](https://discuss.elastic.co/t/converting-date-dynamically-using-logstash/151560)
**Author:** [@punit\_j](https://discuss.elastic.co/u/punit_j)\ **Replies:** 0\ **Last updated:** [October 9, 2018, 6:47am UTC](https://discuss.elastic.co/t/converting-date-dynamically-using-logstash/151560 "2018-10-09T06:47:36Z")
I have date from logs in below format: 2018-10-06 04:25:43 EDT here the timezone could change to SGT or something else as we have global presence of servers being quieries by filebeat. How do I use date filter to handl… --- ## [Logstash stopped parsing new log entries](https://discuss.elastic.co/t/logstash-stopped-parsing-new-log-entries/151558)
**Author:** [@Basil](https://discuss.elastic.co/u/Basil)\ **Replies:** 0\ **Last updated:** [October 9, 2018, 6:35am UTC](https://discuss.elastic.co/t/logstash-stopped-parsing-new-log-entries/151558 "2018-10-09T06:35:48Z")
Hello Support team, We are using ELK stack for doing business monitoring of Air Shopping site. Please find attached the Architecture of ELK stack in the production environment. We recently came across incidents where… --- ## [How to load simple xml file in logstash](https://discuss.elastic.co/t/how-to-load-simple-xml-file-in-logstash/151459)
**Author:** [@balumurari1](https://discuss.elastic.co/u/balumurari1)\ **Replies:** 2\ **Last updated:** [October 9, 2018, 4:39am UTC](https://discuss.elastic.co/t/how-to-load-simple-xml-file-in-logstash/151459 "2018-10-09T04:39:02Z")
The result to be displayed as shown for the given xml { "number":"34", "name":"hey" } { "number":"34", "name":"you" } waiting for your response, thanks in advance --- ## [Logstash mutate if condition](https://discuss.elastic.co/t/logstash-mutate-if-condition/151549)
**Author:** [@ankit2122](https://discuss.elastic.co/u/ankit2122)\ **Replies:** 0\ **Last updated:** [October 9, 2018, 4:36am UTC](https://discuss.elastic.co/t/logstash-mutate-if-condition/151549 "2018-10-09T04:36:23Z")
I have some log value that comes randomly i used mutate plugin for sperate the data from json. --- ## [System.auth.ssh.geoip.country\_iso\_code Field not Available in auth log](https://discuss.elastic.co/t/system-auth-ssh-geoip-country-iso-code-field-not-available-in-auth-log/151418)
**Author:** [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\ **Replies:** 1\ **Last updated:** [October 9, 2018, 3:27am UTC](https://discuss.elastic.co/t/system-auth-ssh-geoip-country-iso-code-field-not-available-in-auth-log/151418 "2018-10-09T03:27:08Z")
Hello Team, I am using Filebeat to send the system log on kibana dashboard. Logs are reporting on kibana dashbord properly. But i am not getting the field system.auth.ssh.geoip.country\_iso\_code on kibana. So its not sho… --- ## [Rubydebug not working when swtiching over to kafka input](https://discuss.elastic.co/t/rubydebug-not-working-when-swtiching-over-to-kafka-input/151518)
**Author:** [@ramzey1981](https://discuss.elastic.co/u/ramzey1981)\ **Replies:** 0\ **Last updated:** [October 8, 2018, 8:34pm UTC](https://discuss.elastic.co/t/rubydebug-not-working-when-swtiching-over-to-kafka-input/151518 "2018-10-08T20:34:38Z")
when i had the following input in logstash with the following specified, stdout { codec =\> rubydebug { metadata =\> true } } i would be able to see all the logs that logstash was processing. input { redis { host =\> "lo… --- ## [File have error and access log](https://discuss.elastic.co/t/file-have-error-and-access-log/150507)
**Author:** [@caesarramy](https://discuss.elastic.co/u/caesarramy)\ **Replies:** 3\ **Last updated:** [October 8, 2018, 7:31pm UTC](https://discuss.elastic.co/t/file-have-error-and-access-log/150507 "2018-10-08T19:31:57Z")
Good Morning Everyone i have a problem this my code input { file { path =\>"/home/ramy/log" start\_position =\> "beginning" codec =\> multiline { pattern =\> "^\\s\\t" what =\> "previous" negate =\> "true" auto\_flush\_int… --- ## [What to put in logstash config if ther is null for lat & long?](https://discuss.elastic.co/t/what-to-put-in-logstash-config-if-ther-is-null-for-lat-long/149600)
**Author:** [@Gosforth](https://discuss.elastic.co/u/Gosforth)\ **Replies:** 6\ **Last updated:** [October 8, 2018, 7:16pm UTC](https://discuss.elastic.co/t/what-to-put-in-logstash-config-if-ther-is-null-for-lat-long/149600 "2018-10-08T19:16:06Z")
What to put in logstash config if ther is null for lat & long? Some rows do not have gps\_long nor gps\_lat coordinates. SQL input returns NULL. But null is rejected by logstash or ES. My logstash config: mutate { r… --- ## [Getting \_grokparsefailure error while inserting a log](https://discuss.elastic.co/t/getting-grokparsefailure-error-while-inserting-a-log/151501)
**Author:** [@kunalwagh](https://discuss.elastic.co/u/kunalwagh)\ **Replies:** 1\ **Last updated:** [October 8, 2018, 7:04pm UTC](https://discuss.elastic.co/t/getting-grokparsefailure-error-while-inserting-a-log/151501 "2018-10-08T19:04:23Z")
pipeline.conf - input { file { path => "C:/ELK-Stack/logstash/event-data/test.log" start\_position => "beginning" } http { host =\> "localhost" port =\> 8088 } } filter { grok { match =\> { "mes… --- ## [Logstash inserting NUL in the apache\_access.log file](https://discuss.elastic.co/t/logstash-inserting-nul-in-the-apache-access-log-file/151502)
**Author:** [@kunalwagh](https://discuss.elastic.co/u/kunalwagh)\ **Replies:** 1\ **Last updated:** [October 8, 2018, 7:03pm UTC](https://discuss.elastic.co/t/logstash-inserting-nul-in-the-apache-access-log-file/151502 "2018-10-08T19:03:55Z")
I am running below command in the Power shell but getting NUL inserted in front of every character: C:\\ELK-Stack\\logstash\> echo '57.85.164.98 - - \[20/Sep/2017:15:31:04 +0200\] "GET /js/main.js" 200 588 "https://codinge x… --- ## [LogStash runner error: An unexpected error occurred! {:error=\>#\](https://discuss.elastic.co/t/logstash-runner-error-an-unexpected-error-occurred-error-socketerror-initialize-name-or-service-not-known/151134)
**Author:** [@bizmate](https://discuss.elastic.co/u/bizmate)\ **Replies:** 3\ **Last updated:** [October 8, 2018, 5:27pm UTC](https://discuss.elastic.co/t/logstash-runner-error-an-unexpected-error-occurred-error-socketerror-initialize-name-or-service-not-known/151134 "2018-10-08T17:27:19Z")
I am trying to run LogStash on my machine through a docker container. I am setting up the env variables to HTTP\_HOST=https://myawses.us-east-1.es.amazonaws.com:443/ HTTP\_PORT=443 Also all the outputs i have in my con… --- ## [Ingest data from file](https://discuss.elastic.co/t/ingest-data-from-file/151401)
**Author:** [@Mohamed.Yousry](https://discuss.elastic.co/u/Mohamed.Yousry)\ **Replies:** 1\ **Last updated:** [October 8, 2018, 4:10pm UTC](https://discuss.elastic.co/t/ingest-data-from-file/151401 "2018-10-08T16:10:16Z")
Hi I am facing the below error when trying to read from file by logstash Configuration file input { file { path =\> "C:\\ELK\_STACK\\cars.csv" start\_position =\> "beginning" sincedb\_path =\> "NUL" … --- ## [Logstash: unable to read a file in windows](https://discuss.elastic.co/t/logstash-unable-to-read-a-file-in-windows/151111)
**Author:** [@mails2amit.k](https://discuss.elastic.co/u/mails2amit.k)\ **Replies:** 2\ **Last updated:** [October 8, 2018, 2:01pm UTC](https://discuss.elastic.co/t/logstash-unable-to-read-a-file-in-windows/151111 "2018-10-08T14:01:47Z")
Hi Team, I dont see the logstash is reading the input file I have given. is there any problem in my config file? Output file is not getting generated. and my input file has 3 lines of text only. config file - Desktop\\… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=671) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=673)