# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=775 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 776 --- ## [Json filer error](https://discuss.elastic.co/t/json-filer-error/128782)
**Author:** [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Replies:** 5\ **Last updated:** [April 20, 2018, 8:39am UTC](https://discuss.elastic.co/t/json-filer-error/128782 "2018-04-20T08:39:14Z")
in kibana message:\`{"dataType":"System.String","host":"xx-PC","name":"TextualDataItem","3dd":"xx.xx.2","time":"2018/4/20 1:34:37","value":"EJbQ7\>ID%l"/2j\ **Author:** [@Yuvaraja\_kannan](https://discuss.elastic.co/u/Yuvaraja_kannan)\ **Replies:** 2\ **Last updated:** [April 20, 2018, 8:09am UTC](https://discuss.elastic.co/t/logstash-config-file-no-output/128665 "2018-04-20T08:09:33Z") HI All, I have a simple log file with the below data, I tried to grok, i used grok debugger and got the expected output, But when I tried the say with logstash config file I am not getting any output. Please advice if t… --- ## [Logstash Error : Error Registering Plugin](https://discuss.elastic.co/t/logstash-error-error-registering-plugin/128824)
**Author:** [@Saurav\_Vasudev](https://discuss.elastic.co/u/Saurav_Vasudev)\ **Replies:** 1\ **Last updated:** [April 20, 2018, 8:08am UTC](https://discuss.elastic.co/t/logstash-error-error-registering-plugin/128824 "2018-04-20T08:08:11Z")
I am getting the following error on running Logstash : \[2018-04-20T13:18:02,661\]\[ERROR\]\[logstash.pipeline \] Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"#\ **Author:** [@Stefano\_Bossi](https://discuss.elastic.co/u/Stefano_Bossi)\ **Replies:** 5\ **Last updated:** [April 20, 2018, 7:42am UTC](https://discuss.elastic.co/t/grok-capture-more-than-one-time/128587 "2018-04-20T07:42:21Z") Dear users, I would like to build a grok filter to capture log files but I am stuck in an easy but not for me problem. One of my log line is like this: Conn=712914 - GET http://o3p.farm.mediaset.it/farmunica/2017/01/1… --- ## [Why i cant use if statement in grock block?](https://discuss.elastic.co/t/why-i-cant-use-if-statement-in-grock-block/128716)
**Author:** [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\ **Replies:** 5\ **Last updated:** [April 20, 2018, 6:30am UTC](https://discuss.elastic.co/t/why-i-cant-use-if-statement-in-grock-block/128716 "2018-04-20T06:30:13Z")
I want to add if statement and check condition in grok block if grok pattern is matched and add module in given json. Bellow Filter shows error please help. grok { match =\> { logs =\> "killed by TERM signa… --- ## [Filter out two keywords from similiar lines](https://discuss.elastic.co/t/filter-out-two-keywords-from-similiar-lines/128610)
**Author:** [@expert1](https://discuss.elastic.co/u/expert1)\ **Replies:** 1\ **Last updated:** [April 20, 2018, 4:40am UTC](https://discuss.elastic.co/t/filter-out-two-keywords-from-similiar-lines/128610 "2018-04-20T04:40:46Z")
Hi, I'm going to filter out user and result(Failed/error/Accepted) from below lines. Failed password for root from 172.16.81.178 port 55152 ssh2 Accepted password for root from 172.16.81.178 port 55152 ssh2 Accepted k… --- ## [Query the doc before update](https://discuss.elastic.co/t/query-the-doc-before-update/128599)
**Author:** [@brunoof1](https://discuss.elastic.co/u/brunoof1)\ **Replies:** 1\ **Last updated:** [April 20, 2018, 3:45am UTC](https://discuss.elastic.co/t/query-the-doc-before-update/128599 "2018-04-20T03:45:45Z")
I want to get a doc before update, follow the example. 1º Event {"date\_sent":"2018-04-18T13:57:42.005","token":"1"}\] 2º Event {"date\_open":"2018-04-18T14:57:42.005","token":"1"}\] 3º Event {"date\_open":"2018-04-18T1… --- ## [Centralized pipeline configuration lost after license expiry - is there a way to get it back? \[Solved\]](https://discuss.elastic.co/t/centralized-pipeline-configuration-lost-after-license-expiry-is-there-a-way-to-get-it-back-solved/128642)
**Author:** [@dswinton](https://discuss.elastic.co/u/dswinton)\ **Replies:** 2\ **Last updated:** [April 20, 2018, 1:01am UTC](https://discuss.elastic.co/t/centralized-pipeline-configuration-lost-after-license-expiry-is-there-a-way-to-get-it-back-solved/128642 "2018-04-20T01:01:35Z")
Hi Guys, I've been googling and picking through my Logstash server to try to fix this issue for a couple of hours now and have come up with nothing... I spent quite a bit of time writing an extensive set of pipeline ru… --- ## [Logstash can't push data to AWS ES](https://discuss.elastic.co/t/logstash-cant-push-data-to-aws-es/127739)
**Author:** [@samwzm](https://discuss.elastic.co/u/samwzm)\ **Replies:** 2\ **Last updated:** [April 19, 2018, 8:07pm UTC](https://discuss.elastic.co/t/logstash-cant-push-data-to-aws-es/127739 "2018-04-19T20:07:49Z")
Hi, I config AWS ES for the elasticsearch service. I can use curl or python to load single doc to it without any problem. However, when I tried to upload data through logstash (I use docker image from another EC2), I al… --- ## [Hardware Required](https://discuss.elastic.co/t/hardware-required/128757)
**Author:** [@vnkasturi](https://discuss.elastic.co/u/vnkasturi)\ **Replies:** 1\ **Last updated:** [April 19, 2018, 6:01pm UTC](https://discuss.elastic.co/t/hardware-required/128757 "2018-04-19T18:01:27Z")
Hi Everyone, I am new to this community and trying to get ELK stack in my office environment. I need some help with the Hardware requirements for standing-up the ELK stack. I want to install winlogbeat on 80 hosts in my … --- ## [Collecte Oracle Logs in windows server](https://discuss.elastic.co/t/collecte-oracle-logs-in-windows-server/128672)
**Author:** [@Amine\_El\_Hijazi](https://discuss.elastic.co/u/Amine_El_Hijazi)\ **Replies:** 1\ **Last updated:** [April 19, 2018, 5:24pm UTC](https://discuss.elastic.co/t/collecte-oracle-logs-in-windows-server/128672 "2018-04-19T17:24:45Z")
Hello everyone ! Well i m looking for a way to collecte logs of my oracle db installed in windows server ! I don't know where the logs are stored ! are they in a specific .log file or in interviewer ? and what is t… --- ## [Logstash cannot index to Elasticsearch](https://discuss.elastic.co/t/logstash-cannot-index-to-elasticsearch/128708)
**Author:** [@haseebsabir](https://discuss.elastic.co/u/haseebsabir)\ **Replies:** 6\ **Last updated:** [April 19, 2018, 4:22pm UTC](https://discuss.elastic.co/t/logstash-cannot-index-to-elasticsearch/128708 "2018-04-19T16:22:29Z")
I am very new to ELK. I am getting error message that logstash Could not index event to Elasticsearch. \[2018-04-18T16:30:06,128\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>4… --- ## [Logstash startup issue](https://discuss.elastic.co/t/logstash-startup-issue/128635)
**Author:** [@Monica1](https://discuss.elastic.co/u/Monica1)\ **Replies:** 5\ **Last updated:** [April 19, 2018, 4:07pm UTC](https://discuss.elastic.co/t/logstash-startup-issue/128635 "2018-04-19T16:07:04Z")
we are having issue in starting up logstash stating Couldn't find any filter plugin 'grok'..Please find the error logs below. \[2018-04-18T13:06:32,281\]\[ERROR\]\[logstash.plugins.registry\] Tried to load a plugin's code, bu… --- ## [Convert unix time to date](https://discuss.elastic.co/t/convert-unix-time-to-date/128638)
**Author:** [@linsie](https://discuss.elastic.co/u/linsie)\ **Replies:** 3\ **Last updated:** [April 19, 2018, 4:05pm UTC](https://discuss.elastic.co/t/convert-unix-time-to-date/128638 "2018-04-19T16:05:59Z")
logstash version : 2.4.1 unix time : 1522092794000 date : 2018-04-10 hello everyone, I want to convert the unix time to date , so how can I do this with date filter ? Thanks so much ! --- ## [Input plugin for returning directory contents having ".xyz" extension](https://discuss.elastic.co/t/input-plugin-for-returning-directory-contents-having-xyz-extension/128515)
**Author:** [@codelearner](https://discuss.elastic.co/u/codelearner)\ **Replies:** 4\ **Last updated:** [April 19, 2018, 3:51pm UTC](https://discuss.elastic.co/t/input-plugin-for-returning-directory-contents-having-xyz-extension/128515 "2018-04-19T15:51:21Z")
I need to return the file names only, from a particular directory as soon as the file is getting created. i tried using "exec" input plugin for this. For achieveing this i wrote a shell script as below : (cd /home/devel… --- ## [Logstash 2.4.1 silently fails if using 2 outputs](https://discuss.elastic.co/t/logstash-2-4-1-silently-fails-if-using-2-outputs/128719)
**Author:** [@sloniki](https://discuss.elastic.co/u/sloniki)\ **Replies:** 2\ **Last updated:** [April 19, 2018, 2:53pm UTC](https://discuss.elastic.co/t/logstash-2-4-1-silently-fails-if-using-2-outputs/128719 "2018-04-19T14:53:51Z")
If I try to use two outputs: output { elasticsearch { hosts =\> \["192.168.1.1", "192.168.1.2"\] } if "prod" in \[tags\] { udp { host =\> "192.168.1.3" port =\> 6177 } } } logstash works ~10… --- ## [Ip is not shown in logstash process](https://discuss.elastic.co/t/ip-is-not-shown-in-logstash-process/128705)
**Author:** [@Monica1](https://discuss.elastic.co/u/Monica1)\ **Replies:** 5\ **Last updated:** [April 19, 2018, 2:22pm UTC](https://discuss.elastic.co/t/ip-is-not-shown-in-logstash-process/128705 "2018-04-19T14:22:02Z")
We are implementing ELK stack for log monitoring. We are having logstash , elasticsearch and kibana running. we have installed filebeat in the servers where we want to monitor the logs. ELK Stack(server IP:10.105.2.70) … --- ## [Bad logstash performance](https://discuss.elastic.co/t/bad-logstash-performance/128003)
**Author:** [@elk2](https://discuss.elastic.co/u/elk2)\ **Replies:** 2\ **Last updated:** [April 19, 2018, 2:13pm UTC](https://discuss.elastic.co/t/bad-logstash-performance/128003 "2018-04-19T14:13:45Z")
Why it happens? count: 7964 - rate 1m: 1172.7211336354694 - \_5m: 1426.257376894607 - 15m: 1473.5596592555096 2018-04-13T14:59:38.408Z count: 14959 - rate 1m: 1190.877357320955 - \_5m: 1425.8200749188456 - 15m: 1473.151… --- ## [Invalid setting for file input plugin- Logstash](https://discuss.elastic.co/t/invalid-setting-for-file-input-plugin-logstash/128711)
**Author:** [@SabRay](https://discuss.elastic.co/u/SabRay)\ **Replies:** 2\ **Last updated:** [April 19, 2018, 2:13pm UTC](https://discuss.elastic.co/t/invalid-setting-for-file-input-plugin-logstash/128711 "2018-04-19T14:13:15Z")
I am trying to load data from a csv file into elasticsearch. When I try to load the data using the command logstash -f "\\Users\\dell\\Desktop\\Data\\logstash\_cars.config" The logstash\_cars.config file is: input{ file{ p… --- ## [How to use grok if some pattern is not want to match?](https://discuss.elastic.co/t/how-to-use-grok-if-some-pattern-is-not-want-to-match/128670)
**Author:** [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\ **Replies:** 4\ **Last updated:** [April 19, 2018, 12:57pm UTC](https://discuss.elastic.co/t/how-to-use-grok-if-some-pattern-is-not-want-to-match/128670 "2018-04-19T12:57:01Z")
I have one input field in which contain boardName and I don’t want to match regex as ".\*tester.\*" . How we can write grok filter to do not match boardname which contain "tester" name. input json is first input { boa… --- ## [Intermittent grok date parse failures](https://discuss.elastic.co/t/intermittent-grok-date-parse-failures/128689)
**Author:** [@AccordISSupport](https://discuss.elastic.co/u/AccordISSupport)\ **Replies:** 3\ **Last updated:** [April 19, 2018, 12:30pm UTC](https://discuss.elastic.co/t/intermittent-grok-date-parse-failures/128689 "2018-04-19T12:30:05Z")
Hi all, I am importing IIS logs into ES using Filebeat and Logstash, using grok to manipulate the input. I am getting strange, intermittent date parse failures with grok (only 65 records out of over nearly 2 million) b… --- ## [Gem to install logstash/event for test/unit](https://discuss.elastic.co/t/gem-to-install-logstash-event-for-test-unit/128388)
**Author:** [@camminati](https://discuss.elastic.co/u/camminati)\ **Replies:** 3\ **Last updated:** [April 19, 2018, 10:18am UTC](https://discuss.elastic.co/t/gem-to-install-logstash-event-for-test-unit/128388 "2018-04-19T10:18:56Z")
Hello, in order to debug/test ruby filters I wrote some tests with test/unit. I have my test-data in json files and would like it to load it into an event (something like LogStash::Event.new(data = json)). This is the… --- ## [Is there way to keep track of logstash progress when a file is being parsed?](https://discuss.elastic.co/t/is-there-way-to-keep-track-of-logstash-progress-when-a-file-is-being-parsed/128681)
**Author:** [@Mayank\_Agrawal](https://discuss.elastic.co/u/Mayank_Agrawal)\ **Replies:** 1\ **Last updated:** [April 19, 2018, 11:23am UTC](https://discuss.elastic.co/t/is-there-way-to-keep-track-of-logstash-progress-when-a-file-is-being-parsed/128681 "2018-04-19T11:23:50Z")
Hi, I am using logstash to read fixed set of log files (parallely) and through .sincedb file, I am figuring it out its progress. I noticed that logstash updates .sincedb file only when it has completely read a file i.e.… --- ## [How can we monitor SAP Adapter through ELK?](https://discuss.elastic.co/t/how-can-we-monitor-sap-adapter-through-elk/128630)
**Author:** [@priti](https://discuss.elastic.co/u/priti)\ **Replies:** 5\ **Last updated:** [April 19, 2018, 11:23am UTC](https://discuss.elastic.co/t/how-can-we-monitor-sap-adapter-through-elk/128630 "2018-04-19T11:23:12Z")
I have to check SAP RFC Connection Status through ELK. How we can do ? --- ## [How to merge lon lat for geo\_point](https://discuss.elastic.co/t/how-to-merge-lon-lat-for-geo-point/128584)
**Author:** [@tdasch](https://discuss.elastic.co/u/tdasch)\ **Replies:** 2\ **Last updated:** [April 19, 2018, 10:54am UTC](https://discuss.elastic.co/t/how-to-merge-lon-lat-for-geo-point/128584 "2018-04-19T10:54:58Z")
ES and LS 6.2. My first post, I apologize if my formatting is incorrect in anyway. Attempting to merge lon lat for geo\_points I can visualize in Kibana. Below is my config file for LS and the error I am receiving. Of not… --- ## [How to add multiple grok pattern in one filter?](https://discuss.elastic.co/t/how-to-add-multiple-grok-pattern-in-one-filter/128622)
**Author:** [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\ **Replies:** 5\ **Last updated:** [April 19, 2018, 10:23am UTC](https://discuss.elastic.co/t/how-to-add-multiple-grok-pattern-in-one-filter/128622 "2018-04-19T10:23:01Z")
Hi, My use case is to match one of the regex using multiple grok in one filter.Tried using multiple filter but not matching one of the pattern even though that pattern is present in input. Here is my filter filter { … --- ## [Netflow module error](https://discuss.elastic.co/t/netflow-module-error/126593)
**Author:** [@Richard\_Ashman](https://discuss.elastic.co/u/Richard_Ashman)\ **Replies:** 3\ **Last updated:** [April 19, 2018, 9:55am UTC](https://discuss.elastic.co/t/netflow-module-error/126593 "2018-04-19T09:55:03Z")
(Sorry for any formatting issues, my first post here and didn't want to put in too much data.) I have ELK 6.2 up and running (single host) and processing data from a variety of sources and can see the data in Kibana. Al… --- ## [Parsing log file](https://discuss.elastic.co/t/parsing-log-file/128669)
**Author:** [@bess](https://discuss.elastic.co/u/bess)\ **Replies:** 0\ **Last updated:** [April 19, 2018, 9:52am UTC](https://discuss.elastic.co/t/parsing-log-file/128669 "2018-04-19T09:52:23Z")
while aprsing my log i have long queries, Can i assign a unique ID to every query and show it with a pop-up in kibana ? Dec 18, 2017 18:14:44:245 |Pal : DEBUG |Persistence-\[CRA\_GENERATION\_THREADS-262\]: COR:vm2: Find que… --- ## [Ruby filter: Ruby exception occurred](https://discuss.elastic.co/t/ruby-filter-ruby-exception-occurred/128550)
**Author:** [@donovans](https://discuss.elastic.co/u/donovans)\ **Replies:** 0\ **Last updated:** [April 18, 2018, 2:21pm UTC](https://discuss.elastic.co/t/ruby-filter-ruby-exception-occurred/128550 "2018-04-18T14:21:35Z")
Hello all I am trying to use a ruby filter to insert the current time if the 'End ' field from the csv is blank here is part of my filter : filter { if \[projectid\] == "sampledata" { csv { columns =\>\[… --- ## [Problem ingesting data with Logstash](https://discuss.elastic.co/t/problem-ingesting-data-with-logstash/128568)
**Author:** [@soumya\_sahoo](https://discuss.elastic.co/u/soumya_sahoo)\ **Replies:** 1\ **Last updated:** [April 19, 2018, 7:15am UTC](https://discuss.elastic.co/t/problem-ingesting-data-with-logstash/128568 "2018-04-19T07:15:47Z")
Hi team, Upon ingesting csv to the elastic engine using logstash, for some of the csv it stops processing at - \[2018-04-18T21:05:05,935\]\[INFO \]\[logstash.outputs.elasticsearch\] New Elasticsearch output {:class=\>"LogStas… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=774) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=776)