# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=788 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 789 --- ## [How logstash work?](https://discuss.elastic.co/t/how-logstash-work/124479)
**Author:** [@retxedue](https://discuss.elastic.co/u/retxedue)\ **Replies:** 7\ **Last updated:** [April 3, 2018, 10:55am UTC](https://discuss.elastic.co/t/how-logstash-work/124479 "2018-04-03T10:55:43Z")
I want to understand if i enabled logatash service via systemctl. How often does it read the logstash config i created? Is there a way i can increase frequency of reading the config file i made? --- ## [Configuring Both X-Pack Monitoring and X-Pack Management with TLS for Logstash](https://discuss.elastic.co/t/configuring-both-x-pack-monitoring-and-x-pack-management-with-tls-for-logstash/126045)
**Author:** [@wanch](https://discuss.elastic.co/u/wanch)\ **Replies:** 2\ **Last updated:** [April 3, 2018, 10:32am UTC](https://discuss.elastic.co/t/configuring-both-x-pack-monitoring-and-x-pack-management-with-tls-for-logstash/126045 "2018-04-03T10:32:58Z")
Hi, Not sure whether to post this in the Logstash or X-Pack category, but I guess I'd try this place first. I am having troubles configuring both the X-Pack Monitoring and X-Pack Centralized Pipeline Management for Log… --- ## [Parsing and filtering text from docx](https://discuss.elastic.co/t/parsing-and-filtering-text-from-docx/126547)
**Author:** [@junocoder](https://discuss.elastic.co/u/junocoder)\ **Replies:** 0\ **Last updated:** [April 3, 2018, 10:22am UTC](https://discuss.elastic.co/t/parsing-and-filtering-text-from-docx/126547 "2018-04-03T10:22:19Z")
I am very new to logstash and elastic search. I have been able to install elk stack and would like to know if it is possible to ingest data or text from a word docx file, parse data by date location and name etc and sav… --- ## [Global field](https://discuss.elastic.co/t/global-field/126545)
**Author:** [@Invictus](https://discuss.elastic.co/u/Invictus)\ **Replies:** 0\ **Last updated:** [April 3, 2018, 10:14am UTC](https://discuss.elastic.co/t/global-field/126545 "2018-04-03T10:14:43Z")
Hi, I have two different logfiles. Construction of the first one: 19.03.18-13:56:06 TabellenName -A 12 -D 323 TabellenName -A 329 -D 323 TabellenName -A 12 -D 12,3 Tab… --- ## [How can I use the date filter (match) with many fields](https://discuss.elastic.co/t/how-can-i-use-the-date-filter-match-with-many-fields/126457)
**Author:** [@Daniel\_Gonzalez1](https://discuss.elastic.co/u/Daniel_Gonzalez1)\ **Replies:** 3\ **Last updated:** [April 3, 2018, 9:57am UTC](https://discuss.elastic.co/t/how-can-i-use-the-date-filter-match-with-many-fields/126457 "2018-04-03T09:57:45Z")
Hello! I want to match the date from a cvs file that comes in different fields with logstash! The date in the cvs file come in four fields (four columns) for example: |Fecha\_Trans|HH|MM|SS| 180314 |16 | 40 |45… --- ## [Can someone help me with the below error](https://discuss.elastic.co/t/can-someone-help-me-with-the-below-error/126532)
**Author:** [@Shiv\_Gupta](https://discuss.elastic.co/u/Shiv_Gupta)\ **Replies:** 0\ **Last updated:** [April 3, 2018, 9:04am UTC](https://discuss.elastic.co/t/can-someone-help-me-with-the-below-error/126532 "2018-04-03T09:04:07Z")
Hello All, My logstash is not working not sure what is happening , please assist what could be the issue. \[ERROR\] 2018-04-03 14:24:07.658 \[main\] Logstash - java.lang.IllegalStateException: org.jruby.exceptions.RaiseExc… --- ## [Stdin input plugin missed some events](https://discuss.elastic.co/t/stdin-input-plugin-missed-some-events/126530)
**Author:** [@Swati\_Sinha](https://discuss.elastic.co/u/Swati_Sinha)\ **Replies:** 0\ **Last updated:** [April 3, 2018, 8:49am UTC](https://discuss.elastic.co/t/stdin-input-plugin-missed-some-events/126530 "2018-04-03T08:49:42Z")
Hi, Trying to load data into elastic search using logstash config. I am using 'stdin' input plugin for reading data from a file, here is the command: /usr/share/logstash/bin/logstash -f logstashConfigFile \< dataFile l… --- ## [How to encode file with base 64](https://discuss.elastic.co/t/how-to-encode-file-with-base-64/126528)
**Author:** [@Yimjunhyeok](https://discuss.elastic.co/u/Yimjunhyeok)\ **Replies:** 0\ **Last updated:** [April 3, 2018, 8:48am UTC](https://discuss.elastic.co/t/how-to-encode-file-with-base-64/126528 "2018-04-03T08:48:29Z")
Hello. I want to save the file that is encoded in elasticsearch So i use the logstash and ruby filter. How to encode the content with ruby filter? help me if you know this. Below is my test.. 13 filter { 14 grok … --- ## [Logstash config with redis](https://discuss.elastic.co/t/logstash-config-with-redis/126424)
**Author:** [@Gaurav\_Agarwal](https://discuss.elastic.co/u/Gaurav_Agarwal)\ **Replies:** 1\ **Last updated:** [April 3, 2018, 8:42am UTC](https://discuss.elastic.co/t/logstash-config-with-redis/126424 "2018-04-03T08:42:35Z")
While using redis as the in-memory database the logstash config file is not working fine whereas when in place redis if I use the file input plugin it does not fail at all. I am not able to understand what is wrong with … --- ## [Drop cdrom field](https://discuss.elastic.co/t/drop-cdrom-field/124636)
**Author:** [@clacko](https://discuss.elastic.co/u/clacko)\ **Replies:** 1\ **Last updated:** [April 3, 2018, 7:46am UTC](https://discuss.elastic.co/t/drop-cdrom-field/124636 "2018-04-03T07:46:15Z")
hi, trying to omit messages with system filesystem type of cdrom. My filter is as per below but seems to not work. They are still showing up in Kibana. Any help would be appreciated. thanks filter { if (\[indexname… --- ## [Logstash - event id to name](https://discuss.elastic.co/t/logstash-event-id-to-name/126348)
**Author:** [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\ **Replies:** 11\ **Last updated:** [April 3, 2018, 7:42am UTC](https://discuss.elastic.co/t/logstash-event-id-to-name/126348 "2018-04-03T07:42:22Z")
Hi All, Am trying to add another field in elasticsearch from the winlogbeat the event id to tier corresponding names but I couldn't execute it filter { if "Security" in \[log\_name\] { translate { dictionary =\> … --- ## [How to Send logstash logs directly to elastic?](https://discuss.elastic.co/t/how-to-send-logstash-logs-directly-to-elastic/126488)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 2\ **Last updated:** [April 3, 2018, 6:24am UTC](https://discuss.elastic.co/t/how-to-send-logstash-logs-directly-to-elastic/126488 "2018-04-03T06:24:30Z")
Hi all Can we send logstash logs directly to elastic?. Currently it sits under logs folder as physical .log files Is there any way we can do so? --- ## [Extract Coordinates From Field "log" and Create A New GeoPoint type from it](https://discuss.elastic.co/t/extract-coordinates-from-field-log-and-create-a-new-geopoint-type-from-it/126187)
**Author:** [@kai\_socrates](https://discuss.elastic.co/u/kai_socrates)\ **Replies:** 3\ **Last updated:** [April 3, 2018, 6:12am UTC](https://discuss.elastic.co/t/extract-coordinates-from-field-log-and-create-a-new-geopoint-type-from-it/126187 "2018-04-03T06:12:34Z")
I have existing logs which are being added to elasticsearch as a single field called, 'log'. Sample of log content is something like below: \[2018-03-28T14:20:16.994Z\] "GET /places/v1/radius?maxresults=25&offset=0&locale… --- ## [How to parse such log](https://discuss.elastic.co/t/how-to-parse-such-log/126100)
**Author:** [@Anoop\_Sharda](https://discuss.elastic.co/u/Anoop_Sharda)\ **Replies:** 7\ **Last updated:** [April 3, 2018, 5:56am UTC](https://discuss.elastic.co/t/how-to-parse-such-log/126100 "2018-04-03T05:56:57Z")
I am getting logs in below format. How can I parse it. if I use msgpack, i am able to see proper results. but i want to parse it in some other way hence need to know what can be done with below mentioned format. \\x8C\\xA… --- ## [I got a "tags:\["\_dateparsefailure"\]" in my logstash rubydebug](https://discuss.elastic.co/t/i-got-a-tags-dateparsefailure-in-my-logstash-rubydebug/126074)
**Author:** [@wentao](https://discuss.elastic.co/u/wentao)\ **Replies:** 2\ **Last updated:** [April 3, 2018, 5:52am UTC](https://discuss.elastic.co/t/i-got-a-tags-dateparsefailure-in-my-logstash-rubydebug/126074 "2018-04-03T05:52:45Z")
my log's data is : 2018-03-29.14:08:16.117 http-apr-8180-exec-10 INFO CouponServiceImpl 3952a0e3aa444bc19cbe69af2e3bf459 : xxxxxxxxxxxxxxxxxxxxxxxxxxx my filter is : filter { if \[type\] == 'sq-strategy-web\_log' { rub… --- ## [Parsing json data which has list of arrays inside array](https://discuss.elastic.co/t/parsing-json-data-which-has-list-of-arrays-inside-array/124080)
**Author:** [@kumarkar](https://discuss.elastic.co/u/kumarkar)\ **Replies:** 5\ **Last updated:** [April 3, 2018, 5:51am UTC](https://discuss.elastic.co/t/parsing-json-data-which-has-list-of-arrays-inside-array/124080 "2018-04-03T05:51:16Z")
Hi All, I am new to logstash, i have a requirement to parse the huge json file. Can someone help me out here? Is it possible to parse the below content using grok script? Below is the sample file content: { "enviro… --- ## [KV Filtering data between square brackets](https://discuss.elastic.co/t/kv-filtering-data-between-square-brackets/126124)
**Author:** [@harinandan04](https://discuss.elastic.co/u/harinandan04)\ **Replies:** 6\ **Last updated:** [April 3, 2018, 4:41am UTC](https://discuss.elastic.co/t/kv-filtering-data-between-square-brackets/126124 "2018-04-03T04:41:29Z")
Hi, How can i parse data which has two layers of square brackets. Inner square brackets Outer square brackets I only want to consider outer square brackets data as key value pairs. Example : \[filename=0\_578cc\[R2\]… --- ## [How to install plugin with .gem file?](https://discuss.elastic.co/t/how-to-install-plugin-with-gem-file/126060)
**Author:** [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Replies:** 3\ **Last updated:** [April 3, 2018, 3:04am UTC](https://discuss.elastic.co/t/how-to-install-plugin-with-gem-file/126060 "2018-04-03T03:04:12Z")
./logstash-plugin install file:///opt/ES/logstash-filter-ip2location-1.0.2.gem ERROR: Invalid pack for: file:///opt/ES/logstash-filter-ip2location-1.0.2.gem, reason: Invalid format, the pack must be in zip format, messa… --- ## [Logstash-output-syslog](https://discuss.elastic.co/t/logstash-output-syslog/126490)
**Author:** [@mun](https://discuss.elastic.co/u/mun)\ **Replies:** 0\ **Last updated:** [April 3, 2018, 2:30am UTC](https://discuss.elastic.co/t/logstash-output-syslog/126490 "2018-04-03T02:30:47Z")
Hi, I installed logstash-output-syslog plugin, and once i have done the configuration part, I try to restart but it fails with this error below: my /var/log/logstash/logstash-plain.log \[2018-04-03T10:03:12,662\]\[INFO \]… --- ## [File Path Directory name extract Regex](https://discuss.elastic.co/t/file-path-directory-name-extract-regex/126455)
**Author:** [@vimal\_P](https://discuss.elastic.co/u/vimal_P)\ **Replies:** 3\ **Last updated:** [April 2, 2018, 8:39pm UTC](https://discuss.elastic.co/t/file-path-directory-name-extract-regex/126455 "2018-04-02T20:39:02Z")
Hi Everyone, I need to extract directory name from the "path" field and assign it to "arcot" field. I require "arcot" value for filtering purpose. Below is the code i am using. path variable is storing "D:\\Program File… --- ## [Grok: multiple occurences of pattern](https://discuss.elastic.co/t/grok-multiple-occurences-of-pattern/125957)
**Author:** [@janvkn](https://discuss.elastic.co/u/janvkn)\ **Replies:** 4\ **Last updated:** [April 2, 2018, 7:29pm UTC](https://discuss.elastic.co/t/grok-multiple-occurences-of-pattern/125957 "2018-04-02T19:29:31Z")
Hello, I am looking for a solution to do the following: I have a log file where a single log entry can contain multiple key/value pairs that I want to extract. The problem is, I want to extract all occurences in a singl… --- ## [How to stop Logstash from processing logs](https://discuss.elastic.co/t/how-to-stop-logstash-from-processing-logs/125409)
**Author:** [@Feedy](https://discuss.elastic.co/u/Feedy)\ **Replies:** 3\ **Last updated:** [April 2, 2018, 7:18pm UTC](https://discuss.elastic.co/t/how-to-stop-logstash-from-processing-logs/125409 "2018-04-02T19:18:56Z")
A few GB of data was sent to logstash and I now want to stop the processing of that data. Is there a way to do so? I've tried restarting and also deleting the index but after about 15 secs its starts to process the data … --- ## [Logstash - data not loading](https://discuss.elastic.co/t/logstash-data-not-loading/126454)
**Author:** [@Yuvaraja\_kannan](https://discuss.elastic.co/u/Yuvaraja_kannan)\ **Replies:** 0\ **Last updated:** [April 2, 2018, 6:09pm UTC](https://discuss.elastic.co/t/logstash-data-not-loading/126454 "2018-04-02T18:09:07Z")
HI ALL, I am trying to load some sample data set to elasticsearch using logstash. But I am not sure if this is working, when i run my config file from command prompt ,log stash stops at this point and doesn't give any i… --- ## [Multiple pipelines in logstash](https://discuss.elastic.co/t/multiple-pipelines-in-logstash/126165)
**Author:** [@pandeesh](https://discuss.elastic.co/u/pandeesh)\ **Replies:** 3\ **Last updated:** [March 31, 2018, 5:42pm UTC](https://discuss.elastic.co/t/multiple-pipelines-in-logstash/126165 "2018-03-31T17:42:58Z")
unable to start multiple pipelines in logstash 5.6.2. it's similar to Logstash Multiple Pipelines Doesn't work, but I am having the file named correctly as pipelines.yml. Here's the message that goes in a loop when I s… --- ## [Geoip creates everything but the geoip.location](https://discuss.elastic.co/t/geoip-creates-everything-but-the-geoip-location/126272)
**Author:** [@Robert\_Bardo](https://discuss.elastic.co/u/Robert_Bardo)\ **Replies:** 9\ **Last updated:** [April 2, 2018, 5:19pm UTC](https://discuss.elastic.co/t/geoip-creates-everything-but-the-geoip-location/126272 "2018-04-02T17:19:02Z")
Hello, I have been just banging my head on the keyboard trying to figure out why all fields are created in my event but the ".location" field I understand that with the correct index mapping: - like so? { "template": … --- ## [Could not find or load main class Files](https://discuss.elastic.co/t/could-not-find-or-load-main-class-files/126443)
**Author:** [@sahadevgd](https://discuss.elastic.co/u/sahadevgd)\ **Replies:** 17\ **Last updated:** [April 2, 2018, 4:42pm UTC](https://discuss.elastic.co/t/could-not-find-or-load-main-class-files/126443 "2018-04-02T16:42:42Z")
HI Team, We are observing below issue when trying to run the logstash. Could you please help us on this. ELK version 6.2.3 OS - Windows JDK - 1.8 Error: Could not find or load main class Files\\Java\\jdk1.7.0\_51\\lib;D… --- ## [Add @timestamp according to actual syslog date-time](https://discuss.elastic.co/t/add-timestamp-according-to-actual-syslog-date-time/126386)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 5\ **Last updated:** [April 2, 2018, 2:45pm UTC](https://discuss.elastic.co/t/add-timestamp-according-to-actual-syslog-date-time/126386 "2018-04-02T14:45:13Z")
I am parsing some syslog logs as follows: grok { match =\> { 'message' =\> '%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:\\\[%{POSINT:syslog\_pid}\\\])?: %{GREEDYDATA:syslog\_… --- ## [How to pick up only ip address and timestamp from log message](https://discuss.elastic.co/t/how-to-pick-up-only-ip-address-and-timestamp-from-log-message/126426)
**Author:** [@Eswar\_Kumar\_Musiboin](https://discuss.elastic.co/u/Eswar_Kumar_Musiboin)\ **Replies:** 1\ **Last updated:** [April 2, 2018, 2:27pm UTC](https://discuss.elastic.co/t/how-to-pick-up-only-ip-address-and-timestamp-from-log-message/126426 "2018-04-02T14:27:52Z")
I am having a big log message and i have to pick up only ip and timestamp in that message. is it work?? . anybody find out help msg: Apr 2 06:34:32 logmessage\[213456\]: this is log message of the system with an ip:123.… --- ## [Index logstash](https://discuss.elastic.co/t/index-logstash/126322)
**Author:** [@nadine](https://discuss.elastic.co/u/nadine)\ **Replies:** 3\ **Last updated:** [April 2, 2018, 2:13pm UTC](https://discuss.elastic.co/t/index-logstash/126322 "2018-04-02T14:13:23Z")
Hello, logstash don't create index. I don't know what is the problem. curl -XGET "localhost:9200/\_cat/indices/" | sort The output: % Total % Received % Xferd Average Speed Time Time Time Current … --- ## [How to delete the record from elasticsearch if record has been deleted from source like oracle database](https://discuss.elastic.co/t/how-to-delete-the-record-from-elasticsearch-if-record-has-been-deleted-from-source-like-oracle-database/126435)
**Author:** [@ajitkaurav](https://discuss.elastic.co/u/ajitkaurav)\ **Replies:** 0\ **Last updated:** [April 2, 2018, 1:57pm UTC](https://discuss.elastic.co/t/how-to-delete-the-record-from-elasticsearch-if-record-has-been-deleted-from-source-like-oracle-database/126435 "2018-04-02T13:57:21Z")
Hi guys I am using logstash config file and get the data from oracle database .config detail given below . it works as expected but while I do delete the data from oracle database . but elasticsearch not deleting that… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=787) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=789)