# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=385 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 386 --- ## [If conditional on nested field with IP address](https://discuss.elastic.co/t/if-conditional-on-nested-field-with-ip-address/224739)
**Author:** [@uklipse](https://discuss.elastic.co/u/uklipse)\ **Replies:** 9\ **Last updated:** [March 24, 2020, 4:58pm UTC](https://discuss.elastic.co/t/if-conditional-on-nested-field-with-ip-address/224739 "2020-03-24T16:58:30Z")
I'm trying to create a simple if conditional on the host.name field if it matches an IP address. This is originating from a syslog source and is a static IP. I've tried == with quotes around the IP, escaping the octet do… --- ## [Date filter to yyyy/MM/dd HH:mm:ss format](https://discuss.elastic.co/t/date-filter-to-yyyy-mm-dd-hhss-format/224569)
**Author:** [@david-vazquez](https://discuss.elastic.co/u/david-vazquez)\ **Replies:** 6\ **Last updated:** [March 24, 2020, 4:25pm UTC](https://discuss.elastic.co/t/date-filter-to-yyyy-mm-dd-hhss-format/224569 "2020-03-24T16:25:00Z")
Hello, I´m trying to parse a date field called "ReceiveTime" which has the following format: ReceiveTime=2020/03/03 16:47:56 i´m using the next filter in Logstash: date { match =\> \[ "ReceiveTime", "yyyy/MM/dd HH:mm:… --- ## [Split json message into a new column in kibana table](https://discuss.elastic.co/t/split-json-message-into-a-new-column-in-kibana-table/224565)
**Author:** [@ARoy](https://discuss.elastic.co/u/ARoy)\ **Replies:** 12\ **Last updated:** [March 24, 2020, 3:07pm UTC](https://discuss.elastic.co/t/split-json-message-into-a-new-column-in-kibana-table/224565 "2020-03-24T15:07:28Z")
Hi, I need to add new column in kibana from the json message. The message is as follows : {"results":\[{"gender":"female","name":{"title":"Ms","first":"Eden","last":"Morel"},"location":{"street":{"number":1412,"name":"R… --- ## [Web socket input issue with certstream](https://discuss.elastic.co/t/web-socket-input-issue-with-certstream/224862)
**Author:** [@Alfalab](https://discuss.elastic.co/u/Alfalab)\ **Replies:** 2\ **Last updated:** [March 24, 2020, 2:52pm UTC](https://discuss.elastic.co/t/web-socket-input-issue-with-certstream/224862 "2020-03-24T14:52:54Z")
hi, here is a tail of log \[2020-03-24T10:41:35,144\]\[WARN \]\[logstash.inputs.websocket\]\[main\] websocket input client threw exception, restarting {:exception=\>#\ **Author:** [@Sagar\_Mandal](https://discuss.elastic.co/u/Sagar_Mandal)\ **Replies:** 2\ **Last updated:** [March 24, 2020, 2:51pm UTC](https://discuss.elastic.co/t/adding-xxxx-in-a-column-through-logstash/224837 "2020-03-24T14:51:07Z") Hi Team, I have this task where I'm inputing some csv data which contain Accc number, which is 10 digit long. so what i want to do is that before storing it into elasticsearch, I want to change the end 5 digit to X. so… --- ## [Kubernetes - Logstash or Fluentd?](https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532)
**Author:** [@Mauricio\_Borges](https://discuss.elastic.co/u/Mauricio_Borges)\ **Replies:** 6\ **Last updated:** [March 24, 2020, 1:18pm UTC](https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532 "2020-03-24T13:18:38Z")
Hi Team! I have started working with ELK since last year and has been using a lot Beats, Elasticsearch, Kibana and a bit about Logstash. I have started POC case using ELK on-premise supporting OCP 4.x and customer aske… --- ## [Split a DateTime field to all Documents in logstash](https://discuss.elastic.co/t/split-a-datetime-field-to-all-documents-in-logstash/224726)
**Author:** [@panwarsagar](https://discuss.elastic.co/u/panwarsagar)\ **Replies:** 3\ **Last updated:** [March 24, 2020, 12:44pm UTC](https://discuss.elastic.co/t/split-a-datetime-field-to-all-documents-in-logstash/224726 "2020-03-24T12:44:18Z")
Hi, I have a Start\_Time field that I wants to split in all documents using the split filter. But split filter only takes string and array, So i tried to use mutate filter to covert Start\_Time field to string. But stil… --- ## [My logstash conf file is not creating a test field](https://discuss.elastic.co/t/my-logstash-conf-file-is-not-creating-a-test-field/223711)
**Author:** [@Axl](https://discuss.elastic.co/u/Axl)\ **Replies:** 16\ **Last updated:** [March 24, 2020, 12:36pm UTC](https://discuss.elastic.co/t/my-logstash-conf-file-is-not-creating-a-test-field/223711 "2020-03-24T12:36:48Z")
I am using ELK 7.6.1. My configuration file /etc/logstash/conf.d/logstash-cowrie.conf does not appear to be taken in account. The logs show however that the file is being read (see below). So, probably it is not doing w… --- ## [Problems with run logstash as a service](https://discuss.elastic.co/t/problems-with-run-logstash-as-a-service/224289)
**Author:** [@mimimike](https://discuss.elastic.co/u/mimimike)\ **Replies:** 2\ **Last updated:** [March 24, 2020, 12:31pm UTC](https://discuss.elastic.co/t/problems-with-run-logstash-as-a-service/224289 "2020-03-24T12:31:04Z")
Hi everyone, I am not be able to run logstash as a service. When i run the command 'sudo systemctl start logstash.service', the status of logstash change to active (running) but it isn't true. When I check the status o… --- ## [Log patterns are different for single pipeline and multiple pipelines](https://discuss.elastic.co/t/log-patterns-are-different-for-single-pipeline-and-multiple-pipelines/224810)
**Author:** [@MioOgbeni](https://discuss.elastic.co/u/MioOgbeni)\ **Replies:** 0\ **Last updated:** [March 24, 2020, 11:05am UTC](https://discuss.elastic.co/t/log-patterns-are-different-for-single-pipeline-and-multiple-pipelines/224810 "2020-03-24T11:05:20Z")
Hi, can I have a question? What may cause that logstash has different log patterns, when I use multiple pipes? I found this problem, bud still without answer. https://github.com/elastic/logstash/issues/11678 Can anyon… --- ## [Can't start logstash with SSL Basic licence](https://discuss.elastic.co/t/cant-start-logstash-with-ssl-basic-licence/224794)
**Author:** [@yhzami](https://discuss.elastic.co/u/yhzami)\ **Replies:** 0\ **Last updated:** [March 24, 2020, 9:44am UTC](https://discuss.elastic.co/t/cant-start-logstash-with-ssl-basic-licence/224794 "2020-03-24T09:44:57Z")
Hello, I'm new with ELK and i start to configure a cluster 3 elasticsearch nodes 1 Kibana 2 logstash nodes without SSL the configuation is good without problem, put after setting SSL, TLS et HTTPS refering to https:… --- ## [Exception in logstash filter](https://discuss.elastic.co/t/exception-in-logstash-filter/224792)
**Author:** [@jagan](https://discuss.elastic.co/u/jagan)\ **Replies:** 0\ **Last updated:** [March 24, 2020, 9:30am UTC](https://discuss.elastic.co/t/exception-in-logstash-filter/224792 "2020-03-24T09:30:37Z")
Hi guru's, Please help me out i am try to run the job from logstash when it is about to run from database and get the proper output in the output plugin.but the issue is am facing the error: \[ERROR\]\[logstash.pipeline… --- ## [Multiple Elasticsearch Output Pipeline](https://discuss.elastic.co/t/multiple-elasticsearch-output-pipeline/224736)
**Author:** [@Marc\_Wolff](https://discuss.elastic.co/u/Marc_Wolff)\ **Replies:** 2\ **Last updated:** [March 24, 2020, 8:04am UTC](https://discuss.elastic.co/t/multiple-elasticsearch-output-pipeline/224736 "2020-03-24T08:04:12Z")
Hi folks, we have a pipeline with beats input and multiple ES Hosts outputs. But if one of the ES Hosts unreachable or the index is full, all other Outputs also doesn´t get data. Is that normal, or a misconfiguration… --- ## [Ingest logs using API from cloud tools](https://discuss.elastic.co/t/ingest-logs-using-api-from-cloud-tools/224775)
**Author:** [@kvpk16](https://discuss.elastic.co/u/kvpk16)\ **Replies:** 0\ **Last updated:** [March 24, 2020, 7:15am UTC](https://discuss.elastic.co/t/ingest-logs-using-api-from-cloud-tools/224775 "2020-03-24T07:15:18Z")
Hi - Can someone guide me how to ingest logs from cloud tool like Aruba wireless, Meraki into ELK using API. --- ## [Can I dynamically change the path to the file to use as an input in Logstash configuration file?](https://discuss.elastic.co/t/can-i-dynamically-change-the-path-to-the-file-to-use-as-an-input-in-logstash-configuration-file/224761)
**Author:** [@david4](https://discuss.elastic.co/u/david4)\ **Replies:** 0\ **Last updated:** [March 24, 2020, 4:51am UTC](https://discuss.elastic.co/t/can-i-dynamically-change-the-path-to-the-file-to-use-as-an-input-in-logstash-configuration-file/224761 "2020-03-24T04:51:16Z")
Hi everyone, I'm new to the ELK stack and figuring out how to solve this problem. Let's say I have a following directory structure: /logs/dir\_x/dir\_y/dir\_z/task1/log1.csv, code1.py /logs/dir\_a/dir\_b/dir\_c/task2/log2.c… --- ## [JDBC plugin: Doesn’t work when select the table name is a number](https://discuss.elastic.co/t/jdbc-plugin-doesn-t-work-when-select-the-table-name-is-a-number/224636)
**Author:** [@111207](https://discuss.elastic.co/u/111207)\ **Replies:** 2\ **Last updated:** [March 24, 2020, 2:43am UTC](https://discuss.elastic.co/t/jdbc-plugin-doesn-t-work-when-select-the-table-name-is-a-number/224636 "2020-03-24T02:43:30Z")
Hi Guy I am using the JDBC plugin and connect to mysql database and I have multiple data table. The name of table is a number.When I run the logstash config have an error Syntax message:You have an error in your SQL sy… --- ## [About event.cancel](https://discuss.elastic.co/t/about-event-cancel/224335)
**Author:** [@buch11](https://discuss.elastic.co/u/buch11)\ **Replies:** 3\ **Last updated:** [March 24, 2020, 12:33am UTC](https://discuss.elastic.co/t/about-event-cancel/224335 "2020-03-24T00:33:26Z")
Hey Guys, In my logstash setup, I have a chain of three plugins input plugin \> custom plugin 1 \> custom plugin 2 \> output plugin In my currently implementation, I have wrapped code of both custom plugins inside begin.… --- ## [Issue with outputting binary files](https://discuss.elastic.co/t/issue-with-outputting-binary-files/224191)
**Author:** [@mikehwang](https://discuss.elastic.co/u/mikehwang)\ **Replies:** 1\ **Last updated:** [March 23, 2020, 11:17pm UTC](https://discuss.elastic.co/t/issue-with-outputting-binary-files/224191 "2020-03-23T23:17:24Z")
I'm trying to use Logstash to ingest emails and write the messages into Elasticsearch and write attachments onto the filesystem (maybe S3 later). My issue is that the files are not getting written correctly so not reada… --- ## [Logstash hold indexes when ES is overloaded](https://discuss.elastic.co/t/logstash-hold-indexes-when-es-is-overloaded/224729)
**Author:** [@Fernando\_Ruben\_Otero](https://discuss.elastic.co/u/Fernando_Ruben_Otero)\ **Replies:** 1\ **Last updated:** [March 23, 2020, 10:27pm UTC](https://discuss.elastic.co/t/logstash-hold-indexes-when-es-is-overloaded/224729 "2020-03-23T22:27:45Z")
Hi all Our current architecture for indexing is: we have a kafka topic from which logstash reads and then indexes in elastic. We sometimes have peaks of indexing which is causing search issues. I would like to be able … --- ## [How can I solve this error with connection reset by peer?](https://discuss.elastic.co/t/how-can-i-solve-this-error-with-connection-reset-by-peer/222972)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 8\ **Last updated:** [March 23, 2020, 9:57pm UTC](https://discuss.elastic.co/t/how-can-i-solve-this-error-with-connection-reset-by-peer/222972 "2020-03-23T21:57:06Z")
I keep seeing this error in Logstash and I'm afraid that it's causing data loss. A get a couple a day: \[2020-03-10T15:08:32,460\]\[INFO \]\[org.logstash.beats.BeatsHandler\] \[local: 0.0.0.0:5044, remote: 192.196.27.1:2406\] H… --- ## [Logstash 7.6.1 - getting illegal access operation error](https://discuss.elastic.co/t/logstash-7-6-1-getting-illegal-access-operation-error/224334)
**Author:** [@snehid](https://discuss.elastic.co/u/snehid)\ **Replies:** 2\ **Last updated:** [March 23, 2020, 9:32pm UTC](https://discuss.elastic.co/t/logstash-7-6-1-getting-illegal-access-operation-error/224334 "2020-03-23T21:32:49Z")
Team, Saw the recommendations for the above mentioned errors in logstash7.6.1 add this in the logstash jvm.options file? and with scheduler logstash service, restarting again and again do suggest! --- ## [Reading regex from file in a filter](https://discuss.elastic.co/t/reading-regex-from-file-in-a-filter/224724)
**Author:** [@mnrdammu](https://discuss.elastic.co/u/mnrdammu)\ **Replies:** 0\ **Last updated:** [March 23, 2020, 8:13pm UTC](https://discuss.elastic.co/t/reading-regex-from-file-in-a-filter/224724 "2020-03-23T20:13:07Z")
I created a log stash filter plugin which runs the regex match against a field, and if it matches, create a new field 'flag' and sets the value as 'Y' Currently I am using the regex inside the ruby file of my filter but… --- ## [Ingesting oracle log file into elasticsearch](https://discuss.elastic.co/t/ingesting-oracle-log-file-into-elasticsearch/224723)
**Author:** [@perezs](https://discuss.elastic.co/u/perezs)\ **Replies:** 0\ **Last updated:** [March 23, 2020, 8:04pm UTC](https://discuss.elastic.co/t/ingesting-oracle-log-file-into-elasticsearch/224723 "2020-03-23T20:04:42Z")
what is the best way to ingest oracle log files in elasticsearch 7.6 --- ## [Duplicate Records in Kibana](https://discuss.elastic.co/t/duplicate-records-in-kibana/224721)
**Author:** [@Amit\_Garg](https://discuss.elastic.co/u/Amit_Garg)\ **Replies:** 0\ **Last updated:** [March 23, 2020, 7:48pm UTC](https://discuss.elastic.co/t/duplicate-records-in-kibana/224721 "2020-03-23T19:48:00Z")
Hi, I am getting duplicate records in Kibana. I am using the below config in logstash. But in the kibana it showing the \_id value as %{\[@metadata\]\[fingerprint\]} Please provide the solution to this. Logstash.conf filt… --- ## [Logstash having problem with elasticsearch filter](https://discuss.elastic.co/t/logstash-having-problem-with-elasticsearch-filter/224235)
**Author:** [@zozo6015](https://discuss.elastic.co/u/zozo6015)\ **Replies:** 4\ **Last updated:** [March 23, 2020, 7:23pm UTC](https://discuss.elastic.co/t/logstash-having-problem-with-elasticsearch-filter/224235 "2020-03-23T19:23:05Z")
Hello, In the logstash pipeline I am querying the same index to get some data from the previous day The code looks like this: elasticsearch { hosts =\> \["127.0.0.1:9200"\] index =\> "index" query =\> "c… --- ## [Variable length log](https://discuss.elastic.co/t/variable-length-log/224702)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 1\ **Last updated:** [March 23, 2020, 7:10pm UTC](https://discuss.elastic.co/t/variable-length-log/224702 "2020-03-23T19:10:25Z")
Grettings, I have disk usage logs, these logs can have information of one or many disk, I am insterested in the "percent full" ex: Disk/\[C\]/percent full=71.63, Disk/\[C\]/MB free=11591, Disk/\[C\]/MB total=40857, Disk/\[E\]… --- ## [Time stamp in http\_poller input plugin](https://discuss.elastic.co/t/time-stamp-in-http-poller-input-plugin/224709)
**Author:** [@Aman\_Joshi](https://discuss.elastic.co/u/Aman_Joshi)\ **Replies:** 1\ **Last updated:** [March 23, 2020, 7:07pm UTC](https://discuss.elastic.co/t/time-stamp-in-http-poller-input-plugin/224709 "2020-03-23T19:07:12Z")
Continuing the discussion from Timestamp operation in http\_poller input: --- ## [How to rotate files in output plugin](https://discuss.elastic.co/t/how-to-rotate-files-in-output-plugin/224698)
**Author:** [@akhilsharma.in](https://discuss.elastic.co/u/akhilsharma.in)\ **Replies:** 1\ **Last updated:** [March 23, 2020, 7:03pm UTC](https://discuss.elastic.co/t/how-to-rotate-files-in-output-plugin/224698 "2020-03-23T19:03:20Z")
Team, I'm writing an output conf file and I want the output to be stored in a file. My conf file will get events once a day. I want when my conf file gets data tomorrow, it replaces the file of previous day to prevent f… --- ## [Missing files input in logstash?](https://discuss.elastic.co/t/missing-files-input-in-logstash/224708)
**Author:** [@Emna1](https://discuss.elastic.co/u/Emna1)\ **Replies:** 0\ **Last updated:** [March 23, 2020, 6:09pm UTC](https://discuss.elastic.co/t/missing-files-input-in-logstash/224708 "2020-03-23T18:09:44Z")
For example when every day i have an incoming csv files, if i have file03-03-2020.csv file05-03-2020.csv as input , how can I display in a field "missing files" that there is file missing in 04-03-2020 please any id… --- ## [Fileinput - force parsing (don't wait for all the files to be discovered)](https://discuss.elastic.co/t/fileinput-force-parsing-dont-wait-for-all-the-files-to-be-discovered/224707)
**Author:** [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\ **Replies:** 0\ **Last updated:** [March 23, 2020, 6:08pm UTC](https://discuss.elastic.co/t/fileinput-force-parsing-dont-wait-for-all-the-files-to-be-discovered/224707 "2020-03-23T18:08:32Z")
Hi there, I'm using the logstash file input plugin to ingest many CSV files in a few directories. Is there any way to force the input plugin to pass the files to the csv filter stage quicker? It seems as though the fil… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=384) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=386)