# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=405 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 406 --- ## [Single field object to Elasticsearch using Logstash](https://discuss.elastic.co/t/single-field-object-to-elasticsearch-using-logstash/215735)
**Author:** [@KrishK](https://discuss.elastic.co/u/KrishK)\ **Replies:** 7\ **Last updated:** [February 13, 2020, 9:31am UTC](https://discuss.elastic.co/t/single-field-object-to-elasticsearch-using-logstash/215735 "2020-02-13T09:31:17Z")
Hi Support, I have group concat field values like :slight\_smile: Document\_Type 1,2,3,4,5,6 In Elasticsearch field mapping with object type "Document\_Type": { "properties": { "d": { "typ… --- ## [No logstash directory in /usr/share](https://discuss.elastic.co/t/no-logstash-directory-in-usr-share/218851)
**Author:** [@kevin5617](https://discuss.elastic.co/u/kevin5617)\ **Replies:** 3\ **Last updated:** [February 13, 2020, 7:27am UTC](https://discuss.elastic.co/t/no-logstash-directory-in-usr-share/218851 "2020-02-13T07:27:18Z")
I am running Logstash with Docker and after I pull the docker image, there is no logstash directory in /usr/share to add config files. --- ## [McAfee EPO logs to Logstash, XML parsing failure, XML: missing tag start, XML fails to generate datastructure](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827)
**Author:** [@Purk](https://discuss.elastic.co/u/Purk)\ **Replies:** 7\ **Last updated:** [February 13, 2020, 6:55am UTC](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827 "2020-02-13T06:55:26Z")
Hi I am looking for a help on how to parse the McAfee ePO XML logs into the logstash. I tried to use a XML filter plugin, but it fails to create an actual datastructure of the XML file. I have seen a similar topic in … --- ## [Sum field values from nested json and put them in new field using logstash ruby plugin](https://discuss.elastic.co/t/sum-field-values-from-nested-json-and-put-them-in-new-field-using-logstash-ruby-plugin/219084)
**Author:** [@Gangadhar\_Mahadevan](https://discuss.elastic.co/u/Gangadhar_Mahadevan)\ **Replies:** 4\ **Last updated:** [February 13, 2020, 1:31am UTC](https://discuss.elastic.co/t/sum-field-values-from-nested-json-and-put-them-in-new-field-using-logstash-ruby-plugin/219084 "2020-02-13T01:31:49Z")
This is my input stored in .log file {"@timestamp":"2020-02-12T17:53:54.305Z","beat":{"hostname":"xx.xx","name":"xx.xx","version":"5.2.1"},"input\_type":"log","log\_dc":"xx","log\_dc\_type":"rg","message":"{\\"server\_name\\":… --- ## [Adjusting a ISO8601 @timestamp field](https://discuss.elastic.co/t/adjusting-a-iso8601-timestamp-field/218874)
**Author:** [@thethomp](https://discuss.elastic.co/u/thethomp)\ **Replies:** 3\ **Last updated:** [February 12, 2020, 10:16pm UTC](https://discuss.elastic.co/t/adjusting-a-iso8601-timestamp-field/218874 "2020-02-12T22:16:20Z")
Hi all, I'm a bit stuck and confused about how to use the Logstash date plugin for what I'm trying to do. My situation is such that I have incoming data (coming from kafka input) populating the @timestamp field in ISO8… --- ## [How to Code a Try/Catch in Logstash Config File?](https://discuss.elastic.co/t/how-to-code-a-try-catch-in-logstash-config-file/218413)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 5\ **Last updated:** [February 12, 2020, 7:15pm UTC](https://discuss.elastic.co/t/how-to-code-a-try-catch-in-logstash-config-file/218413 "2020-02-12T19:15:25Z")
Hi Logstash Iron Chefs, I am running Docker Container Logstash 7.4.0 (I know, I need to get the latest version) and my Logstash filter has a lot of code blocks like these: if \[MyData\]\[dataA\]\[dataB\] == 0 { transla… --- ## [Exclude characters](https://discuss.elastic.co/t/exclude-characters/218648)
**Author:** [@Gleison\_Rodrigo](https://discuss.elastic.co/u/Gleison_Rodrigo)\ **Replies:** 3\ **Last updated:** [February 12, 2020, 7:03pm UTC](https://discuss.elastic.co/t/exclude-characters/218648 "2020-02-12T19:03:38Z")
I have the following entry: mail@domain.com, How can I delete the \<\>, characters ? --- ## [Cassandra to Kibana](https://discuss.elastic.co/t/cassandra-to-kibana/218062)
**Author:** [@Karthik2411](https://discuss.elastic.co/u/Karthik2411)\ **Replies:** 2\ **Last updated:** [February 12, 2020, 6:56pm UTC](https://discuss.elastic.co/t/cassandra-to-kibana/218062 "2020-02-12T18:56:56Z")
I am trying to constantly check the status of cassandra for every one hour and send all the information into kibana so that I can create a watcher that reports whenever I find a failure in the log. Would it be possible f… --- ## [Failed to rpm GPG-KEY-elasticsearch](https://discuss.elastic.co/t/failed-to-rpm-gpg-key-elasticsearch/219050)
**Author:** [@saad](https://discuss.elastic.co/u/saad)\ **Replies:** 0\ **Last updated:** [February 12, 2020, 5:24pm UTC](https://discuss.elastic.co/t/failed-to-rpm-gpg-key-elasticsearch/219050 "2020-02-12T17:24:31Z")
I am using centos 7. java 8 installed and I am trying to get the GPG key but failed rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch how can I solve it please \[root@elk01 ~\]# rpm --import https://artifac… --- ## [Logstash automatically adding port](https://discuss.elastic.co/t/logstash-automatically-adding-port/219047)
**Author:** [@guctum](https://discuss.elastic.co/u/guctum)\ **Replies:** 0\ **Last updated:** [February 12, 2020, 4:54pm UTC](https://discuss.elastic.co/t/logstash-automatically-adding-port/219047 "2020-02-12T16:54:44Z")
I have logstash running on an AWS EC2 instance and am trying to connect it to an AWS Elasticsearch domain. When starting up Logstash, it won't connect to the ES domain, and appends port 443 to the URL of the domain. I ne… --- ## [Can only start an input when all related states are finished with lumberjack protocol error](https://discuss.elastic.co/t/can-only-start-an-input-when-all-related-states-are-finished-with-lumberjack-protocol-error/218933)
**Author:** [@Shlok\_Srivastava1](https://discuss.elastic.co/u/Shlok_Srivastava1)\ **Replies:** 0\ **Last updated:** [February 12, 2020, 9:30am UTC](https://discuss.elastic.co/t/can-only-start-an-input-when-all-related-states-are-finished-with-lumberjack-protocol-error/218933 "2020-02-12T09:30:45Z")
I Had ELK installed on our server, which abruptly stopped working. I am getting the following error in filebeat for the same. I am using 6.8.3 versions for all 2020-02-12T08:49:17.950Z INFO pipeline/output.go:95 Con… --- ## [Logstash date parsing and convert](https://discuss.elastic.co/t/logstash-date-parsing-and-convert/219012)
**Author:** [@Pawel\_Mazurek](https://discuss.elastic.co/u/Pawel_Mazurek)\ **Replies:** 2\ **Last updated:** [February 12, 2020, 3:09pm UTC](https://discuss.elastic.co/t/logstash-date-parsing-and-convert/219012 "2020-02-12T15:09:27Z")
Hi I have some issue with proper parsing string with date value. In my log i have timestamp like this =\> Mon, 10 Feb 2020 09:22:04 CET Now i want to parse this values and create some string field and after all i want … --- ## [How to filter a specific JSON message](https://discuss.elastic.co/t/how-to-filter-a-specific-json-message/217384)
**Author:** [@josemiguellopez](https://discuss.elastic.co/u/josemiguellopez)\ **Replies:** 2\ **Last updated:** [February 12, 2020, 2:58pm UTC](https://discuss.elastic.co/t/how-to-filter-a-specific-json-message/217384 "2020-02-12T14:58:46Z")
Hey guys, how are you? Someone can tell me how I send a specific data from a JSON message, I would like to send the message field "" Broser: chrome "" to elastisearch My question is as a filter to find that specific da… --- ## [Change the output of logstash if some criteria in filter fulfilled](https://discuss.elastic.co/t/change-the-output-of-logstash-if-some-criteria-in-filter-fulfilled/218987)
**Author:** [@Arman\_Ajdani](https://discuss.elastic.co/u/Arman_Ajdani)\ **Replies:** 2\ **Last updated:** [February 12, 2020, 2:56pm UTC](https://discuss.elastic.co/t/change-the-output-of-logstash-if-some-criteria-in-filter-fulfilled/218987 "2020-02-12T14:56:47Z")
Can I have two option on the output module on Logstash (suppose Kafka module) and check if some criteria happen in filter section I should send data to one of the output modules: for example : filter { if \[1\] in sourc… --- ## [Logstash input not ingesting distinct file](https://discuss.elastic.co/t/logstash-input-not-ingesting-distinct-file/218953)
**Author:** [@romainfoulono](https://discuss.elastic.co/u/romainfoulono)\ **Replies:** 1\ **Last updated:** [February 12, 2020, 2:53pm UTC](https://discuss.elastic.co/t/logstash-input-not-ingesting-distinct-file/218953 "2020-02-12T14:53:32Z")
Hello everyone I would like to ingest text and csv files using Logstash to send them into Elasticsearch. The problem is that it's not working and Logstash does not indicate any errors. The index is not created, there's… --- ## [Cannot translate value inside fields](https://discuss.elastic.co/t/cannot-translate-value-inside-fields/217351)
**Author:** [@BIs1](https://discuss.elastic.co/u/BIs1)\ **Replies:** 2\ **Last updated:** [February 12, 2020, 2:07pm UTC](https://discuss.elastic.co/t/cannot-translate-value-inside-fields/217351 "2020-02-12T14:07:16Z")
Hi, I am using logstash 7.3.1 and trying to translate value inside the field after renaming. Basically I want to use translate plugin to translate the value inside the field and populate in same field or different field… --- ## [Logstash jdbc config file with filter](https://discuss.elastic.co/t/logstash-jdbc-config-file-with-filter/218675)
**Author:** [@Vladpov](https://discuss.elastic.co/u/Vladpov)\ **Replies:** 3\ **Last updated:** [February 12, 2020, 2:04pm UTC](https://discuss.elastic.co/t/logstash-jdbc-config-file-with-filter/218675 "2020-02-12T14:04:23Z")
Hey, I'm new with logstash jdbc plugin and somehow I set up the migration of data from MySQL database to Elasticsearch via Logstash by using this configuration: input { jdbc { jdbc\_driver\_library =\> "/usr/share/j… --- ## [Configuration of nginx to send data by tcp to logstash](https://discuss.elastic.co/t/configuration-of-nginx-to-send-data-by-tcp-to-logstash/219001)
**Author:** [@javilez13](https://discuss.elastic.co/u/javilez13)\ **Replies:** 0\ **Last updated:** [February 12, 2020, 1:54pm UTC](https://discuss.elastic.co/t/configuration-of-nginx-to-send-data-by-tcp-to-logstash/219001 "2020-02-12T13:54:09Z")
Hi. I would like to know if it is possible to configure nginx to send the log records by tcp, directly to the logstash. Thanks --- ## [Multiple regions – push or pull?](https://discuss.elastic.co/t/multiple-regions-push-or-pull/218989)
**Author:** [@andrian](https://discuss.elastic.co/u/andrian)\ **Replies:** 0\ **Last updated:** [February 12, 2020, 1:27pm UTC](https://discuss.elastic.co/t/multiple-regions-push-or-pull/218989 "2020-02-12T13:27:16Z")
Which way is better for one ELK cluster running for many regions: Set up a Logstash instance in each region, and push the data to one Elasticsearch (“push”)? Set up a queue (i.e. Redis) in each region, and use one Logs… --- ## [Logstash export variable](https://discuss.elastic.co/t/logstash-export-variable/218983)
**Author:** [@Vivek\_Srivastava](https://discuss.elastic.co/u/Vivek_Srivastava)\ **Replies:** 0\ **Last updated:** [February 12, 2020, 1:02pm UTC](https://discuss.elastic.co/t/logstash-export-variable/218983 "2020-02-12T13:02:37Z")
I have one property file where I have mentioned the HOME path. I want to input this property file in logstash input and then use it in the filter. app.properties: export HOME="C:/Users/ankitamishra2/Desktop/Elastic\_v1/… --- ## [Input PDF in logstash](https://discuss.elastic.co/t/input-pdf-in-logstash/217722)
**Author:** [@Vivek\_Srivastava](https://discuss.elastic.co/u/Vivek_Srivastava)\ **Replies:** 5\ **Last updated:** [February 12, 2020, 12:56pm UTC](https://discuss.elastic.co/t/input-pdf-in-logstash/217722 "2020-02-12T12:56:19Z")
Unable to input PDF in logstash. \[2020-02-03T23:38:35,068\]\[FATAL\]\[logstash.runner \] An unexpected error occurred! {:error=\>#\ **Author:** [@MerceneX](https://discuss.elastic.co/u/MerceneX)\ **Replies:** 1\ **Last updated:** [February 12, 2020, 12:37pm UTC](https://discuss.elastic.co/t/parsing-nginx-for-filebeat-dashboards-with-logstash/218924 "2020-02-12T12:37:44Z") Hi! I've been using Logstash to parse custom logs with great success. What I am having trouble with now, is using that same Logstash and Filebeat configuration to get NGINX logs across. I can get logs to my Kibana insta… --- ## [Logstash cannot parse httpdate even pattern is correct](https://discuss.elastic.co/t/logstash-cannot-parse-httpdate-even-pattern-is-correct/218907)
**Author:** [@rogerwang](https://discuss.elastic.co/u/rogerwang)\ **Replies:** 2\ **Last updated:** [February 12, 2020, 11:38am UTC](https://discuss.elastic.co/t/logstash-cannot-parse-httpdate-even-pattern-is-correct/218907 "2020-02-12T11:38:43Z")
Hi, I was converting the filebeat pipeline to logstash since I need to use the translate filter. everything seems fine except the date filter is giving me the "\_dateparsefailure" and it is not able to parse the time cor… --- ## [Exclude ip range in logstash processing](https://discuss.elastic.co/t/exclude-ip-range-in-logstash-processing/218963)
**Author:** [@ademir.andrade](https://discuss.elastic.co/u/ademir.andrade)\ **Replies:** 0\ **Last updated:** [February 12, 2020, 11:35am UTC](https://discuss.elastic.co/t/exclude-ip-range-in-logstash-processing/218963 "2020-02-12T11:35:28Z")
Good morning, I am facing a big problem regarding the processing of the logstash, it is processing ips like 0.0.0.0 broadcast and multicast, would I have to eliminate these ips in the process? I appreciate the help? --- ## [How to add a new Event in logstash java filter](https://discuss.elastic.co/t/how-to-add-a-new-event-in-logstash-java-filter/218959)
**Author:** [@robert.hauser](https://discuss.elastic.co/u/robert.hauser)\ **Replies:** 0\ **Last updated:** [February 12, 2020, 11:22am UTC](https://discuss.elastic.co/t/how-to-add-a-new-event-in-logstash-java-filter/218959 "2020-02-12T11:22:30Z")
Hi, i cloned the incoming event in the filter, but I do not know how to add it. i have tried. events.add(newEvent); I get an exception: java.util.ConcurrentModificationException: null at java.util.ArrayList$Itr.che… --- ## [Pipelines.yml config - location path inside an S3 bucket](https://discuss.elastic.co/t/pipelines-yml-config-location-path-inside-an-s3-bucket/218957)
**Author:** [@BeMoore](https://discuss.elastic.co/u/BeMoore)\ **Replies:** 0\ **Last updated:** [February 12, 2020, 11:20am UTC](https://discuss.elastic.co/t/pipelines-yml-config-location-path-inside-an-s3-bucket/218957 "2020-02-12T11:20:52Z")
Hi chaps, i know this is a long shot, but has anyone managed to change the pipelines.yml configs to change the location of where the pipeline files are to an S3 bucket before?? i.e. - pipeline.id: main path.config: … --- ## [Opinion on Multiple Pipelines vs Conditionals](https://discuss.elastic.co/t/opinion-on-multiple-pipelines-vs-conditionals/218931)
**Author:** [@moscha](https://discuss.elastic.co/u/moscha)\ **Replies:** 0\ **Last updated:** [February 12, 2020, 9:30am UTC](https://discuss.elastic.co/t/opinion-on-multiple-pipelines-vs-conditionals/218931 "2020-02-12T09:30:10Z")
Hello all, We are working on the implementation of an ELK stack with beats. The general scenario is that we are going to have 1 centralised server for all of the ELK components(Elastic, Logstash, Kibana) and then instal… --- ## [Logstash management with self signed sertificate](https://discuss.elastic.co/t/logstash-management-with-self-signed-sertificate/216981)
**Author:** [@pszemesy](https://discuss.elastic.co/u/pszemesy)\ **Replies:** 3\ **Last updated:** [February 12, 2020, 9:27am UTC](https://discuss.elastic.co/t/logstash-management-with-self-signed-sertificate/216981 "2020-02-12T09:27:08Z")
Hi All, I have secured my Elasticsearch cluster's (3 nodes) http communication as below: xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification\_mode: certifi… --- ## [Kafka output doesnt use winlogbeat template](https://discuss.elastic.co/t/kafka-output-doesnt-use-winlogbeat-template/218310)
**Author:** [@niki](https://discuss.elastic.co/u/niki)\ **Replies:** 1\ **Last updated:** [February 12, 2020, 9:00am UTC](https://discuss.elastic.co/t/kafka-output-doesnt-use-winlogbeat-template/218310 "2020-02-12T09:00:20Z")
I am trying to set up an elk stack with apache kafka as log broker. so here is the situation: elasticsearch, logstash, zookeeper, kafka and kibana installed with docker compose and each one in different container(you … --- ## [Logstash connection to Elasticsearch issue](https://discuss.elastic.co/t/logstash-connection-to-elasticsearch-issue/218911)
**Author:** [@pszemesy](https://discuss.elastic.co/u/pszemesy)\ **Replies:** 0\ **Last updated:** [February 12, 2020, 7:17am UTC](https://discuss.elastic.co/t/logstash-connection-to-elasticsearch-issue/218911 "2020-02-12T07:17:23Z")
Hi All, I have a logstash on an external site and an ES within the local network. The ES can be reached from the site where the logstash is located via an HA-Proxy (https://myapp.mycompany.hu --\> http://myapp.mycompany… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=404) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=406)