# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=180 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 181 --- ## [Replace file path with it's content](https://discuss.elastic.co/t/replace-file-path-with-its-content/289599)
**Author:** [@Mohammed\_Sayeed](https://discuss.elastic.co/u/Mohammed_Sayeed)\ **Replies:** 0\ **Last updated:** [November 18, 2021, 1:33pm UTC](https://discuss.elastic.co/t/replace-file-path-with-its-content/289599 "2021-11-18T13:33:14Z")
I'm trying to read a csv and replace file path with it's content. for example i have a csv file like this: data1, data2, path1 data3, data4, path2 path1 data:- data5 path2 data:- data6 i want to create a document l… --- ## [Query error with .keywords](https://discuss.elastic.co/t/query-error-with-keywords/289587)
**Author:** [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Replies:** 0\ **Last updated:** [November 18, 2021, 12:41pm UTC](https://discuss.elastic.co/t/query-error-with-keywords/289587 "2021-11-18T12:41:42Z")
Hello! I am trying to enrich my index with logstash's elastic filter. the problem is that it analyzes instead of taking the concrete value. How can I query to use the .keyword? im want something like this: elasti… --- ## [413 Request Entity Too Large - how to debug this? \[Elastic\]\[Logstash\]\[Filebeat\]](https://discuss.elastic.co/t/413-request-entity-too-large-how-to-debug-this-elastic-logstash-filebeat/289584)
**Author:** [@bhrt](https://discuss.elastic.co/u/bhrt)\ **Replies:** 0\ **Last updated:** [November 18, 2021, 11:10am UTC](https://discuss.elastic.co/t/413-request-entity-too-large-how-to-debug-this-elastic-logstash-filebeat/289584 "2021-11-18T11:10:59Z")
Hello, I am facing issue with my logstash configuration. I am feeding logstash with log events using filebeat as a client. There are multiple logfiles (with different file formats) parsed in single logstash pipeline (I… --- ## [Combine Information from different log lines into one event](https://discuss.elastic.co/t/combine-information-from-different-log-lines-into-one-event/289518)
**Author:** [@mangeshmj1992](https://discuss.elastic.co/u/mangeshmj1992)\ **Replies:** 3\ **Last updated:** [November 18, 2021, 10:58am UTC](https://discuss.elastic.co/t/combine-information-from-different-log-lines-into-one-event/289518 "2021-11-18T10:58:58Z")
Hello team, I have case where information is being displayed in different lines. These lines are not even consecutive lines. Unique field is order ID. I have to triggered email if orderID with Exit came after 30 min of… --- ## [Problems in outputting files to shared unit with Logstash in Windows](https://discuss.elastic.co/t/problems-in-outputting-files-to-shared-unit-with-logstash-in-windows/289423)
**Author:** [@rrodrmal\_everis](https://discuss.elastic.co/u/rrodrmal_everis)\ **Replies:** 2\ **Last updated:** [November 18, 2021, 8:05am UTC](https://discuss.elastic.co/t/problems-in-outputting-files-to-shared-unit-with-logstash-in-windows/289423 "2021-11-18T08:05:51Z")
Hello all: First publication in this forum for me. I hope I'm not breaking a bunch of guidelines. We have a Logstash 7.11.1 installed over a Windows Server 2016 ingesting log files via filebeat. As output, both an Ela… --- ## [Could not index event to Elasticsearch - object mapping for customersegment\_age\_classe tried to parse field customersegment\_age\_classe as object, but found a concrete value](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-object-mapping-for-customersegment-age-classe-tried-to-parse-field-customersegment-age-classe-as-object-but-found-a-concrete-value/289433)
**Author:** [@Maksonsse](https://discuss.elastic.co/u/Maksonsse)\ **Replies:** 1\ **Last updated:** [November 18, 2021, 7:45am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-object-mapping-for-customersegment-age-classe-tried-to-parse-field-customersegment-age-classe-as-object-but-found-a-concrete-value/289433 "2021-11-18T07:45:19Z")
Hello everyone and thanks for passing by ! I'm actually trying to import data into Elasticsearch (with ELK stack) from a view in a dockerized mysql with Logstash but i'm failing miserably for some weeks now as i'm a com… --- ## [Urgent Help Required :: Need to sync specific Feilds from three different indexes to single Index using Logstash pipeline](https://discuss.elastic.co/t/urgent-help-required-need-to-sync-specific-feilds-from-three-different-indexes-to-single-index-using-logstash-pipeline/289535)
**Author:** [@BASHEER\_DS](https://discuss.elastic.co/u/BASHEER_DS)\ **Replies:** 0\ **Last updated:** [November 18, 2021, 6:59am UTC](https://discuss.elastic.co/t/urgent-help-required-need-to-sync-specific-feilds-from-three-different-indexes-to-single-index-using-logstash-pipeline/289535 "2021-11-18T06:59:20Z")
Hi, I am trying to sync specific fields from three different index in to single index using Logstash pipeline. Am using Elasticsearch as input, filter and output plugins, But while creating Pipeline am getting error(No … --- ## [How to sync specific fields based on common values of three different collections in MongoDB to ElasticSearch using logstash pipeline](https://discuss.elastic.co/t/how-to-sync-specific-fields-based-on-common-values-of-three-different-collections-in-mongodb-to-elasticsearch-using-logstash-pipeline/289532)
**Author:** [@BASHEER\_DS](https://discuss.elastic.co/u/BASHEER_DS)\ **Replies:** 0\ **Last updated:** [November 18, 2021, 6:42am UTC](https://discuss.elastic.co/t/how-to-sync-specific-fields-based-on-common-values-of-three-different-collections-in-mongodb-to-elasticsearch-using-logstash-pipeline/289532 "2021-11-18T06:42:54Z")
In Logstash Pipeline we are using mongoDB as input plugin and we are using three different collection to sync specific Fields in Elasticsearch Index based on common values of different collections. But How to sync specif… --- ## [How to parse long numbers?](https://discuss.elastic.co/t/how-to-parse-long-numbers/288783)
**Author:** [@rachelyang](https://discuss.elastic.co/u/rachelyang)\ **Replies:** 1\ **Last updated:** [November 17, 2021, 1:29pm UTC](https://discuss.elastic.co/t/how-to-parse-long-numbers/288783 "2021-11-17T13:29:46Z")
I have set up NUMBER type for item duration in the pattern of Logstash configuration as %{NUMBER:Duration} It can parse most of the numbers in the log files, like 9994568.872 , 20903.23, etc.. But it cannot parse the… --- ## [Logstash not sending data into kibana](https://discuss.elastic.co/t/logstash-not-sending-data-into-kibana/289004)
**Author:** [@sai92](https://discuss.elastic.co/u/sai92)\ **Replies:** 1\ **Last updated:** [November 17, 2021, 1:21pm UTC](https://discuss.elastic.co/t/logstash-not-sending-data-into-kibana/289004 "2021-11-17T13:21:19Z")
Hello everyone I use logstash to read the json file and ingest data into index in Elasticsearch. But it works fine when run the command of /bin/logstash -f /etc/logstash/conf.d/log.test. But when I start it as service wi… --- ## [Import only delta (differences) from csv into mysql](https://discuss.elastic.co/t/import-only-delta-differences-from-csv-into-mysql/289283)
**Author:** [@Mihai\_Stuparu](https://discuss.elastic.co/u/Mihai_Stuparu)\ **Replies:** 1\ **Last updated:** [November 17, 2021, 1:19pm UTC](https://discuss.elastic.co/t/import-only-delta-differences-from-csv-into-mysql/289283 "2021-11-17T13:19:45Z")
Hello, I am using logstash to call an API (input), save it as .csv (filter) and upload it into a mysql table (output). All is good. But I need to do it each day (or each night rather): call that API, get the .csv and u… --- ## [Logstash http\_poller input / http filter design question](https://discuss.elastic.co/t/logstash-http-poller-input-http-filter-design-question/289012)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 1\ **Last updated:** [November 17, 2021, 1:16pm UTC](https://discuss.elastic.co/t/logstash-http-poller-input-http-filter-design-question/289012 "2021-11-17T13:16:52Z")
Good Afternoon, I am starting to pull in AppDynamics metric information into our Elastic stack for persistence purposes. I have 5 metrics from each of 13 servers that I need to capture and my design in my Stage environ… --- ## [Parse one line as two lines](https://discuss.elastic.co/t/parse-one-line-as-two-lines/289073)
**Author:** [@Sjap1](https://discuss.elastic.co/u/Sjap1)\ **Replies:** 1\ **Last updated:** [November 17, 2021, 1:10pm UTC](https://discuss.elastic.co/t/parse-one-line-as-two-lines/289073 "2021-11-17T13:10:04Z")
Hello, I'm trying to parse multiple items in one line. My grok pattens match but logstash are outputting the data as arrays; How can i make the output look the where are two lines? input data: 2021-11-01 14:09:34 CET … --- ## [How am I supposed to understand why the field is confusing?](https://discuss.elastic.co/t/how-am-i-supposed-to-understand-why-the-field-is-confusing/289132)
**Author:** [@mrchile](https://discuss.elastic.co/u/mrchile)\ **Replies:** 2\ **Last updated:** [November 17, 2021, 1:07pm UTC](https://discuss.elastic.co/t/how-am-i-supposed-to-understand-why-the-field-is-confusing/289132 "2021-11-17T13:07:49Z")
Good afternoon, colleagues. I have been playing for several hours with this incomprehensible error, how can this be - someone can explain the dependencies of fields between different products, you can link to the documen… --- ## [New field in Weblogic access\_log , Logstash and Kibana](https://discuss.elastic.co/t/new-field-in-weblogic-access-log-logstash-and-kibana/288810)
**Author:** [@raultada](https://discuss.elastic.co/u/raultada)\ **Replies:** 1\ **Last updated:** [November 17, 2021, 1:02pm UTC](https://discuss.elastic.co/t/new-field-in-weblogic-access-log-logstash-and-kibana/288810 "2021-11-17T13:02:57Z")
Hello everyone, I´m new in ELK and there is a request to insert a new information came from weblogic access log ==\> "database name" This is the current weblogic log format (logging/tab http/advanced/Extended Logging Fo… --- ## [Trying to fetch documents from Mongodb using jdbc input plugin in logstash but getting this error messages](https://discuss.elastic.co/t/trying-to-fetch-documents-from-mongodb-using-jdbc-input-plugin-in-logstash-but-getting-this-error-messages/289133)
**Author:** [@Husnain](https://discuss.elastic.co/u/Husnain)\ **Replies:** 2\ **Last updated:** [November 17, 2021, 11:36am UTC](https://discuss.elastic.co/t/trying-to-fetch-documents-from-mongodb-using-jdbc-input-plugin-in-logstash-but-getting-this-error-messages/289133 "2021-11-17T11:36:58Z")
Hi All.There are documents in my Mongodb which I am trying to fetch out using logstash input jdbc plugin.I am using mongodb version 4.2.15 and logstash version is 7.10.1.When I run the logstash configuration pipeline I g… --- ## [How to run logstash on WSL (Ubuntu 20.04.3 LTS)](https://discuss.elastic.co/t/how-to-run-logstash-on-wsl-ubuntu-20-04-3-lts/289414)
**Author:** [@Stephy\_Jacob](https://discuss.elastic.co/u/Stephy_Jacob)\ **Replies:** 1\ **Last updated:** [November 17, 2021, 9:43am UTC](https://discuss.elastic.co/t/how-to-run-logstash-on-wsl-ubuntu-20-04-3-lts/289414 "2021-11-17T09:43:39Z")
Hi, Could you please help with running logstash as a service in WSL (Ubuntu 20.04.3 LTS) As of now I get below error message sudo systemctl start logstash.service System has not been booted with systemd as init system… --- ## [Call recursively an API](https://discuss.elastic.co/t/call-recursively-an-api/289284)
**Author:** [@Mihai\_Stuparu](https://discuss.elastic.co/u/Mihai_Stuparu)\ **Replies:** 2\ **Last updated:** [November 17, 2021, 7:47am UTC](https://discuss.elastic.co/t/call-recursively-an-api/289284 "2021-11-17T07:47:30Z")
Hello, Using below basic http\_poller request I will obtain a .json file that contains an attribute called "id". For each "id" I need to call other APIs, having the "id" value in the url: https://..../id. How can I ach… --- ## [Convert my sql to elasticsearch without logstash](https://discuss.elastic.co/t/convert-my-sql-to-elasticsearch-without-logstash/289291)
**Author:** [@Rizky\_Hudha](https://discuss.elastic.co/u/Rizky_Hudha)\ **Replies:** 1\ **Last updated:** [November 16, 2021, 7:45pm UTC](https://discuss.elastic.co/t/convert-my-sql-to-elasticsearch-without-logstash/289291 "2021-11-16T19:45:03Z")
is there a way other than using logstash to convert mysql to Elasticsearch. I just got a new task, to create a generic processor to convert mysql to Elasticsearch --- ## [Logstash - XML parsing including nested objects](https://discuss.elastic.co/t/logstash-xml-parsing-including-nested-objects/289323)
**Author:** [@anandd4](https://discuss.elastic.co/u/anandd4)\ **Replies:** 1\ **Last updated:** [November 16, 2021, 6:10pm UTC](https://discuss.elastic.co/t/logstash-xml-parsing-including-nested-objects/289323 "2021-11-16T18:10:46Z")
Problem - Unable to convert object fields to flat fields upon parsing the XML Input - Dynamic Soap xml Example - \ **Author:** [@kyats5000](https://discuss.elastic.co/u/kyats5000)\ **Replies:** 1\ **Last updated:** [November 16, 2021, 6:02pm UTC](https://discuss.elastic.co/t/using-mutate-rename-source-and-destination-json-plugin-question-for-filter/289337 "2021-11-16T18:02:21Z") HI there I am trying to find out what the syntax is for renaming fields from an incoming json message. We dropped the message at first to remove metadata but it is unclear to me in subsequent segments of the pipeline co… --- ## [Log parsing issue](https://discuss.elastic.co/t/log-parsing-issue/289306)
**Author:** [@Divya\_Bansal](https://discuss.elastic.co/u/Divya_Bansal)\ **Replies:** 5\ **Last updated:** [November 16, 2021, 6:00pm UTC](https://discuss.elastic.co/t/log-parsing-issue/289306 "2021-11-16T18:00:33Z")
I am having a log in the following format:- {"@timestamp":"2021-08-04T09:57:25.141Z","@metadata":{"beat":"filebeat","type":"\_doc","version":"7.6.3"},"log":{"offset":10413931,"file":{"path":"api/api.log"}},"message":"\[ER… --- ## [Logstahs Filter](https://discuss.elastic.co/t/logstahs-filter/289078)
**Author:** [@John\_snow](https://discuss.elastic.co/u/John_snow)\ **Replies:** 9\ **Last updated:** [November 16, 2021, 5:57pm UTC](https://discuss.elastic.co/t/logstahs-filter/289078 "2021-11-16T17:57:46Z")
Can we apply 2 filter in 1 logstash conf file. I wanted to parse 1 log twice and send the message to multiple outputs. e-g 1 filter to remove some fields and send the message to output 2nd filter doing another check an… --- ## [CSV quote char inside data](https://discuss.elastic.co/t/csv-quote-char-inside-data/289290)
**Author:** [@nosql\_injection](https://discuss.elastic.co/u/nosql_injection)\ **Replies:** 1\ **Last updated:** [November 16, 2021, 5:44pm UTC](https://discuss.elastic.co/t/csv-quote-char-inside-data/289290 "2021-11-16T17:44:38Z")
Hi there, I'm trying to ingest syslog data with the CSV filter. I guess the issue is related to the " inside the URL part (acme.org&iid={"-123456\*\*"\*\*:4}&sid=123458&tid=123) of the log, which is the same char as the "… --- ## [Creating pipeline Using beats in Windows and sending logs to logstash in linux](https://discuss.elastic.co/t/creating-pipeline-using-beats-in-windows-and-sending-logs-to-logstash-in-linux/289191)
**Author:** [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\ **Replies:** 4\ **Last updated:** [November 16, 2021, 4:30pm UTC](https://discuss.elastic.co/t/creating-pipeline-using-beats-in-windows-and-sending-logs-to-logstash-in-linux/289191 "2021-11-16T16:30:09Z")
Hi, I am creating a pipeline using beats in windows to extract logs in form of txt file in multiple directories and send it to the Linux(server) to logstash to es/kibana. While running logstash conf pipeline I am getti… --- ## [I can't record the current date from the log string in the @timestamp field](https://discuss.elastic.co/t/i-cant-record-the-current-date-from-the-log-string-in-the-timestamp-field/289201)
**Author:** [@vanro](https://discuss.elastic.co/u/vanro)\ **Replies:** 2\ **Last updated:** [November 16, 2021, 1:29pm UTC](https://discuss.elastic.co/t/i-cant-record-the-current-date-from-the-log-string-in-the-timestamp-field/289201 "2021-11-16T13:29:30Z")
Hello, I can't record the current date from the log string in the @timestamp field and convert the time field from the string type to the date type. Can you tell me what I'm doing wrong? My configuration is shown below.. … --- ## [How to extract fields of an array with logstash?](https://discuss.elastic.co/t/how-to-extract-fields-of-an-array-with-logstash/289289)
**Author:** [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Replies:** 1\ **Last updated:** [November 16, 2021, 11:36am UTC](https://discuss.elastic.co/t/how-to-extract-fields-of-an-array-with-logstash/289289 "2021-11-16T11:36:18Z")
hello! i have a field with this format value: \["x@correo.es","x@correo.com"\] and i want to use a scanner tu extract the nested fields with something like this: ruby { code =\> " event.set('prueba', ev… --- ## [Logstash Error](https://discuss.elastic.co/t/logstash-error/289310)
**Author:** [@Angad\_Panesar1](https://discuss.elastic.co/u/Angad_Panesar1)\ **Replies:** 0\ **Last updated:** [November 16, 2021, 11:04am UTC](https://discuss.elastic.co/t/logstash-error/289310 "2021-11-16T11:04:27Z")
Hi, I am running the logstash config file and I am getting the error shown below. \[ERROR\]\[logstash.outputs.Elasticsearch\]\[main\] Failed to install template {:message=\>"Got response code '400' contacting Elasticsearch at … --- ## [Logstash Elasticsearch filer query to filter data from existing data](https://discuss.elastic.co/t/logstash-elasticsearch-filer-query-to-filter-data-from-existing-data/289304)
**Author:** [@DeepakStile](https://discuss.elastic.co/u/DeepakStile)\ **Replies:** 1\ **Last updated:** [November 16, 2021, 11:02am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-filer-query-to-filter-data-from-existing-data/289304 "2021-11-16T11:02:38Z")
I am trying to add existing data from Elasticsearch index to a current event of logstash using Elasticsearch filter current logstash event is id,status,date 123456789,ByMobile,Success Existing Data like id,status,da… --- ## [Logstash Nested JSON Parsing and Transforming data](https://discuss.elastic.co/t/logstash-nested-json-parsing-and-transforming-data/289255)
**Author:** [@zx8086](https://discuss.elastic.co/u/zx8086)\ **Replies:** 2\ **Last updated:** [November 16, 2021, 5:18am UTC](https://discuss.elastic.co/t/logstash-nested-json-parsing-and-transforming-data/289255 "2021-11-16T05:18:58Z")
How would I using the json filter to split the following json document and target / modify the time stamp (event.ts) ? {"peer":"EC2LEBOQ77XSF46QEKYYYNKBTVATU33K","agent\_name":"test-device","event":{"ts":1637034155000,"t… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=179) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=181)