# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=428 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 429 --- ## [Remove double quotes from grok output](https://discuss.elastic.co/t/remove-double-quotes-from-grok-output/211822)
**Author:** [@Intissar\_Ayadi](https://discuss.elastic.co/u/Intissar_Ayadi)\ **Replies:** 8\ **Last updated:** [December 23, 2019, 1:47pm UTC](https://discuss.elastic.co/t/remove-double-quotes-from-grok-output/211822 "2019-12-23T13:47:46Z")
Hello, Anyone know how to remove the double quotes out of the grok output ? After grok filter, the ouput is "ip": "7.7.7.7", i would like to remove quotes and have an output like "ip": 7.7.7.7 I'm trying with gsub mut… --- ## [Logstash Pipeline To Pipeline Communication Error Using HTTP Output Plugin](https://discuss.elastic.co/t/logstash-pipeline-to-pipeline-communication-error-using-http-output-plugin/212849)
**Author:** [@rameshkokare1](https://discuss.elastic.co/u/rameshkokare1)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 1:44pm UTC](https://discuss.elastic.co/t/logstash-pipeline-to-pipeline-communication-error-using-http-output-plugin/212849 "2019-12-23T13:44:29Z")
Hello, I am using logstash pipelines with http input/ouput plugin. e.g. pipeline1 gives http request to piepline2 on http port. lot of elasticsearch calls and business logic causing below error in logstash logs and in… --- ## [Logstash Failed to maintain memory/persisted queue for solr\_http output](https://discuss.elastic.co/t/logstash-failed-to-maintain-memory-persisted-queue-for-solr-http-output/212846)
**Author:** [@prakhara](https://discuss.elastic.co/u/prakhara)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 12:34pm UTC](https://discuss.elastic.co/t/logstash-failed-to-maintain-memory-persisted-queue-for-solr-http-output/212846 "2019-12-23T12:34:23Z")
I have set up the logstash to index logs to solr using solr\_http plugin and reading events using file input plugin, but when solr is down the events that are recorded in that time period are lost and logstahs solr\_http p… --- ## [How to concatenate multiple fields into single timestamp](https://discuss.elastic.co/t/how-to-concatenate-multiple-fields-into-single-timestamp/212830)
**Author:** [@Bhaskar3184](https://discuss.elastic.co/u/Bhaskar3184)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 10:12am UTC](https://discuss.elastic.co/t/how-to-concatenate-multiple-fields-into-single-timestamp/212830 "2019-12-23T10:12:57Z")
HI, I am loading data from a text file to logstash using multiline grok, but every time stamp is creating for different fields, can you please help me here Input file: DATE: 2019-12-03 Input File/Folder:1.txt … --- ## [Split fileds from message in discover](https://discuss.elastic.co/t/split-fileds-from-message-in-discover/212834)
**Author:** [@Bhaskar3184](https://discuss.elastic.co/u/Bhaskar3184)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 10:43am UTC](https://discuss.elastic.co/t/split-fileds-from-message-in-discover/212834 "2019-12-23T10:43:27Z")
Hi, Can you please help me in creating separate fields for every value like below: Input file: DATE: 2019-12-03 Input File/Folder:1.txt SHA256 hash of 1.txt: 2f1d3f3343dbc60768a290565c9136f754015438e4ea25770… --- ## [Sending alerts through logstash](https://discuss.elastic.co/t/sending-alerts-through-logstash/212844)
**Author:** [@Sagar\_Mandal](https://discuss.elastic.co/u/Sagar_Mandal)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 12:15pm UTC](https://discuss.elastic.co/t/sending-alerts-through-logstash/212844 "2019-12-23T12:15:30Z")
Hi Team, I wan to send alerts from logstash to telegram. any possible ways. Thanks. --- ## [Permission error after update to 7.5.1 from 7.4.0](https://discuss.elastic.co/t/permission-error-after-update-to-7-5-1-from-7-4-0/212842)
**Author:** [@AstroSlazak](https://discuss.elastic.co/u/AstroSlazak)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 12:04pm UTC](https://discuss.elastic.co/t/permission-error-after-update-to-7-5-1-from-7-4-0/212842 "2019-12-23T12:04:06Z")
Hi I'm using 7.4.0 docker version, and after i upgraded it to 7.5.1 i go an error: After i downgrade to 7.4.0 it works, what is a reason of this error ? --- ## [JSON Parsing Error](https://discuss.elastic.co/t/json-parsing-error/212772)
**Author:** [@mancharagopan](https://discuss.elastic.co/u/mancharagopan)\ **Replies:** 3\ **Last updated:** [December 23, 2019, 11:41am UTC](https://discuss.elastic.co/t/json-parsing-error/212772 "2019-12-23T11:41:03Z")
I am reeving json parse error when i am using json filter to parse my data. my logstash config, filter { mutate { gsub =\> \[ "message", "^b\_rooms\_available\_and\_soldout...", "\[", "message", "\],$", "\]", … --- ## [Parsing message data into separate events](https://discuss.elastic.co/t/parsing-message-data-into-separate-events/212763)
**Author:** [@mancharagopan](https://discuss.elastic.co/u/mancharagopan)\ **Replies:** 1\ **Last updated:** [December 23, 2019, 11:39am UTC](https://discuss.elastic.co/t/parsing-message-data-into-separate-events/212763 "2019-12-23T11:39:18Z")
I have a sample event with the message containing the value in following format, b\_rooms\_available\_and\_soldout: \[{"b\_id":50699001,"b\_roomtype\_id":9,"b\_has\_room\_inventory":1,"b\_name":"Deluxe Double Room","b\_blocks":\[{"… --- ## [I want to replace the value of a field if its value is a certain number](https://discuss.elastic.co/t/i-want-to-replace-the-value-of-a-field-if-its-value-is-a-certain-number/212835)
**Author:** [@Jamcast](https://discuss.elastic.co/u/Jamcast)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 11:02am UTC](https://discuss.elastic.co/t/i-want-to-replace-the-value-of-a-field-if-its-value-is-a-certain-number/212835 "2019-12-23T11:02:49Z")
I want to replace the value of a field if its value is a certain number and I'm not sure what I'm doing wrong. Here is the input: "price" =\> 811950, "last\_price\_update" =\> "2019-12-22T11:30:23Z", … --- ## [Why is my document result embedded in "doc" field when importing via Logstash to Elasticsearch?](https://discuss.elastic.co/t/why-is-my-document-result-embedded-in-doc-field-when-importing-via-logstash-to-elasticsearch/212827)
**Author:** [@michalz](https://discuss.elastic.co/u/michalz)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 9:59am UTC](https://discuss.elastic.co/t/why-is-my-document-result-embedded-in-doc-field-when-importing-via-logstash-to-elasticsearch/212827 "2019-12-23T09:59:29Z")
Hi, I created a pipeline in logstash, and and an index in elasticsearch. I am importing data from couch db. Yet the documents I retrieve are wrapped in an additional field "doc" (within "\_source" that I don't want): "hi… --- ## [Multiple hosts in ES output - one of them down](https://discuss.elastic.co/t/multiple-hosts-in-es-output-one-of-them-down/212826)
**Author:** [@elastic22](https://discuss.elastic.co/u/elastic22)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 9:34am UTC](https://discuss.elastic.co/t/multiple-hosts-in-es-output-one-of-them-down/212826 "2019-12-23T09:34:03Z")
Hi I have a 5 node cluster, two of which are used for "hot" data. In my Logstash configs in the Elasticsearch output, I define both hosts: elasticsearch { hosts =\> \[ "https://host1:9200", "https://host2:9200" \] } I… --- ## [Multiple Root Element in single line](https://discuss.elastic.co/t/multiple-root-element-in-single-line/212809)
**Author:** [@mancharagopan](https://discuss.elastic.co/u/mancharagopan)\ **Replies:** 4\ **Last updated:** [December 23, 2019, 8:42am UTC](https://discuss.elastic.co/t/multiple-root-element-in-single-line/212809 "2019-12-23T08:42:33Z")
I need help with splitting multiple root element in single line. My data format like this, b\_rooms\_available\_and\_soldout:\[{"b\_id":123, "b\_blocks":\[{...}\]},{"b\_id":456, "b\_blocks":\[{...}\]}\] My logstash config is, … --- ## [Logstash config file make db busy](https://discuss.elastic.co/t/logstash-config-file-make-db-busy/212821)
**Author:** [@Sagar\_Mandal](https://discuss.elastic.co/u/Sagar_Mandal)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 8:27am UTC](https://discuss.elastic.co/t/logstash-config-file-make-db-busy/212821 "2019-12-23T08:27:16Z")
Hi Team, I have 50+ queries in a logstash config file which fetches data from a db and make individual index for every query. when i do this one of my queries make db busy by which i don't get data in other indices. so… --- ## [Cannot reference a field inside string](https://discuss.elastic.co/t/cannot-reference-a-field-inside-string/212276)
**Author:** [@BerkayOzturk](https://discuss.elastic.co/u/BerkayOzturk)\ **Replies:** 5\ **Last updated:** [December 23, 2019, 7:39am UTC](https://discuss.elastic.co/t/cannot-reference-a-field-inside-string/212276 "2019-12-23T07:39:41Z")
Hello, I'm trying to reference a field from a string inside the output section of my configuration. Instead of the field value, what I get is the literal representation of the string; in this case: '%{myfield}'. Here is… --- ## [Grok issues](https://discuss.elastic.co/t/grok-issues/212813)
**Author:** [@tortillla](https://discuss.elastic.co/u/tortillla)\ **Replies:** 5\ **Last updated:** [December 23, 2019, 8:09am UTC](https://discuss.elastic.co/t/grok-issues/212813 "2019-12-23T08:09:26Z")
I am trying to extract a substring from a file's path. The path is: "/home/kate/logs/phone/messages" I need to get the "phone" part. In my logstash conf file I am trying to use grok to extract this substring. I checke… --- ## [Disabling retries for Elasticsearch output plugin](https://discuss.elastic.co/t/disabling-retries-for-elasticsearch-output-plugin/212798)
**Author:** [@tonyc](https://discuss.elastic.co/u/tonyc)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 2:16am UTC](https://discuss.elastic.co/t/disabling-retries-for-elasticsearch-output-plugin/212798 "2019-12-23T02:16:13Z")
Hi all, Does anyone know if it's possible to configure an Elasticsearch Logstash output to completely ignore failures? I'm setting up a new Elasticsearch cluster and I want my existing Logstash pipeline to output to bo… --- ## [Translate CSV with varios fields](https://discuss.elastic.co/t/translate-csv-with-varios-fields/212771)
**Author:** [@carmezsa](https://discuss.elastic.co/u/carmezsa)\ **Replies:** 0\ **Last updated:** [December 22, 2019, 10:22am UTC](https://discuss.elastic.co/t/translate-csv-with-varios-fields/212771 "2019-12-22T10:22:31Z")
Hi, I am using translate filter to to compara fields from a CSV. CSV have two fields, but now I need a 3 fields. Actual CSV 148.244.114.49,malicious\_IP New CSV 148.244.114.49,malicious, botnet 148.244.113.9,mali… --- ## [How to remove duplicate logs in logstash?](https://discuss.elastic.co/t/how-to-remove-duplicate-logs-in-logstash/212767)
**Author:** [@mord](https://discuss.elastic.co/u/mord)\ **Replies:** 0\ **Last updated:** [December 22, 2019, 7:41am UTC](https://discuss.elastic.co/t/how-to-remove-duplicate-logs-in-logstash/212767 "2019-12-22T07:41:08Z")
How to remove duplicate logs in logstash ? input { beats { port =\> 5044 tags =\> "%{\[fields\]\[log\_type\]}" } } } filter { fingerprint { source =\> "message" target =\> "\[@metadata\]\[fingerprint\]" me… --- ## [How to maintain constant event flow rate in a Filebeat -\> Logstash -\> HTTP POST pipeline?](https://discuss.elastic.co/t/how-to-maintain-constant-event-flow-rate-in-a-filebeat-logstash-http-post-pipeline/212761)
**Author:** [@Nishant\_Das\_Patnaik](https://discuss.elastic.co/u/Nishant_Das_Patnaik)\ **Replies:** 0\ **Last updated:** [December 22, 2019, 4:07am UTC](https://discuss.elastic.co/t/how-to-maintain-constant-event-flow-rate-in-a-filebeat-logstash-http-post-pipeline/212761 "2019-12-22T04:07:52Z")
I'm relatively new to the Elastic ecosystem, so please excuse my ignorance. I have a use-case to use Filebeat to read application log files on VM1 then forward them to Logstash on VM2 and finally Logstash on VM2 forward… --- ## [If in filter drops event](https://discuss.elastic.co/t/if-in-filter-drops-event/212759)
**Author:** [@indikaM](https://discuss.elastic.co/u/indikaM)\ **Replies:** 1\ **Last updated:** [December 22, 2019, 3:38am UTC](https://discuss.elastic.co/t/if-in-filter-drops-event/212759 "2019-12-22T03:38:40Z")
Hey all, I am using CSV filter plugin and I want to check a field value and if exists I want to add a new field. However when I add an "if' condition , the relevant event is missing in the output log event \<14\>Dec 20 … --- ## [Remove first line csv logstash](https://discuss.elastic.co/t/remove-first-line-csv-logstash/212749)
**Author:** [@frangolzmil](https://discuss.elastic.co/u/frangolzmil)\ **Replies:** 1\ **Last updated:** [December 21, 2019, 7:19pm UTC](https://discuss.elastic.co/t/remove-first-line-csv-logstash/212749 "2019-12-21T19:19:11Z")
Hi dudes! I just wanna remove the header (first line) from my csv files. The first line is containing the column names. After parsing csv file, the visualization on Kibana shows the first line parsed as the column name … --- ## [Logstash run once from command line](https://discuss.elastic.co/t/logstash-run-once-from-command-line/212744)
**Author:** [@matandobr](https://discuss.elastic.co/u/matandobr)\ **Replies:** 1\ **Last updated:** [December 21, 2019, 7:16pm UTC](https://discuss.elastic.co/t/logstash-run-once-from-command-line/212744 "2019-12-21T19:16:57Z")
Hey guys, I want to write a simple application that would help me debug my Logstash configurations. I find it pretty annoying to modify configuration files in my server and restarting the Logstash service each time I w… --- ## [Unix time to timestamp error: "Wrong argument type Integer, expected LogStash::Timestamp"](https://discuss.elastic.co/t/unix-time-to-timestamp-error-wrong-argument-type-integer-expected-logstash-timestamp/212722)
**Author:** [@wconnell](https://discuss.elastic.co/u/wconnell)\ **Replies:** 2\ **Last updated:** [December 21, 2019, 5:12pm UTC](https://discuss.elastic.co/t/unix-time-to-timestamp-error-wrong-argument-type-integer-expected-logstash-timestamp/212722 "2019-12-21T17:12:43Z")
I'm reading logs into logstash from kafka that all have a field,ts. This field stores the unix time in seconds (for example 1576275878). I'm using the date plugin in logstash to convert it as follows to be stored in the @… --- ## [Nested grok](https://discuss.elastic.co/t/nested-grok/212670)
**Author:** [@Devopsio](https://discuss.elastic.co/u/Devopsio)\ **Replies:** 4\ **Last updated:** [December 21, 2019, 2:01pm UTC](https://discuss.elastic.co/t/nested-grok/212670 "2019-12-21T14:01:02Z")
Hi. I'm trying to make a 'dynamic' parsing of a URL by this grok pipeline. if \[container\]\[image\]\[name\] =~ /nginx/ { grok{ match =\> { "message" =\> \["%{IPORHOST:\[nginx\]\[access\]\[remote\_ip\]} - %{DATA:\[nginx\]\[access\] \[us… --- ## [Logstash - Parse formatted message with Ruby Code](https://discuss.elastic.co/t/logstash-parse-formatted-message-with-ruby-code/212469)
**Author:** [@GitsBdr](https://discuss.elastic.co/u/GitsBdr)\ **Replies:** 4\ **Last updated:** [December 21, 2019, 7:27am UTC](https://discuss.elastic.co/t/logstash-parse-formatted-message-with-ruby-code/212469 "2019-12-21T07:27:35Z")
Hi, Thanks in advance for your time! I have as input a line like the one below : 2019-12-18 08:28:36.810082121 string \[number\] logLevel string: tag1=val1|tag2=val2|tag3=val3|tag4=val4|tag5=val5|tag6=val6| And I'd li… --- ## [Ingesting a web page](https://discuss.elastic.co/t/ingesting-a-web-page/212721)
**Author:** [@mancharagopan](https://discuss.elastic.co/u/mancharagopan)\ **Replies:** 0\ **Last updated:** [December 21, 2019, 1:45am UTC](https://discuss.elastic.co/t/ingesting-a-web-page/212721 "2019-12-21T01:45:25Z")
Is there a way to access a URL and ingest the response of a web page in logstash? I need to analyze the rating of a company from some web sites that doesn’t have API. --- ## [Streaming Binary files’s decoded data to elastic. the binary files is location is a dynamic library. library need to be continuously checked for new files](https://discuss.elastic.co/t/streaming-binary-files-s-decoded-data-to-elastic-the-binary-files-is-location-is-a-dynamic-library-library-need-to-be-continuously-checked-for-new-files/212714)
**Author:** [@liron\_gofberg](https://discuss.elastic.co/u/liron_gofberg)\ **Replies:** 0\ **Last updated:** [December 20, 2019, 10:36pm UTC](https://discuss.elastic.co/t/streaming-binary-files-s-decoded-data-to-elastic-the-binary-files-is-location-is-a-dynamic-library-library-need-to-be-continuously-checked-for-new-files/212714 "2019-12-20T22:36:06Z")
I am wondering what tools I can use for streaming decoded data from binary files to elastic the the fastest way. Here is what I am trying to do. files are located at library . Those files are binary encoded files to s… --- ## [Viewing dead letter queue](https://discuss.elastic.co/t/viewing-dead-letter-queue/212710)
**Author:** [@Richard\_Neely](https://discuss.elastic.co/u/Richard_Neely)\ **Replies:** 0\ **Last updated:** [December 20, 2019, 9:00pm UTC](https://discuss.elastic.co/t/viewing-dead-letter-queue/212710 "2019-12-20T21:00:26Z")
Is there a good way to tail dead letter queue? It's a binary file so just using tail -f doesn't really work. I'm trying to track down some missing messages in a high volume system. I suspect they are hitting dlq but hard… --- ## [Mutate lowercase filter not been applied](https://discuss.elastic.co/t/mutate-lowercase-filter-not-been-applied/212693)
**Author:** [@francescouk](https://discuss.elastic.co/u/francescouk)\ **Replies:** 2\ **Last updated:** [December 20, 2019, 7:00pm UTC](https://discuss.elastic.co/t/mutate-lowercase-filter-not-been-applied/212693 "2019-12-20T19:00:55Z")
Hello there, Im trying to use the lowercase filter but it seens that is not working. I want to apply the filter on "TargetDomainName" field so the results should be "contoso\_rt" instead of "CONTOSO\_RT" Can someone poin… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=427) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=429)