# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=641 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 642 --- ## [While using elastic search as the input, not able to fetch data using the same index in elastic seach plugin for filter](https://discuss.elastic.co/t/while-using-elastic-search-as-the-input-not-able-to-fetch-data-using-the-same-index-in-elastic-seach-plugin-for-filter/158702)
**Author:** [@AtulShukla\_281194](https://discuss.elastic.co/u/AtulShukla_281194)\ **Replies:** 2\ **Last updated:** [November 29, 2018, 9:44am UTC](https://discuss.elastic.co/t/while-using-elastic-search-as-the-input-not-able-to-fetch-data-using-the-same-index-in-elastic-seach-plugin-for-filter/158702 "2018-11-29T09:44:25Z")
Below is my code : input { elasticsearch { hosts =\> "xxxx" index =\> "abcd\*" query=\> ' { "query":{ "range":{ "@timestamp":{ "gte":"2018-11-29T11:54:01.600113354Z", "lt":"2018-11-29T11:57:01.600113354Z" } } } … --- ## [Unable to change timestamp and parse entry to date](https://discuss.elastic.co/t/unable-to-change-timestamp-and-parse-entry-to-date/158566)
**Author:** [@safaa](https://discuss.elastic.co/u/safaa)\ **Replies:** 2\ **Last updated:** [November 29, 2018, 9:37am UTC](https://discuss.elastic.co/t/unable-to-change-timestamp-and-parse-entry-to-date/158566 "2018-11-29T09:37:57Z")
Could anybody please help me changing the timestamp and parsing the StartTime entry to date . I consume that logstash ignore the date filter and keeps the default timestamp value. This is my config file: input { fil… --- ## [Inject line based log/text files](https://discuss.elastic.co/t/inject-line-based-log-text-files/158548)
**Author:** [@Luke\_Devon](https://discuss.elastic.co/u/Luke_Devon)\ **Replies:** 4\ **Last updated:** [November 29, 2018, 8:49am UTC](https://discuss.elastic.co/t/inject-line-based-log-text-files/158548 "2018-11-29T08:49:50Z")
Hi I have some logs files which are currently unable to send directly to logstash. Those logs are consists of multiple lines of data as data blocks. For example; CompanyName: XYZ Pvt Ltd Date & Time: 2018-09-28 00:03… --- ## [Trouble parsing apache log using custom grok pattern](https://discuss.elastic.co/t/trouble-parsing-apache-log-using-custom-grok-pattern/157833)
**Author:** [@Havoc45](https://discuss.elastic.co/u/Havoc45)\ **Replies:** 2\ **Last updated:** [November 29, 2018, 1:55am UTC](https://discuss.elastic.co/t/trouble-parsing-apache-log-using-custom-grok-pattern/157833 "2018-11-29T01:55:17Z")
Hai, I'm new to Elastic Stack and I'm an intern to a company. One of the managers give me this project and I need you guys help. The use-case here is they want to categorize and analyse old logs from a variety of devices… --- ## [exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, \\", ', } at line 18, column 1 (byte 554) after output{\\n elasticsearch{\\n",](https://discuss.elastic.co/t/exception-logstash-configurationerror-message-expected-one-of-at-line-18-column-1-byte-554-after-output-n-elasticsearch-n/158462)
**Author:** [@sandhyaks](https://discuss.elastic.co/u/sandhyaks)\ **Replies:** 1\ **Last updated:** [November 28, 2018, 11:33pm UTC](https://discuss.elastic.co/t/exception-logstash-configurationerror-message-expected-one-of-at-line-18-column-1-byte-554-after-output-n-elasticsearch-n/158462 "2018-11-28T23:33:42Z")
Am getting the below error message upon trying to upload csv using logstash. Below is my config file. Can someone pls help input{ file{ path =\> "/etc/data/dept\_subsystem.csv" start\_position =\> "beginning" sincedb\_pa… --- ## [Can grok or dissect pattern code lines be wrapped?](https://discuss.elastic.co/t/can-grok-or-dissect-pattern-code-lines-be-wrapped/158652)
**Author:** [@RussellBateman](https://discuss.elastic.co/u/RussellBateman)\ **Replies:** 0\ **Last updated:** [November 28, 2018, 9:23pm UTC](https://discuss.elastic.co/t/can-grok-or-dissect-pattern-code-lines-be-wrapped/158652 "2018-11-28T21:23:04Z")
I mean it would be nice to be able to wrap very long patterns across one or several more lines for easier viewing and maintenance. Can this be done? My own experimentation suggests it's not possible. --- ## [Log to network 150 host](https://discuss.elastic.co/t/log-to-network-150-host/158402)
**Author:** [@Martin\_Sgdz](https://discuss.elastic.co/u/Martin_Sgdz)\ **Replies:** 3\ **Last updated:** [November 28, 2018, 8:33pm UTC](https://discuss.elastic.co/t/log-to-network-150-host/158402 "2018-11-28T20:33:39Z")
Is it possible to configure fewer log stores in the application? Is it necessary to install agents on computers in the network? Can you specify the period for storing logs? How many logs of 150 computers and servers f… --- ## [Logstash Jdbc plugin - How to get reason for remote database connectivity issues populated in a variable?](https://discuss.elastic.co/t/logstash-jdbc-plugin-how-to-get-reason-for-remote-database-connectivity-issues-populated-in-a-variable/158641)
**Author:** [@amitsaxena0709](https://discuss.elastic.co/u/amitsaxena0709)\ **Replies:** 0\ **Last updated:** [November 28, 2018, 7:21pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-how-to-get-reason-for-remote-database-connectivity-issues-populated-in-a-variable/158641 "2018-11-28T19:21:22Z")
Hi all, I am using ELK stack 6.2.4 . For one of our requirements, I am using Logstash Jdbc plugin to fetch the records from the database and then send it to Elasticsearch. While I am not facing any major issues with th… --- ## [How to merge different objects inside the same event](https://discuss.elastic.co/t/how-to-merge-different-objects-inside-the-same-event/158389)
**Author:** [@josemaia](https://discuss.elastic.co/u/josemaia)\ **Replies:** 1\ **Last updated:** [November 28, 2018, 5:29pm UTC](https://discuss.elastic.co/t/how-to-merge-different-objects-inside-the-same-event/158389 "2018-11-28T17:29:06Z")
Hello! I have a fairly complex JSON object as an input, and I want to flatten it so it's easier to work with in Kibana. An example is below: { "sprints": \[ { "id": 17193, "sequence": 17193, "nam… --- ## [Timestamp problem created using dissect](https://discuss.elastic.co/t/timestamp-problem-created-using-dissect/157782)
**Author:** [@RussellBateman](https://discuss.elastic.co/u/RussellBateman)\ **Replies:** 20\ **Last updated:** [November 28, 2018, 4:50pm UTC](https://discuss.elastic.co/t/timestamp-problem-created-using-dissect/157782 "2018-11-28T16:50:16Z")
I have this filter which works very well except for mucking up the date in dissection. (What's in the ellipsis below, ..., is too long and everything is working anyway.) filter { dissect { "message" =\> "%{acme.d… --- ## [How to run pipeline in logstash elastic cloud](https://discuss.elastic.co/t/how-to-run-pipeline-in-logstash-elastic-cloud/158613)
**Author:** [@rguerrero](https://discuss.elastic.co/u/rguerrero)\ **Replies:** 0\ **Last updated:** [November 28, 2018, 3:56pm UTC](https://discuss.elastic.co/t/how-to-run-pipeline-in-logstash-elastic-cloud/158613 "2018-11-28T15:56:59Z")
In my instance of elastic cloud I have logstash and I have created a pipeline. I do not see that it is executed. How can I verify that the schedule works? --- ## [Logstash conf - geoIP not workin](https://discuss.elastic.co/t/logstash-conf-geoip-not-workin/158353)
**Author:** [@Palino1611](https://discuss.elastic.co/u/Palino1611)\ **Replies:** 4\ **Last updated:** [November 28, 2018, 3:52pm UTC](https://discuss.elastic.co/t/logstash-conf-geoip-not-workin/158353 "2018-11-28T15:52:42Z")
Hello guys, please could you help me to identify error in my logstash config? I would like to see in kibana special field for geoip. My logstash config looks like this: input { beats { port =\> 5044 host =\> "0.0.0.0… --- ## [Logstash get same data from database](https://discuss.elastic.co/t/logstash-get-same-data-from-database/158555)
**Author:** [@ali1](https://discuss.elastic.co/u/ali1)\ **Replies:** 9\ **Last updated:** [November 28, 2018, 3:35pm UTC](https://discuss.elastic.co/t/logstash-get-same-data-from-database/158555 "2018-11-28T15:35:54Z")
Hello , im using logstash to get data from my database , and all forks fine , but when i change the number of line in my table for exemple i delete some rows , i still find the same rows as document in my elasticsearch. … --- ## [Logstash сuts long message](https://discuss.elastic.co/t/logstash-uts-long-message/158607)
**Author:** [@pauldon2](https://discuss.elastic.co/u/pauldon2)\ **Replies:** 0\ **Last updated:** [November 28, 2018, 3:24pm UTC](https://discuss.elastic.co/t/logstash-uts-long-message/158607 "2018-11-28T15:24:41Z")
I have long message like this - {"message":"http://oraxe.uislab.com:8080/ords/plp/fcplp/GetParcelList","context":{"status\_code":200,"reason\_phrase":"OK","request\_options":{"json":{"f\_organization":"71D474D44471D009E0509… --- ## [Create response\_time field from message field](https://discuss.elastic.co/t/create-response-time-field-from-message-field/158606)
**Author:** [@sbampa](https://discuss.elastic.co/u/sbampa)\ **Replies:** 0\ **Last updated:** [November 28, 2018, 3:20pm UTC](https://discuss.elastic.co/t/create-response-time-field-from-message-field/158606 "2018-11-28T15:20:35Z")
Hi all, i need to create the field response\_time from a message log: 2018-11-28 16:10:38.361+01:00 EB00548 N:6:: HeavyTaskHandler 10.20ms elapsed How can extract the 10.20ms in a separate field in order to create a vi… --- ## [Mapper\_parsing\_exception string](https://discuss.elastic.co/t/mapper-parsing-exception-string/157739)
**Author:** [@elk2](https://discuss.elastic.co/u/elk2)\ **Replies:** 4\ **Last updated:** [November 28, 2018, 2:48pm UTC](https://discuss.elastic.co/t/mapper-parsing-exception-string/157739 "2018-11-28T14:48:03Z")
I use logstash 6.5.0 and elasticsearch 6.4.2 When I try to index some events this is the result: \[2018-11-21T16:27:55,156\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, … --- ## [Logstash to override meta information](https://discuss.elastic.co/t/logstash-to-override-meta-information/157078)
**Author:** [@Kathir\_J](https://discuss.elastic.co/u/Kathir_J)\ **Replies:** 2\ **Last updated:** [November 28, 2018, 1:41pm UTC](https://discuss.elastic.co/t/logstash-to-override-meta-information/157078 "2018-11-28T13:41:47Z")
I have a scenario where logs are created in one machine(say machine A) and sent to another machine(Say machine B) where I execute filebeat to push logs to elasticsearch). When I try to execute filebeat, it gets the host… --- ## [How to reject incoming logs easily?](https://discuss.elastic.co/t/how-to-reject-incoming-logs-easily/158529)
**Author:** [@pwny](https://discuss.elastic.co/u/pwny)\ **Replies:** 3\ **Last updated:** [November 28, 2018, 1:40pm UTC](https://discuss.elastic.co/t/how-to-reject-incoming-logs-easily/158529 "2018-11-28T13:40:27Z")
Hello, we have a small Elastic Stack setup. There are several sources that are frequently sending logs to Logstash but they are not needed anymore. However to some I don't have access to and the person responsible doe… --- ## [Logstash not reading csv file](https://discuss.elastic.co/t/logstash-not-reading-csv-file/158512)
**Author:** [@fillic2002](https://discuss.elastic.co/u/fillic2002)\ **Replies:** 5\ **Last updated:** [November 28, 2018, 12:57pm UTC](https://discuss.elastic.co/t/logstash-not-reading-csv-file/158512 "2018-11-28T12:57:55Z")
I am not able to see any error on running the pipeline but the data is not read either on stdout or elasticsearch. here is my pipeline. I am using 6.5.0 input{ file{ path=\>"C:\\TR\\d\\Professional\\ELK\\Data\\test.csv" sta… --- ## [Logstash Date Filter Inquiry](https://discuss.elastic.co/t/logstash-date-filter-inquiry/157971)
**Author:** [@peter.alfafara](https://discuss.elastic.co/u/peter.alfafara)\ **Replies:** 2\ **Last updated:** [November 28, 2018, 12:57pm UTC](https://discuss.elastic.co/t/logstash-date-filter-inquiry/157971 "2018-11-28T12:57:18Z")
Hi Experts, We are trying to convert a field to a timefield but we are unable to get the correct data out of the following date filter: date { match =\> \["\[data\]\[START\_TIME\]","yyyy-MM-dd HH:mm:ss"\] target =\> "\[data\]\[… --- ## [Logstash Grok pattern GREEDYDATA does not match](https://discuss.elastic.co/t/logstash-grok-pattern-greedydata-does-not-match/158535)
**Author:** [@rani](https://discuss.elastic.co/u/rani)\ **Replies:** 4\ **Last updated:** [November 28, 2018, 12:28pm UTC](https://discuss.elastic.co/t/logstash-grok-pattern-greedydata-does-not-match/158535 "2018-11-28T12:28:05Z")
Hello there I'm currently building a pipeline with different grok patterns. For the most part it's properly working already but I'm having issues with some Jira logs: Log Message: 2018-11-21 12:50:23,950 http-nio-8080… --- ## [LOGSTASH AND PIPELINE](https://discuss.elastic.co/t/logstash-and-pipeline/158544)
**Author:** [@shruti51](https://discuss.elastic.co/u/shruti51)\ **Replies:** 1\ **Last updated:** [November 28, 2018, 11:53am UTC](https://discuss.elastic.co/t/logstash-and-pipeline/158544 "2018-11-28T11:53:23Z")
Hi, I am using distributor approch for two usecases.My config is: config/pipelines.yml pipeline.id: beats2 config.string: | input { file { path =\> "/opt/webmis/ELKFMDOFMSDKLFJAVA/GA/guyubdjsanbticleGlobal/.txt" st… --- ## [Nuget package to collect HangfireLog in Logstash](https://discuss.elastic.co/t/nuget-package-to-collect-hangfirelog-in-logstash/157895)
**Author:** [@Vincenzo\_Buonagura](https://discuss.elastic.co/u/Vincenzo_Buonagura)\ **Replies:** 1\ **Last updated:** [November 27, 2018, 1:29pm UTC](https://discuss.elastic.co/t/nuget-package-to-collect-hangfirelog-in-logstash/157895 "2018-11-27T13:29:16Z")
Hi all, I just published a new package that will allow you to send logs from Hangfire directly to Logstash. The package is compatible with both .NET and .NET core applications and SQLServer/Redis Hangfire. Feel free… --- ## [Are dead letter queues automatically cleaned up yet?](https://discuss.elastic.co/t/are-dead-letter-queues-automatically-cleaned-up-yet/158472)
**Author:** [@Sarfaraz\_Ahmad](https://discuss.elastic.co/u/Sarfaraz_Ahmad)\ **Replies:** 0\ **Last updated:** [November 28, 2018, 6:17am UTC](https://discuss.elastic.co/t/are-dead-letter-queues-automatically-cleaned-up-yet/158472 "2018-11-28T06:17:53Z")
This is more of a follow up from these Are the DLQs automatically cleaned up. I created a separate logstash pipeline to "re-feed" elasticsearch . That is good and all but the DLQ files are still there on disk. The … --- ## [LOgstash - java.lang.IllegalStateException: Lostash stopped processing because of an error: (LoadError) no such file to load -- bundler/vendor/molinillo/lib/molinillo](https://discuss.elastic.co/t/logstash-java-lang-illegalstateexception-lostash-stopped-processing-because-of-an-error-loaderror-no-such-file-to-load-bundler-vendor-molinillo-lib-molinillo/158459)
**Author:** [@Arunkumar\_Kathirvel](https://discuss.elastic.co/u/Arunkumar_Kathirvel)\ **Replies:** 0\ **Last updated:** [November 28, 2018, 5:33am UTC](https://discuss.elastic.co/t/logstash-java-lang-illegalstateexception-lostash-stopped-processing-because-of-an-error-loaderror-no-such-file-to-load-bundler-vendor-molinillo-lib-molinillo/158459 "2018-11-28T05:33:21Z")
Hi, I am using ELK for the first time and installed Elasticsearch-6.5.1 and Kibana-6.5.1on windows10 in "c:/elk". I have extracted logstash-6.5.1 into "c:/elk/logstash" and executed the command "bin/logstash" and got the… --- ## [Monitoring Multi-Stage Log Forwarding](https://discuss.elastic.co/t/monitoring-multi-stage-log-forwarding/158450)
**Author:** [@karnamonkster](https://discuss.elastic.co/u/karnamonkster)\ **Replies:** 0\ **Last updated:** [November 28, 2018, 3:41am UTC](https://discuss.elastic.co/t/monitoring-multi-stage-log-forwarding/158450 "2018-11-28T03:41:46Z")
Hi, Version details: ES & Kibana - 6.4.3 Logstash shipper - \>5.6.4 In our setup, there are multiple zones from where we ship logs and in between there are logstash instances who use Lumberjack to forward the same log… --- ## [Using json\_batch format in http output plugin](https://discuss.elastic.co/t/using-json-batch-format-in-http-output-plugin/158429)
**Author:** [@HendPro12](https://discuss.elastic.co/u/HendPro12)\ **Replies:** 0\ **Last updated:** [November 27, 2018, 10:30pm UTC](https://discuss.elastic.co/t/using-json-batch-format-in-http-output-plugin/158429 "2018-11-27T22:30:56Z")
I am using Filebeat to send events to Logstash. Filebeat's internal queue has been configured so that events will be combined into events before being sent to the logstash output. I've modified pipeline.batch.size and … --- ## [Container logs - rotating few times per day; logs not showing in kibana](https://discuss.elastic.co/t/container-logs-rotating-few-times-per-day-logs-not-showing-in-kibana/158417)
**Author:** [@Palino1611](https://discuss.elastic.co/u/Palino1611)\ **Replies:** 0\ **Last updated:** [November 27, 2018, 9:07pm UTC](https://discuss.elastic.co/t/container-logs-rotating-few-times-per-day-logs-not-showing-in-kibana/158417 "2018-11-27T21:07:16Z")
Hello guy, I am seting up ELK filebeat -\> logstash -\> elasticsearch -\> kibana I have 20 containers. 11 of which are showing in kibana. 9 are not. I have noticed that the 9 containers which I dont see in kibana - all … --- ## [Array: Extract the values and add the values](https://discuss.elastic.co/t/array-extract-the-values-and-add-the-values/158413)
**Author:** [@rcuevas](https://discuss.elastic.co/u/rcuevas)\ **Replies:** 0\ **Last updated:** [November 27, 2018, 8:32pm UTC](https://discuss.elastic.co/t/array-extract-the-values-and-add-the-values/158413 "2018-11-27T20:32:50Z")
Hi Community, I have this output: { "timeseriesId" =\> "com.dynatrace.builtin:servicemethod.responsetime", "dimensions" =\> \[ \[0\] "SERVICE\_METHOD" \], "@timestamp" =\> 2018-11-27T20:27:01.309Z, "types" =\> , "display… --- ## [Index for a single server](https://discuss.elastic.co/t/index-for-a-single-server/158378)
**Author:** [@garcia](https://discuss.elastic.co/u/garcia)\ **Replies:** 1\ **Last updated:** [November 27, 2018, 6:49pm UTC](https://discuss.elastic.co/t/index-for-a-single-server/158378 "2018-11-27T18:49:44Z")
I have several servers already configured on the elk server via filebeat, but for a separate server I need to configure my index in logstash 2018-11-27 10:49:12,148 - app.routing.router - INFO - name\_app - \[DEVELOPMENT\]… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=640) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=642)