# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=752 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 753 --- ## [Unable to parse multiple format logs](https://discuss.elastic.co/t/unable-to-parse-multiple-format-logs/133088)
**Author:** [@Saketh\_Chandra\_Kolis](https://discuss.elastic.co/u/Saketh_Chandra_Kolis)\ **Replies:** 3\ **Last updated:** [May 28, 2018, 5:43am UTC](https://discuss.elastic.co/t/unable-to-parse-multiple-format-logs/133088 "2018-05-28T05:43:04Z")
here is my full configuration: input { file { type =\> "apache" path =\> "C:/Users/Mahe/Desktop/intership/apache.log" } file { type =\> "csvf" path =\> "C:/data/cars.csv" } } filter { if \[type\] == "apache" { gro… --- ## [Logstash doesn't start after logstash.yml edit](https://discuss.elastic.co/t/logstash-doesnt-start-after-logstash-yml-edit/133394)
**Author:** [@Vladimir\_Fomin](https://discuss.elastic.co/u/Vladimir_Fomin)\ **Replies:** 0\ **Last updated:** [May 26, 2018, 1:40pm UTC](https://discuss.elastic.co/t/logstash-doesnt-start-after-logstash-yml-edit/133394 "2018-05-26T13:40:37Z")
I try to add netflow module to logstash.yml like this: modules: - name: netflow var.input.udp.port: 9996 But after save logstash.yml and service logstash start command logstash can't start. logs form /var/lo… --- ## [Logstash fails to start | Multiple pipelines](https://discuss.elastic.co/t/logstash-fails-to-start-multiple-pipelines/130250)
**Author:** [@Deepak\_Singh](https://discuss.elastic.co/u/Deepak_Singh)\ **Replies:** 8\ **Last updated:** [May 28, 2018, 4:43am UTC](https://discuss.elastic.co/t/logstash-fails-to-start-multiple-pipelines/130250 "2018-05-28T04:43:26Z")
Hi, I am trying to set up multiple pipelines, as described here : https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html I have set up the .conf and pipelines.yml file correctly, but when try to start… --- ## [Architecture for Logstash: AWS Monitoring](https://discuss.elastic.co/t/architecture-for-logstash-aws-monitoring/133257)
**Author:** [@rahulnama](https://discuss.elastic.co/u/rahulnama)\ **Replies:** 1\ **Last updated:** [May 28, 2018, 4:42am UTC](https://discuss.elastic.co/t/architecture-for-logstash-aws-monitoring/133257 "2018-05-28T04:42:05Z")
Hello Team, Currently, we are working on building the monitoring platform for one of the AWS services. As part of that, I'm using the logstash-cloudwatch plugin to pull the logs from cloudwatch. Logs in cloudwatch are … --- ## [Elasticsearch input plugin config "schedule " error](https://discuss.elastic.co/t/elasticsearch-input-plugin-config-schedule-error/133247)
**Author:** [@zldhqq12345](https://discuss.elastic.co/u/zldhqq12345)\ **Replies:** 2\ **Last updated:** [May 28, 2018, 3:37am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-config-schedule-error/133247 "2018-05-28T03:37:21Z")
elasticsearch version:5.6.3 logstash version:5.6.3 \[image\] \[2\]\[3\] --- ## [Filter GROK (Fortigate + ASA )](https://discuss.elastic.co/t/filter-grok-fortigate-asa/131947)
**Author:** [@asalma](https://discuss.elastic.co/u/asalma)\ **Replies:** 1\ **Last updated:** [May 27, 2018, 8:09pm UTC](https://discuss.elastic.co/t/filter-grok-fortigate-asa/131947 "2018-05-27T20:09:43Z")
Please, Iwant to make a config file of two filter one for cisco asa and the other for fortgate, I done this, but the configuration is not Ok, can someone correct me ? input { beats { port =\> "5044" … --- ## [Replacing identical documents in my index](https://discuss.elastic.co/t/replacing-identical-documents-in-my-index/133158)
**Author:** [@Ole](https://discuss.elastic.co/u/Ole)\ **Replies:** 1\ **Last updated:** [May 27, 2018, 7:58pm UTC](https://discuss.elastic.co/t/replacing-identical-documents-in-my-index/133158 "2018-05-27T19:58:41Z")
I've set up Logstash with the following pipeline.conf: input { # stdin {} # beats { # port =\> "5044" # } rss { url =\> "http://feeds.washingtonpost.com/rss/world" interval =\> 3600 tags =\> \["rss"\] } } # … --- ## [How convert timestamp in date](https://discuss.elastic.co/t/how-convert-timestamp-in-date/133383)
**Author:** [@itsgood2bebad](https://discuss.elastic.co/u/itsgood2bebad)\ **Replies:** 4\ **Last updated:** [May 27, 2018, 3:56pm UTC](https://discuss.elastic.co/t/how-convert-timestamp-in-date/133383 "2018-05-27T15:56:43Z")
Hello,Im new user for logstash and i need help . i want use 'timestamp' column in my csv but its not work . If you can help me . Thanks input { file{ path =\>"/Users/itsgood/Desktop/money/coins\_btc.csv" start\_posit… --- ## [Is there the documents about built-in variables on Logstash configuration?](https://discuss.elastic.co/t/is-there-the-documents-about-built-in-variables-on-logstash-configuration/133392)
**Author:** [@Takeo](https://discuss.elastic.co/u/Takeo)\ **Replies:** 1\ **Last updated:** [May 27, 2018, 12:49pm UTC](https://discuss.elastic.co/t/is-there-the-documents-about-built-in-variables-on-logstash-configuration/133392 "2018-05-27T12:49:33Z")
I need the IP address of source host that send data to Logstash input. I wonder if there is the documents about some built-in(?) variables that can be used input plug-in on Logstash configuration. Is there that somewhe… --- ## [Error: mongodb.jdbc.MongoDriver not loaded](https://discuss.elastic.co/t/error-mongodb-jdbc-mongodriver-not-loaded/133423)
**Author:** [@kartsh](https://discuss.elastic.co/u/kartsh)\ **Replies:** 0\ **Last updated:** [May 27, 2018, 12:08pm UTC](https://discuss.elastic.co/t/error-mongodb-jdbc-mongodriver-not-loaded/133423 "2018-05-27T12:08:37Z")
Am trying to load mogodb database to elastic search using logstash. I have written below logstash config file and executing it. While running the config file am getting the below error : Error: mongodb.jdbc.MongoDrive… --- ## [Logstash file input not working for large file](https://discuss.elastic.co/t/logstash-file-input-not-working-for-large-file/133414)
**Author:** [@muhammad](https://discuss.elastic.co/u/muhammad)\ **Replies:** 0\ **Last updated:** [May 27, 2018, 6:40am UTC](https://discuss.elastic.co/t/logstash-file-input-not-working-for-large-file/133414 "2018-05-27T06:40:44Z")
Hello, I am using elk (v5.6.9-1) . I have a file of size 1.3G containing debug logs. When I take a small snippet of file(eg. create a file with 10 lines of the original) , then it correctly reads the file and indexes it… --- ## [Issue installing the logstash-input-salesforce](https://discuss.elastic.co/t/issue-installing-the-logstash-input-salesforce/133401)
**Author:** [@praneel](https://discuss.elastic.co/u/praneel)\ **Replies:** 0\ **Last updated:** [May 26, 2018, 3:08pm UTC](https://discuss.elastic.co/t/issue-installing-the-logstash-input-salesforce/133401 "2018-05-26T15:08:52Z")
When I run this command to install the Salesforce plugin to extract data from Salesforce bin/logstash-plugin install logstash-input-salesforce I have this error Validating logstash-input-salesforce IOError: No route … --- ## [Pipeline Viewer UI confusion](https://discuss.elastic.co/t/pipeline-viewer-ui-confusion/133051)
**Author:** [@mladen](https://discuss.elastic.co/u/mladen)\ **Replies:** 3\ **Last updated:** [May 26, 2018, 1:58pm UTC](https://discuss.elastic.co/t/pipeline-viewer-ui-confusion/133051 "2018-05-26T13:58:16Z")
Hello, from this URL I managed to configure monitoring for my two logstash servers. I add first logstash server and check status in Kibana. In Pipeline Viewer I was manage to see my pipeline with pipeline.id: main. Aft… --- ## [Adding nested Json array as their own fields in Elasticsearch \[6.2\]](https://discuss.elastic.co/t/adding-nested-json-array-as-their-own-fields-in-elasticsearch-6-2/133395)
**Author:** [@Peter\_Boer](https://discuss.elastic.co/u/Peter_Boer)\ **Replies:** 0\ **Last updated:** [May 26, 2018, 1:50pm UTC](https://discuss.elastic.co/t/adding-nested-json-array-as-their-own-fields-in-elasticsearch-6-2/133395 "2018-05-26T13:50:32Z")
Hello, I am trying to detected services on hosts on my local network using Nmap and the version detection option (-sV). I then want to input this data via JSON into my database. When indexing the data I'm getting the… --- ## [Unable to Export Kibana Netflow dashboards](https://discuss.elastic.co/t/unable-to-export-kibana-netflow-dashboards/130106)
**Author:** [@Tabrez.Goilkar](https://discuss.elastic.co/u/Tabrez.Goilkar)\ **Replies:** 8\ **Last updated:** [May 26, 2018, 7:05am UTC](https://discuss.elastic.co/t/unable-to-export-kibana-netflow-dashboards/130106 "2018-05-26T07:05:19Z")
I have setup Logstash Kibana and elastic search on same windows server, i am running netflow module on logstash with below command but still dashboards are not imported in Kibana for netflow. i have tried multple options… --- ## [Help using Logstash XML Filter to Parse Offline Windows Event Logs](https://discuss.elastic.co/t/help-using-logstash-xml-filter-to-parse-offline-windows-event-logs/133360)
**Author:** [@f951298b](https://discuss.elastic.co/u/f951298b)\ **Replies:** 4\ **Last updated:** [May 26, 2018, 5:14am UTC](https://discuss.elastic.co/t/help-using-logstash-xml-filter-to-parse-offline-windows-event-logs/133360 "2018-05-26T05:14:20Z")
I am trying to ingest Windows Event (Application, Security, System) logs into Logstash that were pulled from an offline Windows system; ELK is running on a separate offline system. The logs were saved in XML format (sni… --- ## [Performance Impact of using logstash-filter-aggregate](https://discuss.elastic.co/t/performance-impact-of-using-logstash-filter-aggregate/133365)
**Author:** [@Juliet](https://discuss.elastic.co/u/Juliet)\ **Replies:** 1\ **Last updated:** [May 26, 2018, 12:42am UTC](https://discuss.elastic.co/t/performance-impact-of-using-logstash-filter-aggregate/133365 "2018-05-26T00:42:07Z")
Hello Frns After going through logstash aggregation filter documentation,I understood that if the start event and end event is defined during aggregation then log stash will hold the details related to start event until… --- ## [Same HTTP health check succeeds for ALB and failes for NLB](https://discuss.elastic.co/t/same-http-health-check-succeeds-for-alb-and-failes-for-nlb/131811)
**Author:** [@wes](https://discuss.elastic.co/u/wes)\ **Replies:** 4\ **Last updated:** [May 25, 2018, 9:55pm UTC](https://discuss.elastic.co/t/same-http-health-check-succeeds-for-alb-and-failes-for-nlb/131811 "2018-05-25T21:55:50Z")
Hi, I am trying to setup logstash behind a load balancer which seems to be something that has been mentioned by plenty of other people and seems trivial. I have no issues with the health check when I use an ALB where th… --- ## [How to write if condition inside of the logstash grok pattern?](https://discuss.elastic.co/t/how-to-write-if-condition-inside-of-the-logstash-grok-pattern/133330)
**Author:** [@Yusuf\_Karatoprak](https://discuss.elastic.co/u/Yusuf_Karatoprak)\ **Replies:** 3\ **Last updated:** [May 25, 2018, 4:37pm UTC](https://discuss.elastic.co/t/how-to-write-if-condition-inside-of-the-logstash-grok-pattern/133330 "2018-05-25T16:37:22Z")
My question is related to logstash grok pattern. I created below pattern that's working fine but the big problem is not string values. Sometimes; "Y" and "age" can be null so my grok pattern not create any log in elastic… --- ## [Calculate endtime with given timestamp and duration](https://discuss.elastic.co/t/calculate-endtime-with-given-timestamp-and-duration/133333)
**Author:** [@MarBel78](https://discuss.elastic.co/u/MarBel78)\ **Replies:** 1\ **Last updated:** [May 25, 2018, 3:27pm UTC](https://discuss.elastic.co/t/calculate-endtime-with-given-timestamp-and-duration/133333 "2018-05-25T15:27:54Z")
Hi, i've problems with ruby to calculate endtime from timestamp and given duration in seconds. Is there a easy way to add a calculated endtime field? Merci --- ## [Logstash target down](https://discuss.elastic.co/t/logstash-target-down/133157)
**Author:** [@Tuckson](https://discuss.elastic.co/u/Tuckson)\ **Replies:** 2\ **Last updated:** [May 25, 2018, 9:39am UTC](https://discuss.elastic.co/t/logstash-target-down/133157 "2018-05-25T09:39:00Z")
Hi, Isearched, but this question seems not answered in the last year, so I raise it here. In the current version of logstash, is all output still blocked if you have multiple targets (using filters and outputblocks to … --- ## [Regrok a groked string](https://discuss.elastic.co/t/regrok-a-groked-string/133316)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 1\ **Last updated:** [May 25, 2018, 2:22pm UTC](https://discuss.elastic.co/t/regrok-a-groked-string/133316 "2018-05-25T14:22:37Z")
Hello everybody, I use Logstash and groke to parse this line : 25/05/2018 11:55:28.630 (TACHE) 27/02/2018/D T XX\_CACXXX\_X\_TN\_CAC001MT\_PORTEFEUILLE(945)/SAU\_XXXXXX\_X\_LL\_OPE001\_EFLUID TER STATUS : TN Terminaison normale … --- ## [Multipipelines or one pipeline with lot of filters - Performance](https://discuss.elastic.co/t/multipipelines-or-one-pipeline-with-lot-of-filters-performance/132966)
**Author:** [@gourmax](https://discuss.elastic.co/u/gourmax)\ **Replies:** 1\ **Last updated:** [May 25, 2018, 1:52pm UTC](https://discuss.elastic.co/t/multipipelines-or-one-pipeline-with-lot-of-filters-performance/132966 "2018-05-25T13:52:06Z")
I have one logstash server where I listening for some tweets. Let's say that I have a "global" twitter index. Twitter input \> filter -\> output: elasticsearch (twitter-%{YYYY.MM.dd}) I want know to add new input twit… --- ## [Send data to 2 elasticsearch indexes](https://discuss.elastic.co/t/send-data-to-2-elasticsearch-indexes/133324)
**Author:** [@sana](https://discuss.elastic.co/u/sana)\ **Replies:** 1\ **Last updated:** [May 25, 2018, 1:40pm UTC](https://discuss.elastic.co/t/send-data-to-2-elasticsearch-indexes/133324 "2018-05-25T13:40:27Z")
i'm using logstash to send data from 2 different input:files and elastic search: input { elasticsearch{ hosts =\> "http://server\_name:9200" type =\> "host1" } file{ path =\> "/home/sana/pfe/docker/logs/\*\*/\*.log" typ… --- ## [Grok parsing dynamic log-lines](https://discuss.elastic.co/t/grok-parsing-dynamic-log-lines/132997)
**Author:** [@Dante91](https://discuss.elastic.co/u/Dante91)\ **Replies:** 7\ **Last updated:** [May 25, 2018, 11:17am UTC](https://discuss.elastic.co/t/grok-parsing-dynamic-log-lines/132997 "2018-05-25T11:17:16Z")
Hello together, I want to log different topics from my router, giving me dynamic logs like Apr/25/2018 07:19:37 ipsec,debug,packet 852310ad 5ebd8ba7 5ee02726 b0f1e739 6dafe48d 6c2f812f Apr/25/2018 07:19:37 ipsec,debu… --- ## [Logstash Plugin Xpack](https://discuss.elastic.co/t/logstash-plugin-xpack/133305)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 0\ **Last updated:** [May 25, 2018, 10:50am UTC](https://discuss.elastic.co/t/logstash-plugin-xpack/133305 "2018-05-25T10:50:52Z")
Hello all, We have installed the xpack in our elasticsearch, logstash and kibana as a trial. Then after that, we register a basic license. Everything works fine. We can monitor our elasticsearch nodes, kibana and logsta… --- ## [Query from Kibana and from Elastic search Filter](https://discuss.elastic.co/t/query-from-kibana-and-from-elastic-search-filter/133301)
**Author:** [@UdaySingh\_Ghadage](https://discuss.elastic.co/u/UdaySingh_Ghadage)\ **Replies:** 0\ **Last updated:** [May 25, 2018, 10:13am UTC](https://discuss.elastic.co/t/query-from-kibana-and-from-elastic-search-filter/133301 "2018-05-25T10:13:54Z")
Hi, in logstash version 6.1.0 there is option of aggregation\_fields to read aggregation response in elasticSearch filter plugin. filter { elasticsearch { aggregation\_fields =\> { "my\_agg\_name" =\> "my\_ls\_field" } } … --- ## [Mercifully killing Logstash instance](https://discuss.elastic.co/t/mercifully-killing-logstash-instance/133291)
**Author:** [@randomuser](https://discuss.elastic.co/u/randomuser)\ **Replies:** 1\ **Last updated:** [May 25, 2018, 9:27am UTC](https://discuss.elastic.co/t/mercifully-killing-logstash-instance/133291 "2018-05-25T09:27:36Z")
Hello, I just started using Logstash to migrate MySQL data to ES. Wrote my own configuration, but it has certain problems which I'm currently debugging. The thing is, every time I run LS with myconfig.conf and get an er… --- ## [Logstash doc\_as\_upsert =\> true not updating few fields sometimes](https://discuss.elastic.co/t/logstash-doc-as-upsert-true-not-updating-few-fields-sometimes/133293)
**Author:** [@shoaibjdev](https://discuss.elastic.co/u/shoaibjdev)\ **Replies:** 0\ **Last updated:** [May 25, 2018, 9:20am UTC](https://discuss.elastic.co/t/logstash-doc-as-upsert-true-not-updating-few-fields-sometimes/133293 "2018-05-25T09:20:04Z")
Hi, Have came across a strange issue where in if we are sending the doc\_as\_upsert =\> true for the data to be updated it doesn't update few fields at random. For example if I have 1000 records which after insert are upd… --- ## [Converting ISO8601 to date](https://discuss.elastic.co/t/converting-iso8601-to-date/133165)
**Author:** [@Boukhdhira](https://discuss.elastic.co/u/Boukhdhira)\ **Replies:** 2\ **Last updated:** [May 25, 2018, 8:47am UTC](https://discuss.elastic.co/t/converting-iso8601-to-date/133165 "2018-05-25T08:47:37Z")
Hi, i have a date converting problem. I have logs in this format: 2018-04-12T18:20:13+02:00 And I parse logs timestamp by grok as TIMESTAMP\_ISO8601. And it works perfectly: log parses and fields adds. But i'm tring to… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=751) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=753)