# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=431 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 432 --- ## [Geoip doen't create field correctly](https://discuss.elastic.co/t/geoip-doent-create-field-correctly/212012)
**Author:** [@franco.federico](https://discuss.elastic.co/u/franco.federico)\ **Replies:** 2\ **Last updated:** [December 16, 2019, 4:44pm UTC](https://discuss.elastic.co/t/geoip-doent-create-field-correctly/212012 "2019-12-16T16:44:10Z")
I create this filter filter { # Filter only CEF logs here if "syslog" in \[tags\]{ # Manipulate the message mutate { # Saved the original message into a temporary field add\_field =\> { "tmp\_mess… --- ## [Configuring logstash to run as a service-](https://discuss.elastic.co/t/configuring-logstash-to-run-as-a-service/212029)
**Author:** [@Neeraj\_Rajagopalan\_I](https://discuss.elastic.co/u/Neeraj_Rajagopalan_I)\ **Replies:** 0\ **Last updated:** [December 16, 2019, 3:45pm UTC](https://discuss.elastic.co/t/configuring-logstash-to-run-as-a-service/212029 "2019-12-16T15:45:46Z")
I am getting a file path issue for all the commands. It is related to some path which is incorrect. Do i need to correct some config file to resolve this issue. Any help would be appreciated. --- ## [Parsing nested JSON arrays](https://discuss.elastic.co/t/parsing-nested-json-arrays/211938)
**Author:** [@sera123k](https://discuss.elastic.co/u/sera123k)\ **Replies:** 2\ **Last updated:** [December 16, 2019, 3:41pm UTC](https://discuss.elastic.co/t/parsing-nested-json-arrays/211938 "2019-12-16T15:41:02Z")
Hello, I am trying to parse a message from Office 365 that contains a nested array with some values in each item. I'm not 100% sure how to tackle it, the array i'm working with looks like this: "targetResources" =\> \[ \[… --- ## [Gz log files deleted by logstash when it crashed](https://discuss.elastic.co/t/gz-log-files-deleted-by-logstash-when-it-crashed/211985)
**Author:** [@lzx5033](https://discuss.elastic.co/u/lzx5033)\ **Replies:** 2\ **Last updated:** [December 16, 2019, 3:30pm UTC](https://discuss.elastic.co/t/gz-log-files-deleted-by-logstash-when-it-crashed/211985 "2019-12-16T15:30:53Z")
logstash version: 6.4.1 I was sending gz logs to elasticsearch by logstash. The elasticsearch's disk was full but i leave it, so that the logstash reported many 403 errors. But some day after that, the logstash crashed… --- ## [Bug: KV filter does not handle quoted strings with spaces at the end of a line, when using windows line endings](https://discuss.elastic.co/t/bug-kv-filter-does-not-handle-quoted-strings-with-spaces-at-the-end-of-a-line-when-using-windows-line-endings/211916)
**Author:** [@ridago](https://discuss.elastic.co/u/ridago)\ **Replies:** 3\ **Last updated:** [December 16, 2019, 11:55am UTC](https://discuss.elastic.co/t/bug-kv-filter-does-not-handle-quoted-strings-with-spaces-at-the-end-of-a-line-when-using-windows-line-endings/211916 "2019-12-16T11:55:34Z")
I've come accross an issue with the kv-filter plugin where if I input a file with windows line endings and the last key value pair on the line is a string with spaces, the string is not read correctly, but is instead rea… --- ## [Aggregate filter plugin does not create nested data in Elasticsearch](https://discuss.elastic.co/t/aggregate-filter-plugin-does-not-create-nested-data-in-elasticsearch/211996)
**Author:** [@jtboing](https://discuss.elastic.co/u/jtboing)\ **Replies:** 0\ **Last updated:** [December 16, 2019, 11:37am UTC](https://discuss.elastic.co/t/aggregate-filter-plugin-does-not-create-nested-data-in-elasticsearch/211996 "2019-12-16T11:37:53Z")
Hello, everyone. I've been using Logstash to move data from a MySQL server to Elasticsearch. It's been great so far, but I have run into some trouble when using the aggregate filter. Here's the data I'm using: +----+---… --- ## [Persistently queue input events even when output fails](https://discuss.elastic.co/t/persistently-queue-input-events-even-when-output-fails/211979)
**Author:** [@rcz](https://discuss.elastic.co/u/rcz)\ **Replies:** 0\ **Last updated:** [December 16, 2019, 10:33am UTC](https://discuss.elastic.co/t/persistently-queue-input-events-even-when-output-fails/211979 "2019-12-16T10:33:57Z")
Hello! I have some Logstash questions. I will provide some background info in case the questions don't make sense. Some background: Logstash has been part of my design for a while, but I have identified what seems like … --- ## [Problem with parsing fields starts or ends with dot(.)](https://discuss.elastic.co/t/problem-with-parsing-fields-starts-or-ends-with-dot/211978)
**Author:** [@atahanceylan](https://discuss.elastic.co/u/atahanceylan)\ **Replies:** 0\ **Last updated:** [December 16, 2019, 10:29am UTC](https://discuss.elastic.co/t/problem-with-parsing-fields-starts-or-ends-with-dot/211978 "2019-12-16T10:29:31Z")
Hi, From nested json fields some fields starts with dot or ends with dot this causing problem on Elasticsearch "Object field starting or ending with a \[.\] makes object resolution ambiguous: \]" What is the best optimal … --- ## [How to incrementally(keep updated way) sync data from MSSQL to Elasticsearch by Logstash?](https://discuss.elastic.co/t/how-to-incrementally-keep-updated-way-sync-data-from-mssql-to-elasticsearch-by-logstash/211972)
**Author:** [@111203](https://discuss.elastic.co/u/111203)\ **Replies:** 0\ **Last updated:** [December 16, 2019, 9:50am UTC](https://discuss.elastic.co/t/how-to-incrementally-keep-updated-way-sync-data-from-mssql-to-elasticsearch-by-logstash/211972 "2019-12-16T09:50:41Z")
Is there any documents about this topics? thanks for reply --- ## [View the dashboard without giving access to kibana](https://discuss.elastic.co/t/view-the-dashboard-without-giving-access-to-kibana/211967)
**Author:** [@avihuzini](https://discuss.elastic.co/u/avihuzini)\ **Replies:** 0\ **Last updated:** [December 16, 2019, 9:26am UTC](https://discuss.elastic.co/t/view-the-dashboard-without-giving-access-to-kibana/211967 "2019-12-16T09:26:14Z")
Hi, Im working on moving all my logs to elasticsearch with filebeat and logstash, and i want to provide my customer an option to view those logs. the thing is i dont want the customer to have access to Kibana. so basic… --- ## [Help getting started with SQlite integration in Kibana](https://discuss.elastic.co/t/help-getting-started-with-sqlite-integration-in-kibana/209909)
**Author:** [@enordstrand](https://discuss.elastic.co/u/enordstrand)\ **Replies:** 5\ **Last updated:** [December 16, 2019, 6:43am UTC](https://discuss.elastic.co/t/help-getting-started-with-sqlite-integration-in-kibana/209909 "2019-12-16T06:43:23Z")
Hello I'm quite new on using elastic search and docker. I've used the following references: https://elk-docker.readthedocs.io/ https://www.elastic.co/guide/en/logstash/current/plugins-inputs-sqlite.html I have the f… --- ## [Logstash update nested data](https://discuss.elastic.co/t/logstash-update-nested-data/211622)
**Author:** [@Ram\_29](https://discuss.elastic.co/u/Ram_29)\ **Replies:** 2\ **Last updated:** [December 16, 2019, 5:20am UTC](https://discuss.elastic.co/t/logstash-update-nested-data/211622 "2019-12-16T05:20:10Z")
Hi I'm having a nested field as below "appdata": { "type":"nested", "include\_in\_parent":true, "properties": { "accessType": { "type": "text", "norms": false, "fields": { "keyword": { "type": "keyword", "ignor… --- ## [Not able to connect to Amazon ES service via logstash on amazon EC2 instance](https://discuss.elastic.co/t/not-able-to-connect-to-amazon-es-service-via-logstash-on-amazon-ec2-instance/211937)
**Author:** [@Neeraj\_Rajagopalan\_I](https://discuss.elastic.co/u/Neeraj_Rajagopalan_I)\ **Replies:** 0\ **Last updated:** [December 16, 2019, 3:34am UTC](https://discuss.elastic.co/t/not-able-to-connect-to-amazon-es-service-via-logstash-on-amazon-ec2-instance/211937 "2019-12-16T03:34:15Z")
Hi, I am trying to connect to amazon elastic service via logstash installed on EC2. I am able to make input and output requests in general but using elastic service i am not able to connect to the service. Access po… --- ## [Logstash JSON parsing error if property has specific name](https://discuss.elastic.co/t/logstash-json-parsing-error-if-property-has-specific-name/211932)
**Author:** [@AlexBa](https://discuss.elastic.co/u/AlexBa)\ **Replies:** 0\ **Last updated:** [December 16, 2019, 12:00am UTC](https://discuss.elastic.co/t/logstash-json-parsing-error-if-property-has-specific-name/211932 "2019-12-16T00:00:39Z")
Hey guys, I want to analyze the traffic of my application. I therefore log every HTTP request and push it to a RabbitMQ pipeline in JSON format. A message could look like the following: { "request\_method":"GET", … --- ## [Logstasth Filter and Plugins Elasticsearch to Search and Add New Tag](https://discuss.elastic.co/t/logstasth-filter-and-plugins-elasticsearch-to-search-and-add-new-tag/211931)
**Author:** [@carmezsa](https://discuss.elastic.co/u/carmezsa)\ **Replies:** 0\ **Last updated:** [December 15, 2019, 11:55pm UTC](https://discuss.elastic.co/t/logstasth-filter-and-plugins-elasticsearch-to-search-and-add-new-tag/211931 "2019-12-15T23:55:29Z")
I use Zeek Framework (BroIDS) to monitoring my lab network. I use filebeat to send data to ELK. The logs from Zeek are in json format and they are similar to that: {"ts":"2019-12-15T23:08:37.016852Z","uid":"CzKj6o2MOtx4… --- ## [Transport Error: Cannot get new connection from pool](https://discuss.elastic.co/t/transport-error-cannot-get-new-connection-from-pool/211930)
**Author:** [@gurodrigues](https://discuss.elastic.co/u/gurodrigues)\ **Replies:** 0\ **Last updated:** [December 15, 2019, 11:39pm UTC](https://discuss.elastic.co/t/transport-error-cannot-get-new-connection-from-pool/211930 "2019-12-15T23:39:48Z")
Hi folks, I am using Logstash 7.5.0 to reindex my index from ES 6.8.5 to ES 7.5.0. I also tried to use Logstash 6.8.5 to do the job, but i get the same error in both versions I saw some similar posts here, but anyone … --- ## [Logstash filter seems not working](https://discuss.elastic.co/t/logstash-filter-seems-not-working/211890)
**Author:** [@Pradana](https://discuss.elastic.co/u/Pradana)\ **Replies:** 1\ **Last updated:** [December 15, 2019, 9:13pm UTC](https://discuss.elastic.co/t/logstash-filter-seems-not-working/211890 "2019-12-15T21:13:51Z")
Hi, I hv below configuration with my logststash : input { beats { port =\> 5044 } } filter{ if \[fields\]\[sourcelog\]=="cdrlog" { grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:logdate:date} %{LOGLEVEL:debugty… --- ## [Failure to convert epoch number in scientific format to date](https://discuss.elastic.co/t/failure-to-convert-epoch-number-in-scientific-format-to-date/211871)
**Author:** [@jlevym](https://discuss.elastic.co/u/jlevym)\ **Replies:** 2\ **Last updated:** [December 15, 2019, 5:10pm UTC](https://discuss.elastic.co/t/failure-to-convert-epoch-number-in-scientific-format-to-date/211871 "2019-12-15T17:10:04Z")
my input json file (coming from mongoexport) is showing the time in scientific notation, for example: "lastUpdate":1.575882773525e+12 When I try and convert this to date format using "UNIX\_MX" I get the following l… --- ## [Twitter API do not show the "lang" field](https://discuss.elastic.co/t/twitter-api-do-not-show-the-lang-field/211879)
**Author:** [@orejasdetrapo](https://discuss.elastic.co/u/orejasdetrapo)\ **Replies:** 0\ **Last updated:** [December 14, 2019, 7:13pm UTC](https://discuss.elastic.co/t/twitter-api-do-not-show-the-lang-field/211879 "2019-12-14T19:13:49Z")
Hi, I make a test for get the language of the tweets and not find the 'lang' field. Its necessary the activation inside the API? Thanks! --- ## [Logstash file input not reparsing file](https://discuss.elastic.co/t/logstash-file-input-not-reparsing-file/211854)
**Author:** [@konan\_pustolov](https://discuss.elastic.co/u/konan_pustolov)\ **Replies:** 3\ **Last updated:** [December 14, 2019, 7:10pm UTC](https://discuss.elastic.co/t/logstash-file-input-not-reparsing-file/211854 "2019-12-14T19:10:54Z")
I have the following problem, I need logstash to reparse already parsed files: Scenario that doesn't work but should: upload file to watched folder logstash processes it, saves to elastic, removes it (file\_complet… --- ## [Unable to load data HELP!](https://discuss.elastic.co/t/unable-to-load-data-help/211849)
**Author:** [@Antonio\_Ponce](https://discuss.elastic.co/u/Antonio_Ponce)\ **Replies:** 4\ **Last updated:** [December 14, 2019, 3:38pm UTC](https://discuss.elastic.co/t/unable-to-load-data-help/211849 "2019-12-14T15:38:34Z")
hi guys, im attempting to run the following but received the error below not sure what is wrong, can someone shed some light? input { file { path =\> "c:\\users\\administrator\\downloads\\chicago-food-inspection\\food-insp… --- ## [Unable to load data from logstash to elasticsearch](https://discuss.elastic.co/t/unable-to-load-data-from-logstash-to-elasticsearch/211826)
**Author:** [@ronics](https://discuss.elastic.co/u/ronics)\ **Replies:** 3\ **Last updated:** [December 14, 2019, 11:56am UTC](https://discuss.elastic.co/t/unable-to-load-data-from-logstash-to-elasticsearch/211826 "2019-12-14T11:56:50Z")
I have recently activated xpack monitoring in elasticsearh, kibana and logstash. I tried to push 101MB csv file but I am getting error. My configuratio in logstash.yml is xpack.monitoring.enabled: true xpack.monitoring… --- ## [Delay calculation between two send/receive events](https://discuss.elastic.co/t/delay-calculation-between-two-send-receive-events/211842)
**Author:** [@Sreekanth3](https://discuss.elastic.co/u/Sreekanth3)\ **Replies:** 0\ **Last updated:** [December 13, 2019, 8:11pm UTC](https://discuss.elastic.co/t/delay-calculation-between-two-send-receive-events/211842 "2019-12-13T20:11:43Z")
The Delay on the server should be calculated on the actual server, if the duration between Receive EventID on Server A and Send EventID on Server b (the same server) is 24 minutes , it should shows it on that specific … --- ## [\_grokparsefailure as soon as space in field](https://discuss.elastic.co/t/grokparsefailure-as-soon-as-space-in-field/211711)
**Author:** [@cawoodm](https://discuss.elastic.co/u/cawoodm)\ **Replies:** 4\ **Last updated:** [December 13, 2019, 4:22pm UTC](https://discuss.elastic.co/t/grokparsefailure-as-soon-as-space-in-field/211711 "2019-12-13T16:22:20Z")
I am unable to match any field which has a space. Consider the log: "foo" "foo bar" "foobar" The middle line foo baralways results in \_grokparsefailure. Patterns I've tried to no avail: match =\> { "message" =\> "\\"(?\ **Author:** [@kaushik.vankayala](https://discuss.elastic.co/u/kaushik.vankayala)\ **Replies:** 1\ **Last updated:** [December 13, 2019, 2:11pm UTC](https://discuss.elastic.co/t/add-the-field-value-to-each-new-filed-after-json-filter-processing-in-logstash/211766 "2019-12-13T14:11:38Z") Hi There, I am trying to configure my logstash to add the parent field name to all the fileds of the child (to make it understand where the filed actually came from) Ex: I am having the below as the incoming data in a … --- ## [Logstash-filter-elapsed don\`t generate elapsed\_expired\_error](https://discuss.elastic.co/t/logstash-filter-elapsed-don-t-generate-elapsed-expired-error/211799)
**Author:** [@111229](https://discuss.elastic.co/u/111229)\ **Replies:** 0\ **Last updated:** [December 13, 2019, 2:10pm UTC](https://discuss.elastic.co/t/logstash-filter-elapsed-don-t-generate-elapsed-expired-error/211799 "2019-12-13T14:10:20Z")
Hi, all! Could you help me. I have configuration from logstash-filter-elapsed elapsed { start\_tag =\> "start" end\_tag =\> "stop" unique\_id\_field =\> "ibso\_id" timeout =\> 15 } but after… --- ## [Dreded date with timezone :)](https://discuss.elastic.co/t/dreded-date-with-timezone/211718)
**Author:** [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Replies:** 2\ **Last updated:** [December 13, 2019, 1:57pm UTC](https://discuss.elastic.co/t/dreded-date-with-timezone/211718 "2019-12-13T13:57:44Z")
so i m trying to parse XML and ingest . Here is my XML ReceivedTimestamp and DeliveredTimestamp are in CST (Chicago) \ 2019-12-12T02:10:032019-12-12T02:20:03 Here… --- ## [Reloading of Data](https://discuss.elastic.co/t/reloading-of-data/211127)
**Author:** [@dkraemer](https://discuss.elastic.co/u/dkraemer)\ **Replies:** 2\ **Last updated:** [December 13, 2019, 1:02pm UTC](https://discuss.elastic.co/t/reloading-of-data/211127 "2019-12-13T13:02:05Z")
Hello, currently my plan is to use logstash for gathering data via http\_poller. (Api Requests) For my understanding: Are you able to configure logstash so it will recall api's after a specific time? Or is logstash do… --- ## [Logstash conditonals issue](https://discuss.elastic.co/t/logstash-conditonals-issue/211785)
**Author:** [@kamandohl](https://discuss.elastic.co/u/kamandohl)\ **Replies:** 2\ **Last updated:** [December 13, 2019, 12:34pm UTC](https://discuss.elastic.co/t/logstash-conditonals-issue/211785 "2019-12-13T12:34:09Z")
Issue: Unable to get conditionals to work Description: I have 6 logs that are from IIS. The formats are different on 3 of the logs from IIS. The grok I am using works. But I am unable to get the conditionals to work. I … --- ## [Grok filter pattern not working](https://discuss.elastic.co/t/grok-filter-pattern-not-working/211780)
**Author:** [@BeMoore](https://discuss.elastic.co/u/BeMoore)\ **Replies:** 8\ **Last updated:** [December 13, 2019, 12:32pm UTC](https://discuss.elastic.co/t/grok-filter-pattern-not-working/211780 "2019-12-13T12:32:28Z")
I must be going mad today, i can't get this filter to work. I extracted a single log line from an azure activity log, so i parse it through grok debugger both in kibana and herokuapp and i can't get it to work. so i spl… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=430) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=432)