# Logstash doesn't deliver input to output

**URL:** <https://discuss.elastic.co/t/logstash-doesnt-deliver-input-to-output/367976>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [September 29, 2024, 7:29pm UTC](https://discuss.elastic.co/t/logstash-doesnt-deliver-input-to-output/367976 "2024-09-29T19:29:45Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![evgeniy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evgeniy/32/137942_2.png) [@evgeniy](https://discuss.elastic.co/u/evgeniy)\
**Post date:** [September 29, 2024, 7:29pm UTC](https://discuss.elastic.co/t/logstash-doesnt-deliver-input-to-output/367976/1 "2024-09-29T19:29:45Z")

</div>

I don't understand why logstash doesn't deliver input data to output.  
logstash is running in docker, alongside kibana and elastic.  
logstash.conf

```auto
input {
  file {
    mode => "read"
    codec => "json_lines"
    add_field => {
      "[@metadata][source]" => "modsec"
    }
    path => "/usr/share/logstash/ingest_data/modsec/*.log"
    file_completed_action => "log"
    file_completed_log_path => "/usr/share/logstash/ingest_data/logstash_completed.log"
  }
}
  
filter {
  if "[@metadata][source]" == "modsec" {
    date {
      id => "modsec_date_parse"
      match => ["[transaction][time_stamp]", "EEE MMM dd HH:mm:ss yyyy" ]
      timezone => "Asia/Jerusalem"
      remove_field => ["[transaction][time_stamp]" ]
      enable_metric => true
      target => "@timestamp"
    }
    mutate {
      lowercase => ["fieldname"]
    }
  }
}
  
output {
  elasticsearch {
    index => "logstash-%{+YYYY.MM.dd}"
    hosts=> "${ELASTIC_HOSTS}"
    user=> "${ELASTIC_USER}"
    password=> "${ELASTIC_PASSWORD}"
    cacert=> "certs/ca/ca.crt"
  }
  file {
    id => "logstash_output"
    enable_metric => true
    path => "/usr/share/logstash/ingest_data/logstash_output"
    codec => line {
      format => "custom format: %{message}"
    }
  }
}

```

logs:

```auto
[2024-09-29T19:16:56,073][DEBUG][org.logstash.execution.AbstractPipelineExt] Flow metric registered: `worker_utilization` in namespace `[:stats, :pipelines, :main, :plugins, :outputs, :logstash_output, :flow]`
[2024-09-29T19:16:56,073][DEBUG][logstash.javapipeline] Starting pipeline {:pipeline_id=>"main"}
[2024-09-29T19:16:56,094][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, "pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#<Thread:0x446b43ac /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2024-09-29T19:16:56,770][INFO][logstash.javapipeline][main] Pipeline Java execution initialization time {"seconds"=>0.67}
[2024-09-29T19:16:56,781][INFO][logstash.inputs.file][main] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"/usr/share/logstash/data/plugins/inputs/file/.sincedb_dc0fc26390d66ab73c5c25095c694398", :path=>["/usr/share/logstash/ingest_data/modsec/*.log"]}
[2024-09-29T19:16:56,785][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=>"main"}
[2024-09-29T19:16:56,789][DEBUG][logstash.javapipeline] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x446b43ac /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2024-09-29T19:16:56,797][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2024-09-29T19:16:56,800][INFO][filewatch.observingread][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] START, creating Discoverer, Watch with file and sincedb collections
[2024-09-29T19:16:56,807][DEBUG][filewatch.sincedbcollection][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] open: reading from /usr/share/logstash/data/plugins/inputs/file/.sincedb_dc0fc26390d66ab73c5c25095c694398
[2024-09-29T19:16:56,810][DEBUG][org.logstash.execution.PeriodicFlush][main] Pushing flush onto pipeline.
[2024-09-29T19:16:56,835][DEBUG][logstash.inputs.file][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] Received line {:path=>"/usr/share/logstash/ingest_data/modsec/audit.log", :text=>"{\"transaction\":{\"client_ip\":\"5.33.164.194\",\"time_stamp\":\"Sun Sep 29 19:11:18 2024\",\"server_id\":\"cfbe5a4abbf6f2effb9d89132e930fcb2e9c6ddb\",\"client_port\":60906,\"host_ip\":\"172.18.0.2\",\"host_port\":443,\"unique_id\":\"172763707882.783232\",\"request\":{\"method\":\"GET\",\"http_version\":2.0,\"uri\":\"/api/endpoints/2/docker/containers/395aabd7692a7cbfbc01bf4574c56939cdf884d5258b8f171c34ce8d0db2829e/logs?since=0&stderr=1&stdout=1&tail=100&timestamps=0\",\"headers\":{\"sec-fetch-site\":\"same-origin\",\"user-agent\":\"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0\",\"referer\":\"https://portainer.local/\",\"accept-encoding\":\"gzip, deflate, br, zstd\",\"cookie\":\"portainer_api_key=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MSwidXNlcm5hbWUiOiJqZWx1bSIsInJvbGUiOjEsInNjb3BlIjoiZGVmYXVsdCIsImZvcmNlQ2hhbmdlUGFzc3dvcmQiOmZhbHNlLCJleHAiOjE3Mjc2NjE5NDQsImlhdCI6MTcyNzYzMzE0NH0.NlRLu_WpmTnXXdAn6FVY2k6Q1sKbZqe9P56Qc8YjeTo; _gorilla_csrf=MTcyNzYzMzE0NHxJbVZNZWsxMloyWkpRa1ozV0hodlVtUkVja2RTUm01eFVYbHpTa05LZG5BdlpFWkRSbmhxWVhsSFdXczlJZ289fFZwtEwEGa5acoWBjbH9kkUAogaqxklaCnqtmsIT4Mqm\",\"te\":\"trailers\",\"accept-language\":\"en-US,en;q=0.5\",\"x-csrf-token\":\"i5NTCflIW6WKDpAg6lFytf4dwvoYEqpJPqUPf5l5HdbzL5+3/oBf+Z3IFH3k4OOjhI0IOFo0UDZK9Yq5r8sEXw==\",\"accept\":\"application/json, text/plain, */*\",\"sec-fetch-mode\":\"cors\",\"sec-fetch-dest\":\"empty\",\"host\":\"portainer.local\"}},\"response\":{\"body\":\"\",\"http_code\":200,\"headers\":{\"X-Csrf-Token\":\"sG9B3VkyYmcTibuxEjnP37mxgRSIv37a2q52sk7eb1zI041jXvpmOwRPP+wciF7JwyFL1sqZhKWu/vN0eGx21Q==\",\"Vary\":\"Cookie\",\"Docker-Experimental\":\"false\",\"Content-Type\":\"application/vnd.docker.multiplexed-stream\",\"Api-Version\":\"1.47\",\"X-Xss-Protection\":\"1; mode=block\",\"Ostype\":\"linux\",\"Connection\":\"close\",\"X-Content-Type-Options\":\"nosniff\",\"Date\":\"Sun, 29 Sep 2024 19:11:18 GMT\",\"Server\":\"nginx\",\"Strict-Transport-Security\":\"max-age=31536000\"}},\"producer\":{\"modsecurity\":\"ModSecurity v3.0.13 (Linux)\",\"connector\":\"ModSecurity-nginx v1.0.3\",\"secrules_engine\":\"DetectionOnly\",\"components\":[\"OWASP_CRS/4.7.0\\\"\"]},\"messages\":[{\"message\":\"Remote Command Execution: Direct Unix Command Execution\",\"details\":{\"match\":\"Matched \\\"Operator `Rx' with parameter `(?i)(?:^|b[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?u[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?s[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)? (4180 characters omitted)' against variable `ARGS_NAMES:stdout' (Value: `stdout' )\",\"reference\":\"o0,6v117,6o0,6v126,6\",\"ruleId\":\"932260\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf\",\"lineNumber\":\"519\",\"data\":\"Matched Data: stdout found within ARGS_NAMES:stdout: stdout\",\"severity\":\"2\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"application-multi\",\"language-shell\",\"platform-unix\",\"attack-rce\",\"paranoia-level/1\",\"OWASP_CRS\",\"capec/1000/152/248/88\",\"PCI/6.5.2\"],\"maturity\":\"0\",\"accuracy\":\"0\"}},{\"message\":\"Inbound Anomaly Score Exceeded (Total Score: 10)\",\"details\":{\"match\":\"Matched \\\"Operator `Ge' with parameter `5' against variable `TX:BLOCKING_INBOUND_ANOMALY_SCORE' (Value: `10' )\",\"reference\":\"\",\"ruleId\":\"949110\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf\",\"lineNumber\":\"222\",\"data\":\"\",\"severity\":\"0\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"anomaly-evaluation\",\"OWASP_CRS\"],\"maturity\":\"0\",\"accuracy\":\"0\"}}]}}"}
[2024-09-29T19:16:56,846][DEBUG][logstash.codecs.jsonlines][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] Start periodic runner
[2024-09-29T19:16:56,854][DEBUG][logstash.codecs.jsonlines][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] config LogStash::Codecs::JSONLines/@id = "json_lines_2085c679-0acf-4135-aad9-9778b871f31e"
[2024-09-29T19:16:56,855][DEBUG][logstash.codecs.jsonlines][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] config LogStash::Codecs::JSONLines/@enable_metric = true
[2024-09-29T19:16:56,856][DEBUG][logstash.codecs.jsonlines][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] config LogStash::Codecs::JSONLines/@charset = "UTF-8"
[2024-09-29T19:16:56,856][DEBUG][logstash.codecs.jsonlines][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] config LogStash::Codecs::JSONLines/@delimiter = "\n"
[2024-09-29T19:16:56,859][INFO][logstash.codecs.jsonlines][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[2024-09-29T19:16:56,861][DEBUG][logstash.inputs.file][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] Received line {:path=>"/usr/share/logstash/ingest_data/modsec/audit.log", :text=>"{\"transaction\":{\"client_ip\":\"5.33.164.194\",\"time_stamp\":\"Sun Sep 29 19:11:21 2024\",\"server_id\":\"cfbe5a4abbf6f2effb9d89132e930fcb2e9c6ddb\",\"client_port\":60906,\"host_ip\":\"172.18.0.2\",\"host_port\":443,\"unique_id\":\"172763708118.613418\",\"request\":{\"method\":\"GET\",\"http_version\":2.0,\"uri\":\"/api/endpoints/2/docker/containers/395aabd7692a7cbfbc01bf4574c56939cdf884d5258b8f171c34ce8d0db2829e/logs?since=0&stderr=1&stdout=1&tail=100&timestamps=0\",\"headers\":{\"sec-fetch-site\":\"same-origin\",\"user-agent\":\"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0\",\"referer\":\"https://portainer.local/\",\"accept-encoding\":\"gzip, deflate, br, zstd\",\"cookie\":\"portainer_api_key=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MSwidXNlcm5hbWUiOiJqZWx1bSIsInJvbGUiOjEsInNjb3BlIjoiZGVmYXVsdCIsImZvcmNlQ2hhbmdlUGFzc3dvcmQiOmZhbHNlLCJleHAiOjE3Mjc2NjE5NDQsImlhdCI6MTcyNzYzMzE0NH0.NlRLu_WpmTnXXdAn6FVY2k6Q1sKbZqe9P56Qc8YjeTo; _gorilla_csrf=MTcyNzYzMzE0NHxJbVZNZWsxMloyWkpRa1ozV0hodlVtUkVja2RTUm01eFVYbHpTa05LZG5BdlpFWkRSbmhxWVhsSFdXczlJZ289fFZwtEwEGa5acoWBjbH9kkUAogaqxklaCnqtmsIT4Mqm\",\"te\":\"trailers\",\"accept-language\":\"en-US,en;q=0.5\",\"x-csrf-token\":\"sG9B3VkyYmcTibuxEjnP37mxgRSIv37a2q52sk7eb1zI041jXvpmOwRPP+wciF7JwyFL1sqZhKWu/vN0eGx21Q==\",\"accept\":\"application/json, text/plain, */*\",\"sec-fetch-mode\":\"cors\",\"sec-fetch-dest\":\"empty\",\"host\":\"portainer.local\"}},\"response\":{\"body\":\"\",\"http_code\":200,\"headers\":{\"X-Csrf-Token\":\"xZasp5sYtlbFErP057XTiBSdH+Vd5x2NYRmN/eKIDaW9KmAZnNCyCtLUN6npBEKebg3VJx/B5/IVSQg71DoULA==\",\"Vary\":\"Cookie\",\"Docker-Experimental\":\"false\",\"Content-Type\":\"application/vnd.docker.multiplexed-stream\",\"Api-Version\":\"1.47\",\"X-Xss-Protection\":\"1; mode=block\",\"Ostype\":\"linux\",\"Connection\":\"close\",\"X-Content-Type-Options\":\"nosniff\",\"Date\":\"Sun, 29 Sep 2024 19:11:21 GMT\",\"Server\":\"nginx\",\"Strict-Transport-Security\":\"max-age=31536000\"}},\"producer\":{\"modsecurity\":\"ModSecurity v3.0.13 (Linux)\",\"connector\":\"ModSecurity-nginx v1.0.3\",\"secrules_engine\":\"DetectionOnly\",\"components\":[\"OWASP_CRS/4.7.0\\\"\"]},\"messages\":[{\"message\":\"Remote Command Execution: Direct Unix Command Execution\",\"details\":{\"match\":\"Matched \\\"Operator `Rx' with parameter `(?i)(?:^|b[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?u[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?s[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)? (4180 characters omitted)' against variable `ARGS_NAMES:stdout' (Value: `stdout' )\",\"reference\":\"o0,6v117,6o0,6v126,6\",\"ruleId\":\"932260\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf\",\"lineNumber\":\"519\",\"data\":\"Matched Data: stdout found within ARGS_NAMES:stdout: stdout\",\"severity\":\"2\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"application-multi\",\"language-shell\",\"platform-unix\",\"attack-rce\",\"paranoia-level/1\",\"OWASP_CRS\",\"capec/1000/152/248/88\",\"PCI/6.5.2\"],\"maturity\":\"0\",\"accuracy\":\"0\"}},{\"message\":\"Inbound Anomaly Score Exceeded (Total Score: 10)\",\"details\":{\"match\":\"Matched \\\"Operator `Ge' with parameter `5' against variable `TX:BLOCKING_INBOUND_ANOMALY_SCORE' (Value: `10' )\",\"reference\":\"\",\"ruleId\":\"949110\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf\",\"lineNumber\":\"222\",\"data\":\"\",\"severity\":\"0\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"anomaly-evaluation\",\"OWASP_CRS\"],\"maturity\":\"0\",\"accuracy\":\"0\"}}]}}"}
[2024-09-29T19:16:56,865][DEBUG][logstash.inputs.file][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] Received line {:path=>"/usr/share/logstash/ingest_data/modsec/audit.log", :text=>"{\"transaction\":{\"client_ip\":\"5.33.164.194\",\"time_stamp\":\"Sun Sep 29 19:11:48 2024\",\"server_id\":\"cfbe5a4abbf6f2effb9d89132e930fcb2e9c6ddb\",\"client_port\":60906,\"host_ip\":\"172.18.0.2\",\"host_port\":443,\"unique_id\":\"172763710894.080921\",\"request\":{\"method\":\"GET\",\"http_version\":2.0,\"uri\":\"/api/endpoints/2/docker/containers/395aabd7692a7cbfbc01bf4574c56939cdf884d5258b8f171c34ce8d0db2829e/logs?since=0&stderr=1&stdout=1&tail=100&timestamps=0\",\"headers\":{\"sec-fetch-site\":\"same-origin\",\"user-agent\":\"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0\",\"referer\":\"https://portainer.local/\",\"accept-encoding\":\"gzip, deflate, br, zstd\",\"cookie\":\"portainer_api_key=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MSwidXNlcm5hbWUiOiJqZWx1bSIsInJvbGUiOjEsInNjb3BlIjoiZGVmYXVsdCIsImZvcmNlQ2hhbmdlUGFzc3dvcmQiOmZhbHNlLCJleHAiOjE3Mjc2NjE5NDQsImlhdCI6MTcyNzYzMzE0NH0.NlRLu_WpmTnXXdAn6FVY2k6Q1sKbZqe9P56Qc8YjeTo; _gorilla_csrf=MTcyNzYzMzE0NHxJbVZNZWsxMloyWkpRa1ozV0hodlVtUkVja2RTUm01eFVYbHpTa05LZG5BdlpFWkRSbmhxWVhsSFdXczlJZ289fFZwtEwEGa5acoWBjbH9kkUAogaqxklaCnqtmsIT4Mqm\",\"te\":\"trailers\",\"accept-language\":\"en-US,en;q=0.5\",\"x-csrf-token\":\"PIXPTSDaPPSE3/3xZ/agGVlX2yIpQmvk+Y3Ke+aU4elEOQPzJxI4qJMZeaxpRzEPI8cR4GtkkZuN3U+90Cb4YA==\",\"accept\":\"application/json, text/plain, */*\",\"sec-fetch-mode\":\"cors\",\"sec-fetch-dest\":\"empty\",\"host\":\"portainer.local\"}},\"response\":{\"body\":\"\",\"http_code\":200,\"headers\":{\"X-Csrf-Token\":\"8P3MBQOONTpaWJfDhehmrTqlyWn1Yp9CFqMSDRSVCx6IQQC7BEYxZk2eE56LWfe7QDUDq7dEZT1i85fLIicSlw==\",\"Vary\":\"Cookie\",\"Docker-Experimental\":\"false\",\"Content-Type\":\"application/vnd.docker.multiplexed-stream\",\"Api-Version\":\"1.47\",\"X-Xss-Protection\":\"1; mode=block\",\"Ostype\":\"linux\",\"Connection\":\"close\",\"X-Content-Type-Options\":\"nosniff\",\"Date\":\"Sun, 29 Sep 2024 19:11:48 GMT\",\"Server\":\"nginx\",\"Strict-Transport-Security\":\"max-age=31536000\"}},\"producer\":{\"modsecurity\":\"ModSecurity v3.0.13 (Linux)\",\"connector\":\"ModSecurity-nginx v1.0.3\",\"secrules_engine\":\"DetectionOnly\",\"components\":[\"OWASP_CRS/4.7.0\\\"\"]},\"messages\":[{\"message\":\"Remote Command Execution: Direct Unix Command Execution\",\"details\":{\"match\":\"Matched \\\"Operator `Rx' with parameter `(?i)(?:^|b[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?u[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?s[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)? (4180 characters omitted)' against variable `ARGS_NAMES:stdout' (Value: `stdout' )\",\"reference\":\"o0,6v117,6o0,6v126,6\",\"ruleId\":\"932260\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf\",\"lineNumber\":\"519\",\"data\":\"Matched Data: stdout found within ARGS_NAMES:stdout: stdout\",\"severity\":\"2\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"application-multi\",\"language-shell\",\"platform-unix\",\"attack-rce\",\"paranoia-level/1\",\"OWASP_CRS\",\"capec/1000/152/248/88\",\"PCI/6.5.2\"],\"maturity\":\"0\",\"accuracy\":\"0\"}},{\"message\":\"Inbound Anomaly Score Exceeded (Total Score: 10)\",\"details\":{\"match\":\"Matched \\\"Operator `Ge' with parameter `5' against variable `TX:BLOCKING_INBOUND_ANOMALY_SCORE' (Value: `10' )\",\"reference\":\"\",\"ruleId\":\"949110\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf\",\"lineNumber\":\"222\",\"data\":\"\",\"severity\":\"0\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"anomaly-evaluation\",\"OWASP_CRS\"],\"maturity\":\"0\",\"accuracy\":\"0\"}}]}}"}
[2024-09-29T19:16:56,878][DEBUG][filewatch.sincedbcollection][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] writing sincedb (delta since last write = 1727637416)
[2024-09-29T19:16:56,884][DEBUG][logstash.inputs.file][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] handle_deletable_path {:path=>"/usr/share/logstash/ingest_data/modsec/audit.log"}
[2024-09-29T19:16:56,890][DEBUG][logstash.inputs.file][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] Received line {:path=>"/usr/share/logstash/ingest_data/modsec/audit2.log", :text=>"{\"transaction\":{\"client_ip\":\"5.33.164.194\",\"time_stamp\":\"Sun Sep 29 19:14:45 2024\",\"server_id\":\"cfbe5a4abbf6f2effb9d89132e930fcb2e9c6ddb\",\"client_port\":60906,\"host_ip\":\"172.18.0.2\",\"host_port\":443,\"unique_id\":\"172763728575.845058\",\"request\":{\"method\":\"GET\",\"http_version\":2.0,\"uri\":\"/api/endpoints/2/docker/containers/395aabd7692a7cbfbc01bf4574c56939cdf884d5258b8f171c34ce8d0db2829e/logs?since=0&stderr=1&stdout=1&tail=100&timestamps=0\",\"headers\":{\"sec-fetch-site\":\"same-origin\",\"user-agent\":\"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0\",\"referer\":\"https://portainer.local/\",\"accept-encoding\":\"gzip, deflate, br, zstd\",\"cookie\":\"portainer_api_key=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MSwidXNlcm5hbWUiOiJqZWx1bSIsInJvbGUiOjEsInNjb3BlIjoiZGVmYXVsdCIsImZvcmNlQ2hhbmdlUGFzc3dvcmQiOmZhbHNlLCJleHAiOjE3Mjc2NjE5NDQsImlhdCI6MTcyNzYzMzE0NH0.NlRLu_WpmTnXXdAn6FVY2k6Q1sKbZqe9P56Qc8YjeTo; _gorilla_csrf=MTcyNzYzMzE0NHxJbVZNZWsxMloyWkpRa1ozV0hodlVtUkVja2RTUm01eFVYbHpTa05LZG5BdlpFWkRSbmhxWVhsSFdXczlJZ289fFZwtEwEGa5acoWBjbH9kkUAogaqxklaCnqtmsIT4Mqm\",\"te\":\"trailers\",\"accept-language\":\"en-US,en;q=0.5\",\"x-csrf-token\":\"BAyh68tKnZ48Z2DI3FU/ddl6Wl8ng46wIGP2V5S32X58sG1VzIKZwiuh5JXS5K5jo+qQnWWldM9UM3ORogXA9w==\",\"accept\":\"application/json, text/plain, */*\",\"sec-fetch-mode\":\"cors\",\"sec-fetch-dest\":\"empty\",\"host\":\"portainer.local\"}},\"response\":{\"body\":\"\",\"http_code\":200,\"headers\":{\"X-Csrf-Token\":\"+rOr8Qq6m77PseI8cSSLvWiTkSH4JhUR1w5n4Kd4H3uCD2dPDXKf4th3ZmF/lRqrEgNb47oA726jXuImkcoG8g==\",\"Vary\":\"Cookie\",\"Docker-Experimental\":\"false\",\"Content-Type\":\"application/vnd.docker.multiplexed-stream\",\"Api-Version\":\"1.47\",\"X-Xss-Protection\":\"1; mode=block\",\"Ostype\":\"linux\",\"Connection\":\"close\",\"X-Content-Type-Options\":\"nosniff\",\"Date\":\"Sun, 29 Sep 2024 19:14:46 GMT\",\"Server\":\"nginx\",\"Strict-Transport-Security\":\"max-age=31536000\"}},\"producer\":{\"modsecurity\":\"ModSecurity v3.0.13 (Linux)\",\"connector\":\"ModSecurity-nginx v1.0.3\",\"secrules_engine\":\"DetectionOnly\",\"components\":[\"OWASP_CRS/4.7.0\\\"\"]},\"messages\":[{\"message\":\"Remote Command Execution: Direct Unix Command Execution\",\"details\":{\"match\":\"Matched \\\"Operator `Rx' with parameter `(?i)(?:^|b[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?u[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?s[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)? (4180 characters omitted)' against variable `ARGS_NAMES:stdout' (Value: `stdout' )\",\"reference\":\"o0,6v117,6o0,6v126,6\",\"ruleId\":\"932260\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf\",\"lineNumber\":\"519\",\"data\":\"Matched Data: stdout found within ARGS_NAMES:stdout: stdout\",\"severity\":\"2\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"application-multi\",\"language-shell\",\"platform-unix\",\"attack-rce\",\"paranoia-level/1\",\"OWASP_CRS\",\"capec/1000/152/248/88\",\"PCI/6.5.2\"],\"maturity\":\"0\",\"accuracy\":\"0\"}},{\"message\":\"Inbound Anomaly Score Exceeded (Total Score: 10)\",\"details\":{\"match\":\"Matched \\\"Operator `Ge' with parameter `5' against variable `TX:BLOCKING_INBOUND_ANOMALY_SCORE' (Value: `10' )\",\"reference\":\"\",\"ruleId\":\"949110\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf\",\"lineNumber\":\"222\",\"data\":\"\",\"severity\":\"0\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"anomaly-evaluation\",\"OWASP_CRS\"],\"maturity\":\"0\",\"accuracy\":\"0\"}}]}}"}
[2024-09-29T19:16:56,904][DEBUG][logstash.codecs.jsonlines][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] config LogStash::Codecs::JSONLines/@id = "json_lines_2085c679-0acf-4135-aad9-9778b871f31e"
[2024-09-29T19:16:56,906][DEBUG][logstash.codecs.jsonlines][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] config LogStash::Codecs::JSONLines/@enable_metric = true
[2024-09-29T19:16:56,907][DEBUG][logstash.codecs.jsonlines][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] config LogStash::Codecs::JSONLines/@charset = "UTF-8"
[2024-09-29T19:16:56,907][DEBUG][logstash.codecs.jsonlines][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] config LogStash::Codecs::JSONLines/@delimiter = "\n"
[2024-09-29T19:16:56,909][INFO][logstash.codecs.jsonlines][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[2024-09-29T19:16:56,910][DEBUG][logstash.inputs.file][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] Received line {:path=>"/usr/share/logstash/ingest_data/modsec/audit2.log", :text=>"{\"transaction\":{\"client_ip\":\"5.33.164.194\",\"time_stamp\":\"Sun Sep 29 19:14:48 2024\",\"server_id\":\"cfbe5a4abbf6f2effb9d89132e930fcb2e9c6ddb\",\"client_port\":60906,\"host_ip\":\"172.18.0.2\",\"host_port\":443,\"unique_id\":\"172763728814.485956\",\"request\":{\"method\":\"GET\",\"http_version\":2.0,\"uri\":\"/api/endpoints/2/docker/containers/395aabd7692a7cbfbc01bf4574c56939cdf884d5258b8f171c34ce8d0db2829e/logs?since=0&stderr=1&stdout=1&tail=100&timestamps=0\",\"headers\":{\"sec-fetch-site\":\"same-origin\",\"user-agent\":\"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0\",\"referer\":\"https://portainer.local/\",\"accept-encoding\":\"gzip, deflate, br, zstd\",\"cookie\":\"portainer_api_key=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MSwidXNlcm5hbWUiOiJqZWx1bSIsInJvbGUiOjEsInNjb3BlIjoiZGVmYXVsdCIsImZvcmNlQ2hhbmdlUGFzc3dvcmQiOmZhbHNlLCJleHAiOjE3Mjc2NjE5NDQsImlhdCI6MTcyNzYzMzE0NH0.NlRLu_WpmTnXXdAn6FVY2k6Q1sKbZqe9P56Qc8YjeTo; _gorilla_csrf=MTcyNzYzMzE0NHxJbVZNZWsxMloyWkpRa1ozV0hodlVtUkVja2RTUm01eFVYbHpTa05LZG5BdlpFWkRSbmhxWVhsSFdXczlJZ289fFZwtEwEGa5acoWBjbH9kkUAogaqxklaCnqtmsIT4Mqm\",\"te\":\"trailers\",\"accept-language\":\"en-US,en;q=0.5\",\"x-csrf-token\":\"+rOr8Qq6m77PseI8cSSLvWiTkSH4JhUR1w5n4Kd4H3uCD2dPDXKf4th3ZmF/lRqrEgNb47oA726jXuImkcoG8g==\",\"accept\":\"application/json, text/plain, */*\",\"sec-fetch-mode\":\"cors\",\"sec-fetch-dest\":\"empty\",\"host\":\"portainer.local\"}},\"response\":{\"body\":\"\",\"http_code\":200,\"headers\":{\"X-Csrf-Token\":\"bGkKC9WDDGs+Cxyt9PeWX+LOtSInWra0Qqg5gHJHNRoU1ca10ksINynNmPD6RgdJmF5/4GV8TMs2+LxGRPUskw==\",\"Vary\":\"Cookie\",\"Docker-Experimental\":\"false\",\"Content-Type\":\"application/vnd.docker.multiplexed-stream\",\"Api-Version\":\"1.47\",\"X-Xss-Protection\":\"1; mode=block\",\"Ostype\":\"linux\",\"Connection\":\"close\",\"X-Content-Type-Options\":\"nosniff\",\"Date\":\"Sun, 29 Sep 2024 19:14:49 GMT\",\"Server\":\"nginx\",\"Strict-Transport-Security\":\"max-age=31536000\"}},\"producer\":{\"modsecurity\":\"ModSecurity v3.0.13 (Linux)\",\"connector\":\"ModSecurity-nginx v1.0.3\",\"secrules_engine\":\"DetectionOnly\",\"components\":[\"OWASP_CRS/4.7.0\\\"\"]},\"messages\":[{\"message\":\"Remote Command Execution: Direct Unix Command Execution\",\"details\":{\"match\":\"Matched \\\"Operator `Rx' with parameter `(?i)(?:^|b[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?u[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?s[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)? (4180 characters omitted)' against variable `ARGS_NAMES:stdout' (Value: `stdout' )\",\"reference\":\"o0,6v117,6o0,6v126,6\",\"ruleId\":\"932260\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf\",\"lineNumber\":\"519\",\"data\":\"Matched Data: stdout found within ARGS_NAMES:stdout: stdout\",\"severity\":\"2\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"application-multi\",\"language-shell\",\"platform-unix\",\"attack-rce\",\"paranoia-level/1\",\"OWASP_CRS\",\"capec/1000/152/248/88\",\"PCI/6.5.2\"],\"maturity\":\"0\",\"accuracy\":\"0\"}},{\"message\":\"Inbound Anomaly Score Exceeded (Total Score: 10)\",\"details\":{\"match\":\"Matched \\\"Operator `Ge' with parameter `5' against variable `TX:BLOCKING_INBOUND_ANOMALY_SCORE' (Value: `10' )\",\"reference\":\"\",\"ruleId\":\"949110\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf\",\"lineNumber\":\"222\",\"data\":\"\",\"severity\":\"0\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"anomaly-evaluation\",\"OWASP_CRS\"],\"maturity\":\"0\",\"accuracy\":\"0\"}}]}}"}
[2024-09-29T19:16:56,919][DEBUG][logstash.inputs.file][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] Received line {:path=>"/usr/share/logstash/ingest_data/modsec/audit2.log", :text=>"{\"transaction\":{\"client_ip\":\"5.33.164.194\",\"time_stamp\":\"Sun Sep 29 19:14:57 2024\",\"server_id\":\"cfbe5a4abbf6f2effb9d89132e930fcb2e9c6ddb\",\"client_port\":60906,\"host_ip\":\"172.18.0.2\",\"host_port\":443,\"unique_id\":\"172763729749.516967\",\"request\":{\"method\":\"GET\",\"http_version\":2.0,\"uri\":\"/api/endpoints/2/docker/containers/395aabd7692a7cbfbc01bf4574c56939cdf884d5258b8f171c34ce8d0db2829e/logs?since=0&stderr=1&stdout=1&tail=100&timestamps=0\",\"headers\":{\"sec-fetch-site\":\"same-origin\",\"user-agent\":\"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0\",\"referer\":\"https://portainer.local/\",\"accept-encoding\":\"gzip, deflate, br, zstd\",\"cookie\":\"portainer_api_key=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MSwidXNlcm5hbWUiOiJqZWx1bSIsInJvbGUiOjEsInNjb3BlIjoiZGVmYXVsdCIsImZvcmNlQ2hhbmdlUGFzc3dvcmQiOmZhbHNlLCJleHAiOjE3Mjc2NjE5NDQsImlhdCI6MTcyNzYzMzE0NH0.NlRLu_WpmTnXXdAn6FVY2k6Q1sKbZqe9P56Qc8YjeTo; _gorilla_csrf=MTcyNzYzMzE0NHxJbVZNZWsxMloyWkpRa1ozV0hodlVtUkVja2RTUm01eFVYbHpTa05LZG5BdlpFWkRSbmhxWVhsSFdXczlJZ289fFZwtEwEGa5acoWBjbH9kkUAogaqxklaCnqtmsIT4Mqm\",\"te\":\"trailers\",\"accept-language\":\"en-US,en;q=0.5\",\"x-csrf-token\":\"tfyvdkwu9bFzwxDCEtDqHEmJ6/88UBnFx36LtUjoYzLNQGPIS+bx7WQFlJ8cYXsKMxkhPX5247qzLg5zflp6uw==\",\"accept\":\"application/json, text/plain, */*\",\"sec-fetch-mode\":\"cors\",\"sec-fetch-dest\":\"empty\",\"host\":\"portainer.local\"}},\"response\":{\"body\":\"\",\"http_code\":200,\"headers\":{\"X-Csrf-Token\":\"Wex5wG7A4OhSEyX5qriMdgtrbHuHNye2Xa1v8RQ/MLEhULV+aQjktEXVoaSkCR1gcfumucUR3ckp/eo3Io0pOA==\",\"Vary\":\"Cookie\",\"Docker-Experimental\":\"false\",\"Content-Type\":\"application/vnd.docker.multiplexed-stream\",\"Api-Version\":\"1.47\",\"X-Xss-Protection\":\"1; mode=block\",\"Ostype\":\"linux\",\"Connection\":\"close\",\"X-Content-Type-Options\":\"nosniff\",\"Date\":\"Sun, 29 Sep 2024 19:14:58 GMT\",\"Server\":\"nginx\",\"Strict-Transport-Security\":\"max-age=31536000\"}},\"producer\":{\"modsecurity\":\"ModSecurity v3.0.13 (Linux)\",\"connector\":\"ModSecurity-nginx v1.0.3\",\"secrules_engine\":\"DetectionOnly\",\"components\":[\"OWASP_CRS/4.7.0\\\"\"]},\"messages\":[{\"message\":\"Remote Command Execution: Direct Unix Command Execution\",\"details\":{\"match\":\"Matched \\\"Operator `Rx' with parameter `(?i)(?:^|b[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?u[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?s[\\\\\\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)? (4180 characters omitted)' against variable `ARGS_NAMES:stdout' (Value: `stdout' )\",\"reference\":\"o0,6v117,6o0,6v126,6\",\"ruleId\":\"932260\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf\",\"lineNumber\":\"519\",\"data\":\"Matched Data: stdout found within ARGS_NAMES:stdout: stdout\",\"severity\":\"2\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"application-multi\",\"language-shell\",\"platform-unix\",\"attack-rce\",\"paranoia-level/1\",\"OWASP_CRS\",\"capec/1000/152/248/88\",\"PCI/6.5.2\"],\"maturity\":\"0\",\"accuracy\":\"0\"}},{\"message\":\"Inbound Anomaly Score Exceeded (Total Score: 10)\",\"details\":{\"match\":\"Matched \\\"Operator `Ge' with parameter `5' against variable `TX:BLOCKING_INBOUND_ANOMALY_SCORE' (Value: `10' )\",\"reference\":\"\",\"ruleId\":\"949110\",\"file\":\"/etc/modsecurity/owasp-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf\",\"lineNumber\":\"222\",\"data\":\"\",\"severity\":\"0\",\"ver\":\"OWASP_CRS/4.7.0\",\"rev\":\"\",\"tags\":[\"anomaly-evaluation\",\"OWASP_CRS\"],\"maturity\":\"0\",\"accuracy\":\"0\"}}]}}"}
[2024-09-29T19:16:56,921][DEBUG][logstash.inputs.file][main][2b2622e24c2a9f5d1cc7f117b36fecf61b1605d666db7e17d1fabdd75387734d] handle_deletable_path {:path=>"/usr/share/logstash/ingest_data/modsec/audit2.log"}
[2024-09-29T19:16:58,087][DEBUG][logstash.outputs.file][main] Starting flush cycle

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [September 30, 2024, 12:38am UTC](https://discuss.elastic.co/t/logstash-doesnt-deliver-input-to-output/367976/2 "2024-09-30T00:38:50Z")

</div>

Hi @evgeniy

Most likely wrong codec

> **[Json\_lines codec plugin | Logstash Reference \[8.15\] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-json_lines.html)**

> NOTE: Do not use this codec if your source input is line-oriented JSON, for example, redis or file inputs. Rather, use the json codec.

Try just

`codec => "json"`

Yeah it's confusing.

Also, if logstash has already read the file, it won't read it again...

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 30, 2024, 2:38am UTC](https://discuss.elastic.co/t/logstash-doesnt-deliver-input-to-output/367976/3 "2024-09-30T02:38:54Z")

</div>

To expand on what Stephen said, the file input reads newline delimited lines from the file, strips off the newline, then feeds the lines to the codec. The json\_lines codec collects those lines until it gets a newline. But the file input removes the newlines, so the codec will never get one. It just keeps accumulating.

I think the developers are considering changing this.

Looking at the github history a [change](https://github.com/logstash-plugins/logstash-codec-json_lines/pull/43) was recently made (August) to limit the accumulated text to 20MB before throwing an exception, then that limit was [increased](https://github.com/logstash-plugins/logstash-codec-json_lines/pull/46) to 512 MB, and I think that limit is still being discussed.

Note that your logstash install will not reflect the current github code base until a new version of the codec is packaged/released (not sure what the right word for that is).

I've never seen a use case where I needed to consume a single line of JSON that was 40 MB, but the discussion the elastic developers are having tells me that they need to support it.

And, yeah, change the codec to "json".

---

<div class="post-metadata">

**Author:** ![evgeniy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evgeniy/32/137942_2.png) [@evgeniy](https://discuss.elastic.co/u/evgeniy)\
**Post date:** [September 30, 2024, 6:13am UTC](https://discuss.elastic.co/t/logstash-doesnt-deliver-input-to-output/367976/4 "2024-09-30T06:13:15Z")

</div>

@stephenb , @Badger , guys, you're great! It worked! Thank you very much!!!
