File paths must be absolute, relative path specified error while running logstash on docker

[pradeep_gadkari]

I am running ELK stack on docker using deviantony/docker-elk from Github. When I try reading log files from my local machine, logstash stops with error

[2021-04-20T12:36:43,101][ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<ArgumentError: File paths must be absolute, relative path specified: C:/logs/xyz/abcd*.log>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.2.3/lib/logstash/inputs/file.rb:283:in `block in register'", "org/jruby/RubyArray.java:1809:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.2.3/lib/logstash/inputs/file.rb:281:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:228:in `block in register_plugins'", "org/jruby/RubyArray.java:1809:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:227:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:386:in `start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:311:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:185:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:137:in `block in start'"], "pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#<Thread:0x1431553d run>"}

[2021-04-20T12:36:43,105][INFO ][logstash.javapipeline ][main] Pipeline terminated {"pipeline.id"=>"main"}

[2021-04-20T12:36:43,119][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}

I am using WSL2 based engine in Docker desktop so I do not need to enable file sharing between my localhost and the container. The input section of my logstash conf file is as below

input {
	file{
		path => "C:/logs/xyz/abcd*.log"
		start_position => 'beginning'
		sincedb_path => "NULL"
#		type => "rad_minol"
	}
}

My docker-compose file is

version: '3.2'

services:
  elasticsearch:
    build:
      context: elasticsearch/
      args:
        ELK_VERSION: $ELK_VERSION
    volumes:
      - type: bind
        source: ./elasticsearch/config/elasticsearch.yml
        target: /usr/share/elasticsearch/config/elasticsearch.yml
        read_only: true
      - type: volume
        source: elasticsearch
        target: /usr/share/elasticsearch/data
    ports:
      - "9200:9200"
      - "9300:9300"
    environment:
      ES_JAVA_OPTS: "-Xmx256m -Xms256m"
      ELASTIC_PASSWORD: *****
      # Use single node discovery in order to disable production mode and avoid bootstrap checks.
      # see: https://www.elastic.co/guide/en/elasticsearch/reference/current/bootstrap-checks.html
      discovery.type: single-node
    networks:
      - elk

  logstash:
    build:
      context: logstash/
      args:
        ELK_VERSION: $ELK_VERSION
    volumes:
      - type: bind
        source: ./logstash/config/logstash.yml
        target: /usr/share/logstash/config/logstash.yml
        read_only: true
      - type: bind
        source: ./logstash/pipeline
        target: /usr/share/logstash/pipeline
        read_only: true
    ports:
      - "5044:5044"
      - "5000:5000/tcp"
      - "5000:5000/udp"
      - "9600:9600"
    environment:
      LS_JAVA_OPTS: "-Xmx256m -Xms256m"
    networks:
      - elk
    depends_on:
      - elasticsearch

  kibana:
    build:
      context: kibana/
      args:
        ELK_VERSION: $ELK_VERSION
    volumes:
      - type: bind
        source: ./kibana/config/kibana.yml
        target: /usr/share/kibana/config/kibana.yml
        read_only: true
    ports:
      - "5601:5601"
    networks:
      - elk
    depends_on:
      - elasticsearch

networks:
  elk:
    driver: bridge

volumes:
  elasticsearch:

[Badger]

@pradeep_gadkari

		path => "C:/logs/xyz/abcd*.log"
		start_position => 'beginning'
		sincedb_path => "NULL"

If you do not want the in-memory sincedb persisted across restarts then set sincedb_path to "NUL". When you set it to "NULL" it is persisted in a file called NULL in the working directory of logstash.

The error message is telling you that Pathname.relative? is returning true. Note that the comment about is just testing for a leading / is nonsense. It tests for a drive letter too, if appropriate.

That said, your docker compose file appears to be building a UNIX filesystem. Your path option refers to a Windows filesystem. That is not valid on a UNIX machine.

[pradeep_gadkari]

I updated the docker-compose file and added

      - type: bind
        source: C:/logs/xyz
        target: /logstash/logs/xyz

in volumes under logstash