S3 input plugin - add path field, preserve folder hierarchy during backup?


(LogSpree) #1

While pushing logs from local to ES using file input plugin, a path field gets added which contains the actual path of the log file which the document was created from.
But this field doesn't show up when using the AWS s3 input plugin. How do I add it?

Also, I have set the backup_to_dir to a parent folder which contains sub-folders. Eg. :

  1.    /p_dir
    
  2.        /dir1
    
  3.               log0
    
  4.               log1
    
  5.               log 2
    
  6.        /dir2
    
  7.               log0
    
  8.               log1
    

In this case, I am facing an error during backup.

Please help me resolve these issues.

My s3 input plugin:

s3 {
                "access_key_id" => "***"
                "secret_access_key" => "***"
                "backup_to_dir" => "/home/xyz/aws_s3_backup_data/"
                "bucket" => "bucket1"
                "prefix" => "bucket1a/p_dir/"
        }

(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.