How to collect data in character special device like /dev/kmsg?

[linrl3]

Like the title said, how can I use filebeat to collect from character device, for example /dev/kmsg.

[jsoriano (Jaime Soriano)]

Hey @linrl3, welcome to discuss

Filebeat is not able to collect logs from character devices. Its filestream input is only able to collect logs from regular files. It also expects to be able to make seek operations, what is not allowed in character devices.

Though, I have done a quick test removing the seeks, and the code that checks if the file is a regular one, and it became able of collecting from /dev/kmsg.

So maybe Filebeat could have an option, or input, to allow collecting information from character devices. Taking into account that, without seek, it may collect repeated information if restarted.
Feel free to open an issue requesting that.

FTR, this is the patch I used to remove the seeks.

diff --git a/filebeat/input/filestream/filestream.go b/filebeat/input/filestream/filestream.go
index 0dce4b772d..4f558df6ad 100644
--- a/filebeat/input/filestream/filestream.go
+++ b/filebeat/input/filestream/filestream.go
@@ -67,11 +67,6 @@ func newFileReader(
        config readerConfig,
        closerConfig closerConfig,
 ) (*logFile, error) {
-       offset, err := f.Seek(0, os.SEEK_CUR)
-       if err != nil {
-               return nil, err
-       }
-
        readerCtx := ctxtool.WithCancelContext(ctxtool.FromCanceller(canceler))
        tg := unison.TaskGroupWithCancel(readerCtx)
 
@@ -84,7 +79,6 @@ func newFileReader(
                closeInactive:      closerConfig.OnStateChange.Inactive,
                closeRemoved:       closerConfig.OnStateChange.Removed,
                closeRenamed:       closerConfig.OnStateChange.Renamed,
-               offset:             offset,
                lastTimeRead:       time.Now(),
                backoff:            backoff.NewExpBackoff(canceler.Done(), config.Backoff.Init, config.Backoff.Max),
                readerCtx:          readerCtx,
diff --git a/filebeat/input/filestream/input.go b/filebeat/input/filestream/input.go
index 2bd53c0e4d..febb963db1 100644
--- a/filebeat/input/filestream/input.go
+++ b/filebeat/input/filestream/input.go
@@ -286,22 +286,12 @@ func (inp *filestream) openFile(log *logp.Logger, path string, offset int64) (*o
 }
 
 func checkFileBeforeOpening(fi os.FileInfo) error {
-       if !fi.Mode().IsRegular() {
-               return fmt.Errorf("tried to open non regular file: %q %s", fi.Mode(), fi.Name())
-       }
 
        return nil
 }
 
 func (inp *filestream) initFileOffset(file *os.File, offset int64) error {
-       if offset > 0 {
-               _, err := file.Seek(offset, io.SeekCurrent)
-               return err
-       }
-
-       // get offset from file in case of encoding factory was required to read some data.
-       _, err := file.Seek(0, io.SeekCurrent)
-       return err
+       return nil
 }

[linrl3]

Thank you @jsoriano ! I will try this code on my side.