Date Column has some rows with NULL - strict_date_optional_time causes Exception

[stephenb (Stephen Brown)]

@Ethan777100

How did extra rows appear in my Index?

A mystery for you to solve!

You could try real quick with filebeat see if you get the same.

[Ethan777100 (Ethan Chia)]

A new docker container to host filebeat?

Venturing into smth new just to experiment.

Is this the only config file I need?

[stephenb (Stephen Brown)]

That was just a suggestion

I wouldn't bother with docker. ...I would just download the filebeat .zip and try from the command line I gave you the filebeat.yml above you just need to adjust a bit

@Ethan777100

filter {
    mutate {
        gsub => ["message", "\r", ""]
    }
}

I wonder if you are having trouble with the newlines?

I am just giving suggestions....

You gotta figure it out...

[Ethan777100 (Ethan Chia)]

I don't even know where to start looking for the new lines issue

But I just managed to get 2 big csv files ingested for this data category.

I think I will continue data analysis first as per smooth-business. From more intimate usage, i might see any flaws in quality to hinder my work.