How to remove a line break from a field in logstash

Hello, everyone!

I'm trying to ingest a csv like this in logstash using csv filter:

201905; DADOS PARA O CADASTRO;2019-05-22 09:48:08;2019-05-22 10:06:05;"Criar acesso para
Larissa Santana Garcia
Maria das Dores Alves de Menezes
Ana Luiza Lacerda Cunha
Ana Caroline Araújo de Andrade

How you can see, there`s a line break after "Criar acesso para" and the logstash consider a line break as a new event.

The error in convert:

[2019-05-30T12:24:40,368][WARN ][logstash.filters.csv ] Error parsing csv {:field=>"message", :source=>"2019052238000853;RES: DADOS PARA O CADASTRO DO ELAW;2019-05-22 09:48:08;2019-05-22 10:06:05;;Consultores;new;;Via Varejo;ADM::CRIA├ç├âO DE USU├üRIO;SLA - Medio;Ajuste/Corre├º├úo em Configura├º├Áes;rogerio.owsiany;0;2;Leliana Souza;;Sem atua├º├úo;;Sem atua├º├úo;"Criar acesso para\r", :exception=>#<CSV::MalformedCSVError: Unclosed quoted field on line 1.>}

And the logstash create a event to "NOME", "CPF", ETC....

I tried:

mutate {
gsub => [ "message", "\r", "" ]
gsub => [ "Resumo do chamado", "\r", "" ]

but no sucess....

Some idea what to do?


Unless you have config.support_escapes enables you do not refer to a newline using \r, you have to use a literal newline in the configuration:

mutate { gsub => [ "message", "
", "" ] }

But that is not going to fix your problem, which is that you need to consume multiple lines as a single event. You may be able to do that using a multiline codec on the input.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.