How to remove a line break from a field in logstash

Elastic Stack Logstash
1

Hello, everyone!

I'm trying to ingest a csv like this in logstash using csv filter:

column1;column2;column3;column4;column5;
201905; DADOS PARA O CADASTRO;2019-05-22 09:48:08;2019-05-22 10:06:05;"Criar acesso para
NOME
CPF
E-MAIL
Larissa Santana Garcia
094.350.614-00
larissa.garcia@wsb.adv.br
Maria das Dores Alves de Menezes
107.243.164-50
maria.menezes@wsb.adv.br
Ana Luiza Lacerda Cunha
089.140.404-02
ana.lacerda@wsb.adv.br
Ana Caroline Araújo de Andrade
700.856.754-58
ana.araujo@wsb.adv.br";

How you can see, there`s a line break after "Criar acesso para" and the logstash consider a line break as a new event.

The error in convert:

[2019-05-30T12:24:40,368][WARN ][logstash.filters.csv ] Error parsing csv {:field=>"message", :source=>"2019052238000853;RES: DADOS PARA O CADASTRO DO ELAW;2019-05-22 09:48:08;2019-05-22 10:06:05;;Consultores;new;leandro.coelho@viavarejo.com.br;Via Varejo;ADM::CRIA├ç├âO DE USU├üRIO;SLA - Medio;Ajuste/Corre├º├úo em Configura├º├Áes;rogerio.owsiany;0;2;Leliana Souza;;Sem atua├º├úo;;Sem atua├º├úo;"Criar acesso para\r", :exception=>#<CSV::MalformedCSVError: Unclosed quoted field on line 1.>}

And the logstash create a event to "NOME", "CPF", ETC....

I tried:

mutate {
gsub => [ "message", "\r", "" ]
gsub => [ "Resumo do chamado", "\r", "" ]
}

but no sucess....

Some idea what to do?

Thanks!

2

Unless you have config.support_escapes enables you do not refer to a newline using \r, you have to use a literal newline in the configuration:

mutate { gsub => [ "message", "
", "" ] }

But that is not going to fix your problem, which is that you need to consume multiple lines as a single event. You may be able to do that using a multiline codec on the input.

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.