daniel_a
(daniel_aug)
May 12, 2021, 11:25pm
1
How do I remove single quotes from the message field using Logstash gsub filter? I've tried the syntax below, and it didn't work.
mutate {
gsub => [ "[message]", "'", "" ]
}
mutate {
gsub => [ "[message]", "\\'", "" ]
}
Example:
{ "description": "These are just random users' credentials." }
In the example above, the single quote is right after the word users . I just want to remove it, I don't need to replace it with anything else.
Badger
May 12, 2021, 11:34pm
2
mutate { gsub => [ "[message]", "'", "" ] }
is the way to do it.
daniel_a
(daniel_aug)
May 13, 2021, 12:31am
3
I was thinking the same, but I'm still getting an error.
This is the input/filter code.
input {
generator {
lines => [
'{ "description": "These are just random users' credentials." }'
]
count => 1
}
}
filter {
mutate {
gsub => [ "[message]", "'", "" ]
}
Error:
[2021-05-03T04:30:41,717][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"{\", \",\", \"]\" at line 4, column 55 (byte 92) after input {\n generator {\n lines => [\n '{ \"description\": \"These are just random users' ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:184:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:365:in `block in converge_state'"]}
Badger
May 13, 2021, 12:45am
4
The ' after users is closing out the ' at the start of the line, you need to escape it.
daniel_a
(daniel_aug)
May 13, 2021, 1:14am
5
This makes sense now. It seems it's just harder to test this with the generator. I should be all good when I'm ready to pull logs via its primary method, logs won't be wrapped with single quotes.
Thanks!
I use a generator input a lot for testing things, but sometimes it is easier to put a line in a file, set sincedb_path to /dev/null, and use --config.reload.automatic
to re-read the file every time you edit the configuration.
1 Like
system
(system)
Closed
June 10, 2021, 1:49am
7
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.