# Messages escaped in Filebeat **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [June 15, 2020, 11:36am UTC](https://discuss.elastic.co/t/messages-escaped-in-filebeat/237125 "2020-06-15T11:36:31Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![JeanN](https://avatars.discourse-cdn.com/v4/letter/j/b38774/32.png) [@JeanN](https://discuss.elastic.co/u/JeanN)\ **Post date:** [June 15, 2020, 11:36am UTC](https://discuss.elastic.co/t/messages-escaped-in-filebeat/237125/1 "2020-06-15T11:36:31Z")
Hi, I have an issue with Filebeat, I have escaping characters that I can't remove from the message value, or I don't know how. I know there already are some topics on that matter but none solved my issue, that's why I'm allowing myself to open this one. Please kindly tell me if there is already a solution I wouldn't have found. I've configured a Filebeat to query a MISP instance (without using the MISP module) and to send the results to a Logstash instance in UDP. **Here is the Filebeat conf:** ```auto filebeat.inputs: - type: httpjson url: "https://10.0.0.5/events/restSearch" http_method: "POST" json_objects_array: "response" http_headers: Authorization: "XXXXXXXXXXXXXXXX" Accept: "application/json" Content-Type: "application/json" interval: 15 pagination.enabled: "true" pagination.extra_body_content: max_items: "1" ssl: verification_mode: "none" client_authentication: "none" # fields_under_root: true # json.keys_under_root: true output.logstash: hosts: ["10.0.0.1:5044"] ``` **Here is the Logstash beat's input conf:** ```auto input { udp { host => "10.0.0.2" port => 6044 } } filter { json { source => "message" } prune { whitelist_names => ["message"] } mutate { gsub => ["message",'\"','"'] } } output { #for debug purposes only stdout { codec => rubydebug } } ``` I've tried the use of fields\_under\_root and json.keys\_under\_root set to true but none of that helped. I've also tried to modify the message value on the second Logstash in the filter but it does not work either. In the end, I would like to extract the message value without the escaping. **Here is the output I get :** ```auto { "host" => { "os" => { "codename" => "bionic", "kernel" => "4.15.0-55-generic", "family" => "debian", "version" => "18.04.4 LTS (Bionic Beaver)", "name" => "Ubuntu", "platform" => "ubuntu" }, "hostname" => "misp", "containerized" => false, "id" => "cd4d32abc1224be991e9394f8ddfeef6", "architecture" => "x86_64", "name" => "misp" }, "message" => "{\"Event\":{\"Attribute\":[{\"Galaxy\":[],\"ShadowAttribute\":[],\"category\":\"Artifacts dropped\",\"comment\":\"c'est un hash\",\"deleted\":false,\"disable_correlation\":false,\"distribution\":\"5\",\"event_id\":\"3\",\"first_seen\":null,\"id\":\"4\",\"last_seen\":null,\"object_id\":\"0\",\"object_relation\":null,\"sharing_group_id\":\"0\",\"timestamp\":\"1591806080\",\"to_ids\":true,\"type\":\"text\",\"uuid\":\"5ee10880-656c-44ce-8389-02eec0a83c0f\",\"value\":\"hashhashhash\"}],\"Galaxy\":[],\"Object\":[],\"Org\":{\"id\":\"1\",\"local\":true,\"name\":\"ORGNAME\",\"uuid\":\"5ec6f868-6020-45b5-b9d5-503a84db2e2f\"},\"Orgc\":{\"id\":\"1\",\"local\":true,\"name\":\"ORGNAME\",\"uuid\":\"5ec6f868-6020-45b5-b9d5-503a84db2e2f\"},\"RelatedEvent\":[],\"ShadowAttribute\":[],\"analysis\":\"0\",\"attribute_count\":\"1\",\"date\":\"2020-06-10\",\"disable_correlation\":false,\"distribution\":\"1\",\"extends_uuid\":\"\",\"id\":\"3\",\"info\":\"Malware\",\"locked\":false,\"org_id\":\"1\",\"orgc_id\":\"1\",\"proposal_email_lock\":false,\"publish_timestamp\":\"0\",\"published\":false,\"sharing_group_id\":\"0\",\"threat_level_id\":\"1\",\"timestamp\":\"1591806080\",\"uuid\":\"5ee10817-5a74-4d8a-8c58-02e2c0a83c0f\"}}", "ecs" => { "version" => "1.4.0" }, "@timestamp" => 2020-06-15T11:12:17.223Z, "@version" => "1", "input" => { "type" => "httpjson" }, "agent" => { "id" => "8268922d-2b13-40fe-94a0-026aa9faa72d", "hostname" => "misp", "ephemeral_id" => "9cad9c43-7973-4caa-a2e9-0ba2169df152", "version" => "7.6.2", "type" => "filebeat" }, "event" => { "created" => "2020-06-15T11:12:17.223Z" }, "tags" => [ [0] "beats_input_codec_plain_applied" ] } ``` Would anyone have any idea of what could cause this and what could help remediate it? Thank you in advance for your help. Edit: I have clean my configuration. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [June 15, 2020, 3:06pm UTC](https://discuss.elastic.co/t/messages-escaped-in-filebeat/237125/2 "2020-06-15T15:06:44Z")
Hey @JeanN, I think that processing MISP can be complicated. Is there any reason why you are not using the MISP module? Most modules use ingest pipelines, that can be tricky to use with Logstash, but in the case of the MISP module processing is done client side, in filebeat, using [these processors](https://github.com/elastic/beats/blob/v7.7.1/x-pack/filebeat/module/misp/threat/config/input.yml), and this [javascript pipeline](https://github.com/elastic/beats/blob/v7.7.1/x-pack/filebeat/module/misp/threat/config/pipeline.js). ---
**Author:** ![JeanN](https://avatars.discourse-cdn.com/v4/letter/j/b38774/32.png) [@JeanN](https://discuss.elastic.co/u/JeanN)\ **Post date:** [June 15, 2020, 3:25pm UTC](https://discuss.elastic.co/t/messages-escaped-in-filebeat/237125/3 "2020-06-15T15:25:16Z")
Hello @jsoriano , I simplified my configuration. I don't want to use the MISP module, and I would like to keep the original raw message, so that I can send it later. As in this example: MISP with filebeat -\> logstash -\> MISP ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [June 15, 2020, 5:53pm UTC](https://discuss.elastic.co/t/messages-escaped-in-filebeat/237125/4 "2020-06-15T17:53:50Z")
> [@JeanN](#): > > I've tried the use of fields\_under\_root and json.keys\_under\_root set to true but none of that helped. `fields_under_root` only applies to the custom fields added with `fields`. `json.keys_under_root` is not supported by the `httpjson` input. Something you can try is to decode json already in filebeat by using the `decode_json_fields` processor, as in the example in the `httpjson` input docs: [HTTP JSON input | Filebeat Reference [7.7] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.7/filebeat-input-httpjson.html) To merge the decoded fields at the root level, you need to [use `target: ""` in `decode_json_fields`](https://www.elastic.co/guide/en/beats/filebeat/7.7/decode-json-fields.html). ---
**Author:** ![JeanN](https://avatars.discourse-cdn.com/v4/letter/j/b38774/32.png) [@JeanN](https://discuss.elastic.co/u/JeanN)\ **Post date:** [June 18, 2020, 12:16pm UTC](https://discuss.elastic.co/t/messages-escaped-in-filebeat/237125/5 "2020-06-18T12:16:31Z")
Thanks for your help. You solved my problem. ---
**Author:** ![aguida79](https://avatars.discourse-cdn.com/v4/letter/a/eb8c5e/32.png) [@aguida79](https://discuss.elastic.co/u/aguida79)\ **Post date:** [June 22, 2020, 8:45pm UTC](https://discuss.elastic.co/t/messages-escaped-in-filebeat/237125/6 "2020-06-22T20:45:30Z")
Hi Jean, Can you share your final config file of filebeat.yml to get MISP events? I don't want to use the MISP module too, is not working right for me. Thanks in advance. Regards, Alejandro ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 20, 2020, 10:45pm UTC](https://discuss.elastic.co/t/messages-escaped-in-filebeat/237125/7 "2020-07-20T22:45:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.