# How do I get the actual content of the HTTP messages body?

**URL:** <https://discuss.elastic.co/t/how-do-i-get-the-actual-content-of-the-http-messages-body/102951>\
**Category:** Beats\
**Tags:** packetbeat\
**Created:** [October 6, 2017, 6:45am UTC](https://discuss.elastic.co/t/how-do-i-get-the-actual-content-of-the-http-messages-body/102951 "2017-10-06T06:45:24Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![mtrivedi](https://avatars.discourse-cdn.com/v4/letter/m/d2c977/32.png) [@mtrivedi](https://discuss.elastic.co/u/mtrivedi)\
**Post date:** [October 6, 2017, 6:45am UTC](https://discuss.elastic.co/t/how-do-i-get-the-actual-content-of-the-http-messages-body/102951/1 "2017-10-06T06:45:24Z")

</div>

I am using Packetbeat on a remote machine running CentOS.

```
> [root@bng1 bin]# ./packetbeat -version
> packetbeat version 5.6.1 (amd64), libbeat 5.6.1

```

I want to fetch the raw content of the http 200 request and the response body, but instead, I only get the field **`Content-Length`**

Here is a screenshot of a packet as shown in kibana.

 ![Capture](https://us1.discourse-cdn.com/elastic/original/3X/4/e/4eaf615dc6fcd00a8603c0c5b5c4c7267df4e2a5.PNG)

My `packetbeat.yml` file below:

```
packetbeat.protocols.http:
  # Configure the ports where to listen for HTTP traffic. You can disable
  # the HTTP protocol by commenting out the list of ports.
  ports: [80, 8080, 8000, 5000, 8002]

  send_request: true
  send_response: true
  include_body_for: ['image/webp','application/x-javascript','*/*','application/octet-stream','text/xml','application/xml','application/xhtml+xml','text/plain','text/html','application/json','text/javascript','application/javascript','application/x-www-form-urlencoded']
  split_cookie: true
  send_all_headers: true
```

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [October 6, 2017, 10:48am UTC](https://discuss.elastic.co/t/how-do-i-get-the-actual-content-of-the-http-messages-body/102951/2 "2017-10-06T10:48:30Z")

</div>

The HTTP module compares the `include_body_for` with the `contents-type` strings (must be exact match). I don't see the `Contents-Type` header in your sample.

---

<div class="post-metadata">

**Author:** ![mtrivedi](https://avatars.discourse-cdn.com/v4/letter/m/d2c977/32.png) [@mtrivedi](https://discuss.elastic.co/u/mtrivedi)\
**Post date:** [October 6, 2017, 11:34am UTC](https://discuss.elastic.co/t/how-do-i-get-the-actual-content-of-the-http-messages-body/102951/3 "2017-10-06T11:34:13Z")

</div>

Yes, I figured that out.

**BUT**

I am able to see the `Content-Body` only for `Content-Type='text/html'` which are always HTTP 401 messages.

Currently, in my project, I need to capture the SOAP messages (which are in XML format by the way) in HTTP 200 messages.

But, the problem is there is no `Content-Type` for these type of packets. I am unable to see the `Content-Type` even in **Wireshark** tool. But, I am able to see a `Data` field which have the SOAP requests and responses. Why can't I see the same in **Packetbeat**?

Here is a screenshot of a HTTP 200 packet in the Wireshark tool. (Captured as a `.pcap` file using `tcpdump` command under the same environment):

 ![Capture2](https://us1.discourse-cdn.com/elastic/original/3X/b/c/bcb595deacacb9a2af303f4256aeb3ece5375d9b.PNG)

---

<div class="post-metadata">

**Author:** ![mtrivedi](https://avatars.discourse-cdn.com/v4/letter/m/d2c977/32.png) [@mtrivedi](https://discuss.elastic.co/u/mtrivedi)\
**Post date:** [October 6, 2017, 11:40am UTC](https://discuss.elastic.co/t/how-do-i-get-the-actual-content-of-the-http-messages-body/102951/4 "2017-10-06T11:40:10Z")

</div>

Why can't I get the request/response body irrespective of whether I use `include_body_for` or not? I see that Packetbeat needs this field to distinguish between the type of body in these HTTP packets.

There must be something that I am missing out.

**OR**

There must be an issue or a limitation in Packetbeat itself.

---

<div class="post-metadata">

**Author:** ![mtrivedi](https://avatars.discourse-cdn.com/v4/letter/m/d2c977/32.png) [@mtrivedi](https://discuss.elastic.co/u/mtrivedi)\
**Post date:** [October 7, 2017, 2:10pm UTC](https://discuss.elastic.co/t/how-do-i-get-the-actual-content-of-the-http-messages-body/102951/5 "2017-10-07T14:10:37Z")

</div>

No replies...  
😞

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [October 8, 2017, 11:35am UTC](https://discuss.elastic.co/t/how-do-i-get-the-actual-content-of-the-http-messages-body/102951/6 "2017-10-08T11:35:36Z")

</div>

It is kind of a limitation in packetbeat itself (on purpose though). Packetbeat requires the presence of `Content-Type` + it requires the type to match `include_body_for`. Without contents type there is no really guarantee on the actual contents. It's just some bytes. Packetbeat will not try to index just something potentially being binary.

The easiest fix (I hope) is to have your application properly set the contents type. I understand it's not always possible to modify an application/library. Feel free to open an [enhancement request](https://github.com/elastic/beats/issues) or a [PR](https://github.com/elastic/beats/pulls) introducing a new setting to include the body if the contents type is missing ([related code path is here](https://github.com/elastic/beats/blob/master/packetbeat/protos/http/http.go#L586))

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 5, 2017, 11:35am UTC](https://discuss.elastic.co/t/how-do-i-get-the-actual-content-of-the-http-messages-body/102951/7 "2017-11-05T11:35:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
