# Can i collect and parse TCP response messages from a server and port ? if so kindly help me with the config file and steps

**URL:** <https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269>\
**Category:** Logstash\
**Created:** [August 14, 2018, 2:12am UTC](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269 "2018-08-14T02:12:23Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![gopesh\_reddy](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gopesh\_reddy](https://discuss.elastic.co/u/gopesh_reddy)\
**Post date:** [August 14, 2018, 2:12am UTC](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269/1 "2018-08-14T02:12:23Z")

</div>

input {  
tcp {  
port =\> 5000  
type =\> syslog  
}  
udp {  
port =\> 5000  
type =\> syslog  
}  
}

filter {  
if [type] == "syslog" {  
grok {  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" }  
add\_field =\> ["received\_at", "%{@timestamp}"]  
add\_field =\> ["received\_from", "%{host}"]  
}  
date {  
match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]  
}  
}  
}

output {  
elasticsearch { hosts =\> ["localhost:9200"] }  
stdout { codec =\> rubydebug }

---

<div class="post-metadata">

**Author:** ![Teja](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/teja/32/16324_2.png) [@Teja](https://discuss.elastic.co/u/Teja)\
**Post date:** [August 14, 2018, 3:33am UTC](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269/2 "2018-08-14T03:33:56Z")

</div>

To collect and parse (TCP/UDP) packets install packetbeat in the host where you want to trap the packets then point it to logstash.

Refer this link for more details [https://www.elastic.co/products/beats/packetbeat](https://www.elastic.co/products/beats/packetbeat)

---

<div class="post-metadata">

**Author:** ![gopesh\_reddy](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gopesh\_reddy](https://discuss.elastic.co/u/gopesh_reddy)\
**Post date:** [August 14, 2018, 3:56am UTC](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269/3 "2018-08-14T03:56:19Z")

</div>

thanks for your thoughts.

one more clarification

The logs message which i get from the server will be binary format Will the Packetbeat parse the data to human readable format ?

---

<div class="post-metadata">

**Author:** ![Teja](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/teja/32/16324_2.png) [@Teja](https://discuss.elastic.co/u/Teja)\
**Post date:** [August 14, 2018, 5:38am UTC](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269/4 "2018-08-14T05:38:01Z")

</div>

Are you trying to collect and parse network packets or rawlogs from server?

Can you please elaborate more about your server and binary format?

Packetbeat is the application which is like the traffic listener where it will listen actuall network traffic(Request and Response/ inbound and outbound) in any interface then it will convert them to message(plain text).

---

<div class="post-metadata">

**Author:** ![gopesh\_reddy](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gopesh\_reddy](https://discuss.elastic.co/u/gopesh_reddy)\
**Post date:** [August 14, 2018, 9:39pm UTC](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269/5 "2018-08-14T21:39:12Z")

</div>

I want to connect to a server(IP address) through a port and send request to the server and then collect and store the response from the server which is word and byte format depending on the request. in a ELK stack and visualize the logs in Kibana

---

<div class="post-metadata">

**Author:** ![Teja](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/teja/32/16324_2.png) [@Teja](https://discuss.elastic.co/u/Teja)\
**Post date:** [August 15, 2018, 5:31am UTC](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269/6 "2018-08-15T05:31:11Z")

</div>

As per your response, My understanding is that you want to capture request and response from the server. In this case please go head and install packetbeat and trap the packets to visualize in kibana.

Before that it would be great try to capture your request and response with TCPDUMP or WireShark then see how your packets look alike.

---

<div class="post-metadata">

**Author:** ![gopesh\_reddy](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gopesh\_reddy](https://discuss.elastic.co/u/gopesh_reddy)\
**Post date:** [August 15, 2018, 9:36pm UTC](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269/7 "2018-08-15T21:36:14Z")

</div>

Really thanks for sharing your knowledge domain

i need to send request to an IP address through an port number and the response needs to be collected and visualized using kinban . Can you help with the packetbeat config file setup ?

---

<div class="post-metadata">

**Author:** ![gopesh\_reddy](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gopesh\_reddy](https://discuss.elastic.co/u/gopesh_reddy)\
**Post date:** [August 16, 2018, 2:29am UTC](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269/8 "2018-08-16T02:29:03Z")

</div>

PS C:\Elastic Search\Packetbeat\packetbeat-6.3.2-windows-x86\_64\> .\packetbeat.exe setup --dashboards  
Exiting: error initializing publisher: missing required field accessing 'output.elasticsearch.hosts'  
Kindly tell me where i have gone wrong

Attache is the snippet of the yml file  
packetbeat.interfaces.device: 1

#================================== Flows =====================================

# Set `enabled: false` or comment out all options to disable flows reporting.

packetbeat.flows:

# Set network flow timeout. Flow is killed if no packet is received before being

# timed out.

timeout: 30s

# Configure reporting period. If set to -1, only killed flows will be reported

period: 10s

#========================== Transaction protocols =============================

packetbeat.protocols:

- type: icmp

- type: amqp

- type: cassandra  
#Cassandra port for traffic monitoring.  
ports: [9042]

- type: dns

- type: http

- type: memcache

- type: mysql

- type: pgsql

- type: redis

- type: thrift

- type: mongodb

- type: nfs

- type: tls

#==================== Elasticsearch template setting ==========================

setup.template.settings:  
index.codec: best\_compression  
index.number\_of\_shards: 1

#\_source.enabled: false

#================================ General =====================================

# The name of the shipper that publishes the network data. It can be used to group

# all the transactions sent by a single shipper in the web interface.

#name:

# The tags of the shipper are included in their own field with each

# transaction published.

#tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the

# output.

#fields:

# env: staging

#============================== Dashboards =====================================

# These settings control loading the sample dashboards to the Kibana index. Loading

# the dashboards is disabled by default and can be enabled either by setting the

# options here, or by using the `-setup` CLI flag or the `setup` command.

setup.dashboards.enabled: true  
setup.dashboards.index: packetbeat-\*  
#setup.dashboards.beat: packetbeat

# The URL from where to download the dashboards archive. By default this URL

# has a value which is computed based on the Beat name and version. For released

# versions, this URL points to the dashboard archive on the [artifacts.elastic.co](http://artifacts.elastic.co)

# website.

#setup.dashboards.url:

#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.

# This requires a Kibana endpoint configuration.

setup.kibana:

# Kibana Host

# Scheme and port can be left out and will be set to the default (http and 5601)

# In case you specify and additional path, the scheme is required: [http://localhost:5601/path](http://localhost:5601/path)

# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601

host: "localhost:5601"

#============================= Elastic Cloud ==================================

# These settings simplify using packetbeat with the Elastic Cloud ([https://cloud.elastic.co/](https://cloud.elastic.co/)).

# The cloud.id setting overwrites the `output.elasticsearch.hosts` and

# `setup.kibana.host` options.

# You can find the `cloud.id` in the Elastic Cloud web UI.

#cloud.id:

# The cloud.auth setting overwrites the `output.elasticsearch.username` and

# `output.elasticsearch.password` settings. The format is `<user>:<pass>`.

#cloud.auth:

#================================ Outputs =====================================

# Configure what output to use when sending the data collected by the beat.

#-------------------------- Elasticsearch output ------------------------------  
output.elasticsearch:

hosts: ["localhost:9200"]

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 13, 2018, 2:29am UTC](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269/9 "2018-09-13T02:29:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.