Tomcat HTTP requests are not showing up in kibana


#1

I installed Elastic Search, Kibana and Packet Beat. I have apache tomcat running on my same windows machine. localhost:8080 requests are not showing up.

packetbeat.yml file

#################### Packetbeat Configuration Example #########################

# This file is an example configuration file highlighting only the most common
# options. The packetbeat.full.yml file from the same directory contains all the
# supported options with more comments. You can use it as a reference.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/packetbeat/index.html

#============================== Network device ================================

# Select the network interface to sniff the data. On Linux, you can use the
# "any" keyword to sniff on all connected interfaces.
packetbeat.interfaces.device: 0

#================================== Flows =====================================

# Set `enabled: false` or comment out all options to disable flows reporting.
packetbeat.flows:
  # Set network flow timeout. Flow is killed if no packet is received before being
  # timed out.
  timeout: 30s

  # Configure reporting period. If set to -1, only killed flows will be reported
  period: 10s

#========================== Transaction protocols =============================

packetbeat.protocols.icmp:
  # Enable ICMPv4 and ICMPv6 monitoring. Default: false
  enabled: true

packetbeat.protocols.amqp:
  # Configure the ports where to listen for AMQP traffic. You can disable
  # the AMQP protocol by commenting out the list of ports.
  ports: [5672]

packetbeat.protocols.cassandra:
  #Cassandra port for traffic monitoring.
  ports: [9042]

packetbeat.protocols.dns:
  # Configure the ports where to listen for DNS traffic. You can disable
  # the DNS protocol by commenting out the list of ports.
  ports: [53]

  # include_authorities controls whether or not the dns.authorities field
  # (authority resource records) is added to messages.
  include_authorities: true

  # include_additionals controls whether or not the dns.additionals field
  # (additional resource records) is added to messages.
  include_additionals: true

packetbeat.protocols.http:
  # Configure the ports where to listen for HTTP traffic. You can disable
  # the HTTP protocol by commenting out the list of ports.
  enabled: true
  ports: [80, 8080, 8000, 5000, 8002]
  send_request: true
  send_response: true

packetbeat.protocols.memcache:
  # Configure the ports where to listen for memcache traffic. You can disable
  # the Memcache protocol by commenting out the list of ports.
  ports: [11211]

packetbeat.protocols.mysql:
  # Configure the ports where to listen for MySQL traffic. You can disable
  # the MySQL protocol by commenting out the list of ports.
  ports: [3306]

packetbeat.protocols.pgsql:
  # Configure the ports where to listen for Pgsql traffic. You can disable
  # the Pgsql protocol by commenting out the list of ports.
  ports: [5432]

packetbeat.protocols.redis:
  # Configure the ports where to listen for Redis traffic. You can disable
  # the Redis protocol by commenting out the list of ports.
  ports: [6379]

packetbeat.protocols.thrift:
  # Configure the ports where to listen for Thrift-RPC traffic. You can disable
  # the Thrift-RPC protocol by commenting out the list of ports.
  ports: [9090]

packetbeat.protocols.mongodb:
  # Configure the ports where to listen for MongoDB traffic. You can disable
  # the MongoDB protocol by commenting out the list of ports.
  ports: [27017]

packetbeat.protocols.nfs:
  # Configure the ports where to listen for NFS traffic. You can disable
  # the NFS protocol by commenting out the list of ports.
  ports: [2049]

#=========================== Monitored processes ==============================
#packetbeat.procs:

#  enabled: true
#   monitored:

#    - process: tomcat7

#      cmdline_grep: tomcat7

#================================ General =====================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:10.232.41.180

# The tags of the shipper are included in their own field with each
# transaction published.
tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the
# output.
#fields:
#  env: staging

#================================ Outputs =====================================

# Configure what outputs to use when sending the data collected by the beat.
# Multiple outputs may be used.

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]	

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"

#----------------------------- Logstash output --------------------------------
#output.logstash:
  # The Logstash hosts
  #hosts: ["localhost:5044"]

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  # Certificate for SSL client authentication
  #ssl.certificate: "/etc/pki/client/cert.pem"

  # Client Certificate Key
  #ssl.key: "/etc/pki/client/cert.key"

#================================ Logging =====================================

# Sets log level. The default log level is info.
# Available log levels are: critical, error, warning, info, debug
#logging.level: debug

# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publish", "service".
#logging.selectors: ["*"]

(Andrew Kroh) #2

You a probably not monitoring the loopback device. See https://www.elastic.co/guide/en/beats/packetbeat/current/faq.html#packetbeat-loopback-interface

What is device 0 using the output of ./packetbeat.exe -devices?


#3

Hey Andrew
My device
0: \Device\NPF_{XXXXXXXXXXXX} (Marvell Yukon Ethernet Controller)(XXXXXXXXX XX.XX.XX.XX)


(Andrew Kroh) #4

So that interface will not see your "localhost" traffic; it will see traffic in/out on 10.232.41.180. For localhost you need to use the npcap driver and configure packetbeat to listen to the loopback interface (as stated in the FAQ).


#5

Thanks andrew. i'll change the configuration and give a try. If i access any http url in the browser, i'm not getting the logs in kibana. What am i missing?


(Andrew Kroh) #6

You should be seeing that traffic and DNS traffic in Elasticsearch. What do you see when you do GET _cat/indices request to ES?


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.