Pipeline with id [packetbeat-8.15.2-routing] does not exist, dropping event!

Elastic Stack Beats
1

Hi, I'm testing out packetbeat to ship dns data to our self-hosted elastic server, I have tried version x64 windows 8.15.2 and 8.15.0 and neither version created the routing pipeline

{"log.level":"warn","@timestamp":"2024-10-02T15:38:09.327+0100","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.(*Client).applyItemStatus","file.name":"elasticsearch/client.go","file.line":489},"message":"Cannot index event '/// snipped packet data /// (status=400): {"type":"illegal_argument_exception","reason":"pipeline with id [packetbeat-8.15.2-routing] does not exist"}, dropping event!","service.name":"packetbeat","log.type":"event","ecs.version":"1.6.0"}

i do appear to have the -default and -geoip pipelines though
i have added every role going to the user account, so i'm assuming it's not a permissions issue as it does create the other pipelines

this is my first foray into beats and elastic, any idea what is going wrong here?

Cheers

2

Did you ran the packetbeat setup before to install the ingest pipelines and dashboards?

3

hi, yes i ran the setup, didn't see any errors in the output
I do have some pipelines, just not the one it wants...

image
image1353×232 15.4 KB

4

Hi @jimB100 Welcome to the community

How did you install apckbeat and how did you run setup? ... did you let it finish?

I just create a fresh Elastic / Kibana / Packetbeat 8.15.2

./packetbeat setup -e

Here are the all the ingest pipelines.... perhaps run setup again...including the routing one...

Screenshot 2024-10-02 at 9.46.09 AM
Screenshot 2024-10-02 at 9.46.09 AM3352×1662 457 KB

Can you rerun setup capture the output and share...

5

If you run

.\packetbeat setup --pipelines -e -d "*"

You should see all the pipelines get loaded , there will be some odd debug messages but that is ok...

If you run this again then it will show each is loaded...

Seem like it did not finish for you..

6

hi @stephenb i'll give the --pipelines switch a go and let you know how i get on, cheers!

7

no good, i'm afraid - still only got 2 pipelines...

image
image2209×296 25.1 KB

here's the output from the command, i can't see anything obvious in it, but i don't really know what i'm looking for other than something saying error!

PS C:\Program Files\packetbeat> .\packetbeat setup --pipelines -e -d "*"
{"log.level":"info","@timestamp":"2024-10-03T09:12:28.553+0100","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":828},"message":"Home path: [C:\\Program Files\\packetbeat] Config path: [C:\\Program Files\\packetbeat] Data path: [C:\\Program Files\\packetbeat\\data] Logs path: [C:\\Program Files\\packetbeat\\logs]","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:28.553+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).loadMeta","file.name":"instance/beat.go","file.line":950},"message":"Beat metadata path: C:\\Program Files\\packetbeat\\data\\meta.json","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-03T09:12:28.553+0100","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":836},"message":"Beat ID: 5364af0a-7ba5-4d3b-99a4-9277765b4a8a","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:28.573+0100","log.logger":"conditions","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/conditions.NewCondition","file.name":"conditions/conditions.go","file.line":98},"message":"New condition contains: map[]","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:28.574+0100","log.logger":"processors","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/processors.New","file.name":"processors/processor.go","file.line":114},"message":"Generated new processors: drop_fields={\"Fields\":[\"host\"],\"RegexpFields\":[],\"IgnoreMissing\":false}","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:28.591+0100","log.logger":"processors","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/processors.New","file.name":"processors/processor.go","file.line":114},"message":"Generated new processors: add_host_metadata=[netinfo.enabled=[true], cache.ttl=[5m0s]]","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:28.592+0100","log.logger":"docker","log.origin":{"function":"github.com/elastic/elastic-agent-autodiscover/docker.NewClient","file.name":"docker/client.go","file.line":49},"message":"Docker client will negotiate the API version on the first request.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:28.593+0100","log.logger":"add_docker_metadata","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/processors/add_docker_metadata.buildDockerMetadataProcessor","file.name":"add_docker_metadata/add_docker_metadata.go","file.line":89},"message":"add_docker_metadata: docker environment not detected: protocol not available","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:28.592+0100","log.logger":"add_cloud_metadata","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/processors/add_cloud_metadata.(*addCloudMetadata).fetchMetadata","file.name":"add_cloud_metadata/providers.go","file.line":147},"message":"add_cloud_metadata: starting to fetch metadata, timeout=3s","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.594+0100","log.logger":"add_cloud_metadata","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/processors/add_cloud_metadata.(*addCloudMetadata).fetchMetadata","file.name":"add_cloud_metadata/providers.go","file.line":193},"message":"add_cloud_metadata: timed-out waiting for all responses","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.595+0100","log.logger":"add_cloud_metadata","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/processors/add_cloud_metadata.(*addCloudMetadata).fetchMetadata.func1","file.name":"add_cloud_metadata/providers.go","file.line":150},"message":"add_cloud_metadata: fetchMetadata ran for 3.0010304s","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.602+0100","log.logger":"add_cloud_metadata","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/processors/add_cloud_metadata.(*addCloudMetadata).init.func1","file.name":"add_cloud_metadata/add_cloud_metadata.go","file.line":100},"message":"add_cloud_metadata: hosting provider type not detected.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.604+0100","log.logger":"processors","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/processors.New","file.name":"processors/processor.go","file.line":114},"message":"Generated new processors: if contains: map[] then drop_fields={\"Fields\":[\"host\"],\"RegexpFields\":[],\"IgnoreMissing\":false} else add_host_metadata=[netinfo.enabled=[true], cache.ttl=[5m0s]], add_cloud_metadata={}, add_docker_metadata=[match_fields=[] match_pids=[process.pid, process.parent.pid]], detect_mime_type=http.request.body.content->http.request.mime_type, detect_mime_type=http.response.body.content->http.response.mime_type","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.605+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1385},"message":"Beat info","service.name":"packetbeat","system_info":{"beat":{"path":{"config":"C:\\Program Files\\packetbeat","data":"C:\\Program Files\\packetbeat\\data","home":"C:\\Program Files\\packetbeat","logs":"C:\\Program Files\\packetbeat\\logs"},"type":"packetbeat","uuid":"5364af0a-7ba5-4d3b-99a4-9277765b4a8a"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.607+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1394},"message":"Build info","service.name":"packetbeat","system_info":{"build":{"commit":"26daf71e4ec87172523af7f0e916cba9f79dc0d0","libbeat":"8.15.2","time":"2024-09-19T09:33:49.000Z","version":"8.15.2"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.608+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1397},"message":"Go runtime info","service.name":"packetbeat","system_info":{"go":{"os":"windows","arch":"amd64","max_procs":4,"version":"go1.22.6"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.619+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1403},"message":"Host info","service.name":"packetbeat","system_info":{"host":{"architecture":"x86_64","native_architecture":"x86_64","boot_time":"2024-10-02T15:28:19+01:00","name":"redacted-client-name","ip":["fe80::3fcc:dc1f:9e71:a1b3","10.31.4.53","::1","127.0.0.1"],"kernel_version":"10.0.19041.4780 (WinBuild.160101.0800)","mac":["00:50:56:b9:8f:3a"],"os":{"type":"windows","family":"windows","platform":"windows","name":"Windows 10 Enterprise","version":"10.0","major":10,"minor":0,"patch":0,"build":"19045.4780"},"timezone":"BST","timezone_offset_sec":3600,"id":"a8712cd6-1132-43ed-bde6-73484e1b9ecb"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.620+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1432},"message":"Process info","service.name":"packetbeat","system_info":{"process":{"cwd":"C:\\Program Files\\packetbeat","exe":"C:\\Program Files\\packetbeat\\packetbeat.exe","name":"packetbeat.exe","pid":10212,"ppid":2400,"start_time":"2024-10-03T09:12:28.426+0100"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.621+0100","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).createBeater","file.name":"instance/beat.go","file.line":341},"message":"Setup Beat: packetbeat; Version: 8.15.2","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.632+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).createBeater","file.name":"instance/beat.go","file.line":369},"message":"Initializing output plugins","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.648+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.NewConnection","file.name":"eslegclient/connection.go","file.line":133},"message":"elasticsearch url: https://redacted-server-name:9200","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-10-03T09:12:31.649+0100","log.logger":"tls","log.origin":{"function":"github.com/elastic/elastic-agent-libs/transport/tlscommon.(*TLSConfig).ToConfig","file.name":"tlscommon/tls_config.go","file.line":107},"message":"SSL/TLS verifications disabled.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.649+0100","log.logger":"publisher","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*eventConsumer).run","file.name":"pipeline/consumer.go","file.line":110},"message":"start pipeline event consumer","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.649+0100","log.logger":"publisher","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*queueReader).run","file.name":"pipeline/queue_reader.go","file.line":49},"message":"pipeline event consumer queue reader: start","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.650+0100","log.logger":"publisher","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/publisher/pipeline.LoadWithSettings","file.name":"pipeline/module.go","file.line":105},"message":"Beat name: redacted-client-name","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.653+0100","log.logger":"npcap","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/beater.installNpcap.func1","file.name":"beater/install_npcap.go","file.line":56},"message":"npcap version: Npcap version 1.79, based on libpcap version 1.10.4","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.656+0100","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/procs.(*ProcessesWatcher).init","file.name":"procs/procs.go","file.line":114},"message":"Process watcher disabled","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.663+0100","log.logger":"flows","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/flows.newFlowsWorker","file.name":"flows/worker.go","file.line":159},"message":"new flows worker. timeout=30s, period=10s, tick=10s, ticksTO=3, ticksP=1","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.663+0100","log.logger":"main","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/beater.setupSniffer","file.name":"beater/processor.go","file.line":234},"message":"Initializing protocol plugins","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.664+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: sip","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.665+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: mysql","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.665+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: nfs","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.666+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: amqp","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.666+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: dns","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.667+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: pgsql","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.668+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: tls","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.668+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: mongodb","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.669+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: thrift","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.669+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: dhcpv4","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.670+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: cassandra","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.670+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: http","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.671+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: memcache","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.671+0100","log.logger":"protos","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos.ProtocolsStruct.InitFiltered","file.name":"protos/protos.go","file.line":123},"message":"registered protocol plugin: redis","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.675+0100","log.logger":"memcache","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos/memcache.(*memcache).init","file.name":"memcache/memcache.go","file.line":157},"message":"init memcache plugin","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.679+0100","log.logger":"memcache","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos/memcache.(*memcache).setFromConfig","file.name":"memcache/memcache.go","file.line":187},"message":"transaction timeout: 10s","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.680+0100","log.logger":"memcache","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos/memcache.(*memcache).setFromConfig","file.name":"memcache/memcache.go","file.line":188},"message":"udp transaction timeout: 10s","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.683+0100","log.logger":"memcache","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos/memcache.(*memcache).setFromConfig","file.name":"memcache/memcache.go","file.line":189},"message":"maxValues = 0","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.684+0100","log.logger":"memcache","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos/memcache.(*memcache).setFromConfig","file.name":"memcache/memcache.go","file.line":190},"message":"maxBytesPerValue = 2147483647","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.692+0100","log.logger":"mongodb","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos/mongodb.(*mongodbPlugin).init","file.name":"mongodb/mongodb.go","file.line":87},"message":"Init a MongoDB protocol parser","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-10-03T09:12:31.695+0100","log.logger":"cfgwarn","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/protos/sip.New","file.name":"sip/plugin.go","file.line":68},"message":"BETA: packetbeat SIP protocol is used","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.696+0100","log.logger":"sniffer","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/sniffer.New","file.name":"sniffer/sniffer.go","file.line":114},"message":"interface: 0, BPF filter: ''","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.724+0100","log.logger":"sniffer","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/sniffer.New","file.name":"sniffer/sniffer.go","file.line":155},"message":"Sniffer type: pcap device: \\Device\\NPF_{47EB5153-834B-47D0-A8DC-1D51F647258F}","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.724+0100","log.logger":"sniffer","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/sniffer.(*Sniffer).Stop","file.name":"sniffer/sniffer.go","file.line":483},"message":"sending stop to all sniffers","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.728+0100","log.logger":"sniffer","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/sniffer.(*Sniffer).Stop","file.name":"sniffer/sniffer.go","file.line":485},"message":"sending closing to default_route","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.729+0100","log.logger":"flows","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/flows.(*worker).stop","file.name":"flows/worker.go","file.line":77},"message":"stop flows worker","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.729+0100","log.logger":"flows","log.origin":{"function":"github.com/elastic/beats/v7/packetbeat/flows.(*worker).stop","file.name":"flows/worker.go","file.line":80},"message":"stopped flows worker","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.730+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.NewConnection","file.name":"eslegclient/connection.go","file.line":133},"message":"elasticsearch url: https://redacted-server-name:9200","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-10-03T09:12:31.730+0100","log.logger":"tls","log.origin":{"function":"github.com/elastic/elastic-agent-libs/transport/tlscommon.(*TLSConfig).ToConfig","file.name":"tlscommon/tls_config.go","file.line":107},"message":"SSL/TLS verifications disabled.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.731+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Ping","file.name":"eslegclient/connection.go","file.line":302},"message":"ES Ping(url=https://redacted-server-name:9200)","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-10-03T09:12:31.732+0100","log.logger":"tls","log.origin":{"function":"github.com/elastic/elastic-agent-libs/transport/tlscommon.(*TLSConfig).ToConfig","file.name":"tlscommon/tls_config.go","file.line":107},"message":"SSL/TLS verifications disabled.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.753+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/elastic-agent-libs/transport/httpcommon.(*HTTPTransportSettings).RoundTripper.LoggingDialer.func2","file.name":"transport/logging.go","file.line":43},"message":"Completed dialing successfully","service.name":"packetbeat","network":"tcp","address":"redacted-server-name:9200","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.755+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Ping","file.name":"eslegclient/connection.go","file.line":321},"message":"Ping status code: 200","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.757+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Ping","file.name":"eslegclient/connection.go","file.line":322},"message":"Attempting to connect to Elasticsearch version 8.15.1 (default)","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.771+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.774+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"PUT https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  map[description:Pipeline for processing amqp traffic on_failure:[map[append:map[field:error.message value:Processor \"{{ _ingest.on_failure_processor_type }}\" with tag \"{{ _ingest.on_failure_processor_tag }}\" in pipeline \"{{ _ingest.on_failure_pipeline }}\" failed with message \"{{ _ingest.on_failure_message }}\"]] map[set:map[field:event.kind value:pipeline_error]]] processors:[map[set:map[field:ecs.version value:8.11.0]] map[gsub:map[field:host.mac ignore_missing:true pattern:[-:.] replacement: tag:gsubmac]] map[gsub:map[field:host.mac ignore_missing:true pattern:(..)(?!$) replacement:$1- tag:gsubmac]] map[uppercase:map[field:host.mac ignore_missing:true]] map[append:map[allow_duplicates:false field:related.hosts if:ctx.observer?.hostname != null && ctx.observer?.hostname != '' value:{{{observer.hostname}}}]] map[foreach:map[field:observer.ip if:ctx.observer?.ip != null && ctx.observer.ip instanceof List processor:map[append:map[allow_duplicates:false field:related.ip value:{{{_ingest._value}}}]] tag:foreachip]] map[remove:map[field:host if:ctx.host != null && ctx.tags != null && ctx.tags.contains('forwarded')]] map[pipeline:map[if:ctx._conf?.geoip_enrich != null && ctx._conf.geoip_enrich name:{{ IngestPipeline \"geoip\" }} tag:pipelineprocessor]] map[remove:map[field:_conf ignore_missing:true]]]]","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.884+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":135},"message":"Elasticsearch pipeline loaded.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.884+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.890+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"PUT https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  map[description:GeoIP enrichment. on_failure:[map[append:map[field:error.message value:Processor \"{{ _ingest.on_failure_processor_type }}\" with tag \"{{ _ingest.on_failure_processor_tag }}\" in pipeline \"{{ _ingest.on_failure_pipeline }}\" failed with message \"{{ _ingest.on_failure_message }}\"]] map[set:map[field:event.kind value:pipeline_error]]] processors:[map[geoip:map[field:source.ip ignore_missing:true tag:source_geo target_field:source.geo]] map[geoip:map[database_file:GeoLite2-ASN.mmdb field:source.ip ignore_missing:true properties:[asn organization_name] tag:source_geo target_field:source.as]] map[rename:map[field:source.as.asn ignore_missing:true target_field:source.as.number]] map[rename:map[field:source.as.organization_name ignore_missing:true target_field:source.as.organization.name]] map[geoip:map[field:destination.ip ignore_missing:true tag:destination_geo target_field:destination.geo]] map[geoip:map[database_file:GeoLite2-ASN.mmdb field:destination.ip ignore_missing:true properties:[asn organization_name] tag:destination_geo target_field:destination.as]] map[rename:map[field:destination.as.asn ignore_missing:true target_field:destination.as.number]] map[rename:map[field:destination.as.organization_name ignore_missing:true target_field:destination.as.organization.name]] map[geoip:map[field:server.ip ignore_missing:true tag:server_geo target_field:server.geo]] map[geoip:map[database_file:GeoLite2-ASN.mmdb field:server.ip ignore_missing:true properties:[asn organization_name] tag:server_geo target_field:server.as]] map[rename:map[field:server.as.asn ignore_missing:true target_field:server.as.number]] map[rename:map[field:server.as.organization_name ignore_missing:true target_field:server.as.organization.name]] map[geoip:map[field:client.ip ignore_missing:true tag:client_geo target_field:client.geo]] map[geoip:map[database_file:GeoLite2-ASN.mmdb field:client.ip ignore_missing:true properties:[asn organization_name] tag:client_geo target_field:client.as]] map[rename:map[field:client.as.asn ignore_missing:true target_field:client.as.number]] map[rename:map[field:client.as.organization_name ignore_missing:true target_field:client.as.organization.name]]]]","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-03T09:12:31.984+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":135},"message":"Elasticsearch pipeline loaded.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:31.998+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.004+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.004+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.007+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.007+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.010+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.011+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.013+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.019+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.021+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.021+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.024+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.024+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.026+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.026+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.027+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.027+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.034+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.034+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.035+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.036+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.037+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.038+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.039+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.040+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.042+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.042+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.043+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.047+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.050+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.050+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.052+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.052+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.053+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.053+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.055+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.055+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.057+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.057+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.058+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.063+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.065+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.065+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.067+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.067+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.069+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.069+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.070+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.071+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.072+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.073+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.074+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.080+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.082+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.083+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.085+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.085+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.087+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.087+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-default  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.088+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.088+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Request","file.name":"eslegclient/connection.go","file.line":377},"message":"GET https://redacted-server-name:9200/_ingest/pipeline/packetbeat-8.15.2-geoip  <nil>","service.name":"packetbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-10-03T09:12:32.090+0100","log.logger":"pipeline","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.LoadPipeline","file.name":"fileset/pipelines.go","file.line":122},"message":"Pipeline already exists in Elasticsearch.","service.name":"packetbeat","ecs.version":"1.6.0"}
Loaded Ingest pipelines
PS C:\Program Files\packetbeat>
8

OK Weird... I just actually ran this on Windows, and I am seeing the same thing, only 2 pipelines....

Let me look and try to get back

9

@jimB100

So here is what I know...

Something is not right... and it it not you :slight_smile: .. I see the same thing

If you can find a mac or linux box and download packetbeat and run the full setup it will load everything correct.

.. why the windows binary does not I am not sure ...

Then you should be able to run the window packetbeat ...

I did this and it worked...

I put an internal conversation in but not sure when I will hear back

This is data from my windows box after I ran setup elsewhere

Screenshot 2024-10-03 at 3.09.18 PM
Screenshot 2024-10-03 at 3.09.18 PM3220×1474 361 KB

10

@jimB100 Yup bug...

Time being you will need to use my workaround

11

brilliant, thanks @stephenb - appreciate you looking into it

i'll find a linux box to run the setup on

12

packetbeat installed on linux and all working great!

image
image1616×676 77.3 KB

1 Like