How do I transfer data across networks?

[wajika]

I want to transfer the data to different networks

How should I set the url of logstash/filebeat?

filebeat conf

output.redis:
hosts: ["redis.xxx.net:7887"]
key: data
db: 0
timeout: 30

Failed to connect to redis(tcp://redis.xxx.net:7887): dial tcp 1.1.1.1:7887: i/o timeout

logstash conf

output {
kafka {
bootstrap_servers => "kafka.xxx.net:8998"
topic_id => "data"
codec => json
compression_type => "gzip"
}
}

No data received on kafka

[pierhugues (Pier-Hugues Pellerin)]

Hello @wajika,

I am not sure I understand what you are trying to achieve here, I am looking at your diagram, I see Kafka -> Filebeat or Logstash -> nginx -> Redis?

Can you clarify a bit more your problem or use case?

[wajika]

Thanks for your reply. my English is not good.
My problem has been solved. but I found a small problem.
The logstash tcp input mode cannot process multiple rows of data.
For example, the collection of nginx logs, which are not line-by-line transfers.

[wajika]

@pierhugues

I see Kafka -> Filebeat or Logstash -> nginx -> Redis?

Yeah, that's what I wanted (Kafka -> Filebeat or Logstash -> nginx -> Redis?), but now I'm doing it in a different way

[admlko]

If you are receiving multiline messages, you have to change the codec for the input. Default is 'line' (https://www.elastic.co/guide/en/logstash/7.2/plugins-codecs-line.html), which uses '\n' as a delimiter.
Check out other codecs: https://www.elastic.co/guide/en/logstash/7.2/codec-plugins.html

[wajika]

No, the problem is this.
filebeat tcp output >> logstash tcp input .not line-by-line transfers.

[admlko]

What?
You cannot just change the architecture and not describe the issue and expect to get good answers