Hi
i have a packetbeat flow index with dest.ip :x.x.x.x which is working fine. What i need to do now is if it is possible to search based on CIDR notation instead of individual ips . Like if i want to find how much byte to and fro a particular CIDR address for a given time .
Really appreciate your kind help in advacne
Specifically for CIDR notation you would use a term query. This requires that the field is indexed as the type ip. Other options to query for ranges of IP addresses are the query_string and range queries. Below are examples of each. The field used in the examples is from ElastiFlow but you should get the idea of how they work:
term:
{
"query": {
"term": {
"flow.dst_addr": "192.168.12.0/24"
}
}
}
query_string:
{
"query": {
"query_string": {
"query": "flow.dst_addr: \"192.168.0.0/24\""
}
}
}
{
"query": {
"query_string": {
"query": "flow.dst_addr:[192.168.12.1 TO 192.168.12.255]"
}
}
}
range:
{
"query": {
"range": {
"flow.dst_addr": {
"gte": "192.168.12.1",
"lte": "192.168.12.255"
}
}
}
}
Rob
Robert Cowart (rob@koiossian.com)
www.koiossian.com
True Turnkey SOLUTIONS for the Elastic Stack
Hi Rob sorry for the silly question i am aware that this code need to b execute somewhere . Please note i have been use elk stack for quite sometime but i an yet familiar with it throughly. So really i don't understand where this code should be and also is this allow me to do search on the Kibana search field . i hope you understand
The search bar uses Lucene syntax by default, which is the syntax for a query_string query. So put this in the search bar:
dest.ip: "192.168.1.0/24"
NOTE: The quotes around the CIDR subnet are important. It won't work without them.
thanks Rob ... now i can see a result ... but only when i used individual ips not the CIDR notaiotn like dest.ip"8.8.0.0/16"
You are missing a colon: dest.ip"8.8.0.0/16"should be dest.ip: "8.8.0.0/16"
Hi Rob
This is what is shown
this is the success search
thanks Rob
However i can do this ... i think the maximum is a /24
dest.ip:[192.168.2.1 TO 192.168.2.255]
You should look at the index pattern and ensure that dest.ip was indexed as an IP and not a string. Here is an example from my system that shows this working:
All data:
10.0.0.0/8
192.168.3.0/24
Rob
Robert Cowart (rob@koiossian.com)
www.koiossian.com
True Turnkey SOLUTIONS for the Elastic Stack
Hi Rob
I believe it is index as required as this is what i have similar to your but the only difference is i cannot use the CIDR
Notice how your you have a t next to dest-ip in your screenshot. This indicates that the field is indexed as a string (or text).
Look at mine...
The little PC icon indicates it is indexed as an IP address.
Go to Management --> Index Patterns and find the field. It must be an IP, like this...
I expect you will see a string. As I mentioned earlier the field MUST be indexed as the type ip for CIDR queries to work. The range query only looked like it was working because it was basically treating it with alphabetical sorting.
I just took a look at the packetbeat 6.2.4 index template and it does in fact type dest.ip as a keyword.
"dest": {
"properties": {
"ip": {
"ignore_above": 1024,
"type": "keyword"
},
I didn't think about this initially. We don't use any beats data in its out-of-the-box form (data model is just too whacky), and when we normalize it to our own schema we handle the data types properly.
I recommend that you open as issue for this in the Beats repository on GitHub.
any suggestion what i should do with this .....actually this is a result from packetbeat
Like I said...
They need to fix this in Beats.
thanks
what are you using ??? syslog + logstash ???
i try to change this but not work either
Our solutions take in a number of log and network flow sources, normalize this data to common schemas, and provide a suite of visualizations, dashboards, alerts and anomaly detection methods that work seamless across all data sources.
We are primarily focused on security and network flow analytics. However we have also helped our customers in the other areas, such as performance management, and even IoT/Smart Cities projects.
We currently send Packetbeat data to Logstash, where it is transformed to meet the requirements of our customers. In the future this function will likely be moved into Kafka Streams jobs (faster and more scalable).
Most of our customers understand the potential of what can be achieved with the Elastic Stack. However they lack the time required to learn all of the little (sometimes undocumented) details needed to build it themselves. We typically deliver in just a couple days, what it would take them months to do on their own.
We also contribute to the community with projects for basic network flow and syslog collection.
NOTE: if you change the index template, making dest.ip (and other IP addresses) an ip type, you will have to wait until a new index is created. Index templates only have effect at index creation.
You also need to prevent Packetbeat from overwriting your template, as it will by default.
https://www.elastic.co/guide/en/beats/packetbeat/master/packetbeat-template.html
so you suggest i delete the index and recreate
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.