Doing geoip on more than one address...

Here’s another topic for discussion!

We have events with more than one IP and want to do geoip on all of them. Instead of being “top level” field, what about source.geoip and such? Attach the geoip block to the thing it belongs to.

[robcowart]

Why even include the term “geoip”? This makes an assumption about the source of the data filling this field. While it could come from a GeoIP DB like Maxmind, it could just as easily come from an inventory system or other source (like a Logstash translate filter). Don’t forget GeoIP DBs only include public IP addresses. However there are plenty of users interested in populating such fields for private IP addresses as well. For this reason, I would recommend something as simple as: source.city, source.country, etc.

What is also missing here is the fact that the source and destination are actually attributes of a higher level object… a network connection. In our solution - which is already proven by our dozens of out-of-the-box integrations, upon which we provide a single fully-integrated Kibana, ML and Alerting experience across all data sources, and which is used successfully in production at dozens of customers - these fields are named conn.src_city, conn.src_country, etc. This same method is also used in ElastiFlow, albeit with a simplified schema only for flow data. ElastiFlow is deployed successfully by 1000+ users, so I feel pretty confident that this is the way to go.

Since nesting was discussed in this thread I will say that i.would.be.wary.of.the.overuse.of.nested.fields, which I see in Beats and ECS. It feels like some kind of containment model is being forced onto the data when there is nothing in the stack that can actually take advantage of it. Consider this… as a user, would you rather work with a field called system.cpu.total.norm.pct or simply cpu.util? Our experience, which includes solutions other than just Elastic, the preference would clearly be for the latter. In my opinion ECS should not be leaning on the ideas from Beats, rather ECS should be designed to apply to ANY data source, and Beats should be modified to support it. While there could be potential use-cases for some degree of nesting. There would have to be features added in other components to take advantage of it (if this is planned, let’s hear about it). Even then the nesting is arguable still excessive in some places.

[ruflin]

The idea of the geoip fields is that they are reusable at every level. The same applies for example also for user or url fields. On the url fields it is mentioned on how it could be used: https://github.com/elastic/ecs#url

We should update the geoip description to make it clear that these are “reusable”. So the structure I would suggest for your use case would be source.geoip.* and destination.geoip.*. Does that make sense?