SIP package

Folks,

may I propose including a SIP package (RFC3261). I suggest it would include the fields as bellow, modelled after the HTTP package. What’s different:

• SIP requests have many more URLs in them. Therefore the suggestions Use the url field set to store the url of the request. only applies to the request-URI which represents the URI in the request as received (SIP proxy elements may also alter it during forwarding). For all the other URIs, I included only the unparsed form full.
• other fields that offer themselves for use in SIP events are event.duration for reporting SIP call length, user_agent.* for SIP User Agent (example: AVM FRITZ!Box Fon WLAN 7270 v3 74.06.05 (Apr 9 2014), host.name for receiving SIP server’s DNS name, host.id for SIP server’s unique identification, host.ip for receiving server’s IP address, client.ip and client.port for client’s IP:port, user.id for digest authentication identity, client.geo.* for client’s geographic info.
• client.transport would be useful but doesn’t exist (#418 )
• event.action seems the proper candidate for values describing the actual nature of an event like call-attempt(failed INVITE request), call-start (succesful INVITE request), call-stop (received or initated BYE), register-new for a newly added contact binding, register-del for removed contact binding, register-expired for one that has expired.
• in SIP, response elements often mirror their request counterparts. Where identical for requests and response, I didn’t prefix the field name with request/response – I hope that’s the proper use of the ECS conventions.

There are couple of fields which would be helpful too but I don’t see them as SIP specific – see the list bellow, merely for sake of context, if there is interest I can create separate issues for them.

• session id. It is often useful to correlate some events belonging to each other – for example a SIP call’s id may be used to correlate events describing start and stop of a VoIP call, or a shopping session id may be used to correlate additions to and removals from a shopping basket.
• for use with SIP proxy elements it would be useful to describe in events how such a proxy element describes changes to forwarded requests: most importantly how it is changing the URI, where it is forwarding a request to, and if a response was generated locally or proxied. However this doesn’t seem SIP-specific, I think the same data would be useful for example for HTTP proxy servers like nginx or ha-proxy ,
• some topological information. Often, especially in multi-tenant environments, flows are correlated with their actual owners – that helps in many aggregations. I havent’ exactly captured the difference between source and client packages but I would say that for sake of tenant analysis a more “logical” view would help. For example it could use UUID to describe traffic from multiple parts of an organization (liberland’s embassy to the US, liberland’s embassy to Australia, etc)

Suggestions / clarification requests welcome.

  - name: sip
    title: SIP
    group: 2
    description: >
      Fields related to SIP activity.   Use the `url` field to store SIP request URI.
    type: group
    fields: 

   # HTTP-like SIP fields

    - name: method
      level: extended
      type: keyword
      short: SIP  request/response method.
      description: >
        SIP request/response method.

        The field value must be normalized to uppercase for querying.
      example: INVITE, BYE, REGISTER, SUBSCRIBE, NOTIFY

    - name: request.body.content
      level: extended
      type: keyword
      description: >
        The full SIP request body, typically SDP.
      example: See RFC3264 for SIP/SDP examples.

    - name: response.status_code
      level: extended
      type: long
      description: >
        SIP response status code.
      example: 404

    - name: response.body.content
      level: extended
      type: keyword
      description: >
        The full SIP  response body.
      example: See RFC3264 for SIP/SDP examples.

   # SIP-specific fields

    - name: call_id
      level: extended
      type: keyword
      short: SIP Call ID
      description: >
        SIP call id is used to relate dialog-initiating and in-dialog SIP transactions with each other.
        The value is case-sensitive. 
      example: 11e6585d-6017-4f83-a95b-4861d47fe678

    - name: request.contact
      level: extended
      type: keyword
      short: Full unparsed Contact URL.
      description: >
        This is the full Contact URI most often found in SIP REGISTER and INVITE requests.
         Note that it is NOT a full Contact header value which may additionally contain header field
         parameters, such as expires=3600.
      example: sip:gw+d15d5fa6-6868-4b45-84f9-851db9d54ca3@212.83.136.53:5080;transport=udp;gw=d15d5fa6-6868-4b45-84f9-851db9d54ca3

    - name: from
      level: extended
      type: keyword
      short: Full unparsed From  URL.
      description: >
        This is the full From URI mandatory for any SIP requests. 
         Note that it is NOT a full From header value which may additionally contain header field
         parameters, such as tag=tpc6D8ce0NccD.
      example: sips:[e-mail]

    - name: request.to
      level: extended
      type: keyword
      short: Full unparsed To  URL.
      description: >
        This is the full To URI mandatory for any SIP requests. 
         Note that it is NOT a full To header value which may additionally contain header field
         parameters, such as tag=tpc6D8ce0NccD.
         Also note that the RFC3261 permits the To URI to be different in response.
      example: sips:[e-mail]

    - name: response.to
      level: extended
      type: keyword
      short: Full unparsed To  URL.
      description: >
        This is the full To URI mandatory for any SIP response. 
         Note that it is NOT a full To header value which may additionally contain header field
         parameters, such as tag=tpc6D8ce0NccD.
         Also note that the RFC3261 permits the To URI to be different than in the initiating request.
      example: sips:[e-mail]

    - name: response.reason_phrase
      level: extended
      type: text
      short: human-readable form of status code
      description: >
         human-readable form of SIP status code, see RFC3261, section 7.2

    # Metrics
    - name: request.bytes
      level: extended
      type: long
      format: bytes
      description: >
        Total size in bytes of the request (body and headers).
      example: 1437

    - name: request.body.bytes
      level: extended
      type: long
      format: bytes
      description: >
        Size in bytes of the request body.
      example: 887

    - name: response.bytes
      level: extended
      type: long
      format: bytes
      description: >
        Total size in bytes of the response (body and headers).
      example: 1437

    - name: response.body.bytes
      level: extended
      type: long
      format: bytes
      description: >
        Size in bytes of the response body.
      example: 887

[dainperkins]

@ebeahan I do, will take a look.

[dainperkins]

I went thru a request and 2 responses and came up with what seems like will cover a good bit, there’s a couple that might be optional (normalizing uri’s & contacts for instance, nonce, and opaque values for auth requests). I didn’t really try to match field names, tho the ones that overlap (other than not prefixing each request or response header field and allowing the top level identifier to provide the separation - e.g. to/from/contact - its all more or less the same fields, and easy enough to search for sip.method = register && sip.from.user = John.Doe instead of specifying sip.request.from or sip.response.from…

Sample Packet Capture & ECS implementation in the sheet in this directory: https://drive.google.com/drive/folders/1yebz9dGyUiQGYOzVSi_iCdE4NuT8ZkzK?usp=sharing

I suspect logs will require significantly more analysis / intelligence in terms of integration, but it should work as well.

Will add a bunch of SIP responses (register failure for auth, proxy auth for invite, etc.) as well as start refining the original invites tonight or tomorrow.