Include milliseconds in pip log timestamps

What’s the problem this feature will solve?

Currently (pip 19.1.1) when running pip install --log example.log ..., the log messages traced into example.log have timestamps that includes up to the second. For example:

2019-06-09T22:29:08 Created temporary directory: /tmp/user/1000/pip-install-a6hvpvbz
2019-06-09T22:29:08 Requirement already satisfied: requests in ./.venv/lib/python3.7/site-packages (2.22.0)
2019-06-09T22:29:08 Requirement already satisfied: certifi>=2017.4.17 in ./.venv/lib/python3.7/site-packages (from requests) (2019.3.9)

As a result:

1. log records cannot be easily correlated with logs from tools like strace, ltrace, and sysdig since these tools may output several thousand events in a given second
2. delays less than 2 seconds long are harder to identify from the logs alone - this is relevant when running pip many times as part of integration tests for e.g. build backends or dev tools

Describe the solution you’d like

It would be great if pip could include milliseconds in the log timestamps.

Alternative Solutions

Create a ./sitecustomize.py containing

import time
from pip._internal.utils.logging import IndentingFormatter

def formatTime(self, record, _format=None):
    return time.strftime(
        f'%Y-%m-%dT%H:%M:%S,{record.msecs:03.0f} ', self.converter(record.created)
    )

IndentingFormatter.formatTime = formatTime

then execute pip like PYTHONPATH=$PWD pip install --log example.log ..., which results in

2019-06-10T02:36:11,474 Created temporary directory: /tmp/user/1000/pip-install-oc0qwb1k
2019-06-10T02:36:11,476 Requirement already satisfied: requests in ./.venv/lib/python3.7/site-packages (2.22.0)
2019-06-10T02:36:11,486 Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./.venv/lib/python3.7/site-packages (from requests) (3.0.4)

This isn’t really a good approach since it uses pip implementation details.

The actual implementation could be to define default_time_format = '%Y-%m-%dT%H:%M:%S' and default_msec_format = '%s,%03d ' on IndentingFormatter.

Additional context

N/A

[pradyunsg]

formatTime uses time.strftime

Whoops! My bad. Thanks for flagging that.

---

(on mobile)

I don’t know if this is a py2/py3 is an issue here, since formatTime uses time in both versions.

That said, it shouldn’t be difficult to add a formatTime method to our formatter, that does use datetime.

There’s also %(msecs)03d that can be added after the time when formatting, instead of overriding formatTime.

Basically, if we want to, this is definitely do-able. Almost any approach we take likely work IMO so let’s defer discussion until we see a PR for this functionality.

[cjerdonek]

If this is going to make things messy, do we really need for this feature to be available in 2.7?