Add HTTP User-Agent header to API calls

According to the MediaWiki API docs, there should be an HTTP User-Agent header added to API calls. We should follow the policy by adding a user agent to our HTTP GET calls.

Probably something like the following should do it (reference):

import wikipron
import requests


HTTP_HEADERS = {
    "User-Agent": (
        f"WikiPron/{wikipron.__version__} "
        "(https://github.com/kylebgorman/wikipron) "
        f"requests/{requests.__version__}"
    ),
}

requests.get(..., headers=HTTP_HEADERS)

I think there are only two places in the codebase where a requests.get call (or equivalent) is made:

https://github.com/kylebgorman/wikipron/blob/fab40759d4fc95579769f44a9decf1c087771e75/wikipron/scrape.py#L44

https://github.com/kylebgorman/wikipron/blob/fab40759d4fc95579769f44a9decf1c087771e75/wikipron/scrape.py#L63

[lfashby]

There are three GET calls to Wiktionary in data/src/codes.py: https://github.com/kylebgorman/wikipron/blob/ac821c5d0a7d70e7e700f45f9d01b2dfb4ecae9d/data/src/codes.py#L61 https://github.com/kylebgorman/wikipron/blob/ac821c5d0a7d70e7e700f45f9d01b2dfb4ecae9d/data/src/codes.py#L86 https://github.com/kylebgorman/wikipron/blob/ac821c5d0a7d70e7e700f45f9d01b2dfb4ecae9d/data/src/codes.py#L110 Do we need a User-Agent header for these as well?

[jacksonllee]

Hi @Alireza-Sampour , thank you for your interest in contributing! I just found two more spots – I forgot about tests/:

https://github.com/kylebgorman/wikipron/blob/fab40759d4fc95579769f44a9decf1c087771e75/tests/test_wikipron/test_config.py#L193

https://github.com/kylebgorman/wikipron/blob/fab40759d4fc95579769f44a9decf1c087771e75/tests/test_wikipron/__init__.py#L16

We can probably define the HTTP_HEADERS dict just once, in wikipron/srape.py. Then we can use it directly for the first and second locations I’ve mentioned above. For the third case in tests/test_wikipron/test_config.py, HTTP_HEADERS could be imported through this line. For the fourth case in tests/test_wikipron/__init__.py, I believe we could do from wikipron.scrape import HTTP_HEADERS just before this line – I was thinking through and trying to see if we’d run into surprises like circular imports, ending up typing all this out…

Here are the notes for contributors.