dynamically added value to addopts with pytest_configure is not reflecting in cmd line

Objective:

Adding -n [var_num] to addopts for my pytest.ini file wherein the var_num is dynamically taken accordingly from command line for pytest-xdist run.

python3.6, pytest==4.6.3, pytest-xdist==1.28.0

My project struct:

Python-package
    - pytest.ini
    - setup.py
    - src (py package)
         - conftest.py
         - config.py
         - tests
             - test_tc1.py
             - test_tc2.py

pytest.ini

[pytest]
addopts = -v -s -p no:warnings
filterwarnings =
    ignore::UserWarning

conftest.py

def pytest_configure(config):
    config.addinivalue_line("markers", "env(name): mark test to run only on named environment")
    count = len(config.getoption('--env'))
    print('count: : ', count)
    args = "-n" + str(count)
    print('args is: ', args)
    config.addinivalue_line("addopts", args)
    print(config.getini("addopts"))

output is

(venv) [root@localhost tests]$ pytest test_tc1.py --env ENV1 --env ENV2 --env ENV3
count: :  3
args is:  -n3
['-v', '-s', '-p', 'no:warnings', '-n3']
===================================================================== test session starts ======================================================================
platform linux -- Python 3.6.3, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- [ROOT_DIR]/venv/bin/python3.6
cachedir: .pytest_cache
rootdir: [ROOT_DIR],
inifile: pytest.ini
plugins: forked-1.0.2, rerunfailures-7.0, xdist-1.28.0
collecting ... [shortened data]
collected 3 items                                                                                                                                              

test_tc1.py::TestClass1::test_tc1[ENV1] 
[Shortened Data]
PASSED[Shortened Data]

test_tc1.py::TestClass1::test_tc1[ENV2] 
[Shortened Data]
PASSED[Shortened Data]

test_tc1.py::TestClass1::test_tc1[ENV3] 
[Shortened Data]
PASSED[Shortened Data]


================================================================== 3 passed in 10.49 seconds ===================================================================

Expected Output

Expectation was should be same as when I parse the -n flag to pytest.ini

pytest.ini

[pytest]
addopts = -v -s -p no:warnings -n3
filterwarnings =
    ignore::UserWarning

conftest.py

def pytest_configure(config):
    config.addinivalue_line("markers", "env(name): mark test to run only on named environment")
    print(config.getini("addopts"))

Result is shown as below

(venv) root@localhost tests]$ pytest test_tc1.py --env ENV1 --env ENV2 --env ENV3
['-v', '-s', '-p', 'no:warnings', '-n3']
===================================================================== test session starts ======================================================================
platform linux -- Python 3.6.3, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- [ROOT_DIR]/venv/bin/python3.6
cachedir: .pytest_cache
rootdir: [ROOT_DIR], inifile: pytest.ini
plugins: forked-1.0.2, rerunfailures-7.0, xdist-1.28.0
[gw0] linux Python 3.6.3 cwd: [TEST_DIR]/tests
[gw1] linux Python 3.6.3 cwd: [TEST_DIR]/tests
[gw2] linux Python 3.6.3 cwd: [TEST_DIR]/tests
[gw0] Python 3.6.3 (default, Jun 25 2019, 18:53:50)  -- [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
[gw1] Python 3.6.3 (default, Jun 25 2019, 18:53:50)  -- [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
[gw2] Python 3.6.3 (default, Jun 25 2019, 18:53:50)  -- [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
gw0 ok / gw1 ok / gw2 ok/[Shortened Data]
gw0 [3] / gw1 [3] / gw2 [3]
scheduling tests via LoadScheduling

test_tc1.py::TestClass1::test_tc1[ENV3] 
test_tc1.py::TestClass1::test_tc1[ENV2] 
test_tc1.py::TestClass1::test_tc1[ENV1] 
[gw0] PASSED  test_tc1.py::TestClass1::test_tc1[ENV1] 
[gw2] PASSED  test_tc1.py::TestClass1::test_tc1[ENV3] 
[gw1] PASSED  test_tc1.py::TestClass1::test_tc1[ENV2] 

=================================================================== 3 passed in 6.05 seconds ===================================================================

As you see even if both the pytest.ini addopts having the same list of data, the resulting behaviour is different. The first attempt is not even considering the flag -n. What is the reason for this ? How to get this fixed ?

EDIT

If I add both the pytest.ini and conftest.py files; the list data is

['-v', '-s', '-p', 'no:warnings', '-n3', '-n3']

[nicoddemus]

Thanks! OK, now things are starting to click together for me. 👍

It does not seem to be possible to actually change the command-line arguments via a plugin, because it is processed very early in the initialization process.

You can however change the option directly in a hook:

# conftest.py
def pytest_cmdline_preparse(config, args):
    config.option.numprocesses = 2

pytest-xdist will check config.option.numprocesses during pytest_cmdline_main:

https://github.com/pytest-dev/pytest-xdist/blob/9f9b70748115cf0da7ae06aa1fb1127b6ba757a9/src/xdist/plugin.py#L166-L175

[marcobazzani]

Thanks! OK, now things are starting to click together for me. 👍

It does not seem to be possible to actually change the command-line arguments via a plugin, because it is processed very early in the initialization process.

You can however change the option directly in a hook:

# conftest.py
def pytest_cmdline_preparse(config, args):
    config.option.numprocesses = 2

pytest-xdist will check config.option.numprocesses during pytest_cmdline_main:

https://github.com/pytest-dev/pytest-xdist/blob/9f9b70748115cf0da7ae06aa1fb1127b6ba757a9/src/xdist/plugin.py#L166-L175

you saved my day works on xdist too