DEV: CLI concept for Pandas

CLI concept for Pandas

Inspired by the new SciPy CLI I propose a do.it + rich.click CLI for Pandas.

The goal is to create an one-stop tool for contributors that includes useful developer commands and tools distributed across our documentation.

Maintainers possibly build their own local solutions over time, but for new contributors it takes time to discover and get used to all the functionalities and we can facilitate a smoother ramp up, and in-terminal documentation via help functionality.

I would be happy to keep maintaining the CLI as the functionalities included change over time.

In a series of PRs I can introduce the do.it CLI, and propose that it includes the following functionalities (the proposed workflows are examples for illustration only, what is possible and maintainable will be determined during implementation):

• build and install package on path
• build the docs (full and partial)
  • illustration:

  $ python do.py docs
  Building the docs locally.
  entering direcory /Users/username/pandas/docs
  Running Sphinx v4.5.0
  loading pickled environment... done
  ...
  entering directory /Users/username/pandas
  Building the docs locally completed. Please note the build processes errors and warnings. All generated output files are located in /Users/username/pandas/docs/build.
  

• run the test suite (full and partial, e.g. for specific modules, tests)
  • illustration:

  $ python do.py run-test-suite
  ================================================================================ test session starts ================================================================================
  platform darwin -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
  ...
  
  $ python do.py run-test-module test_multiindex.py
  

  • latter example might need full path. I will assess this during development.
• install pre commit
  • illustration:

  $ python do.py pre-commit
  Installing pre-commit.
  Collecting pre-commit
    Using cached pre_commit-2.19.0-py2.py3-none-any.whl (199 kB)
  ...
  pre-commit installed at .git/hooks/pre-commit
  Installing pre-commit completed. You can now run pre-commit checks. 
  

• run pre commit checks (local or upstream/main)
  • illustration:

  $ python do.py pre-commit --file_name1 --file_name2
  running pre-commit on local files: file_name1, file_name2
  absolufy-imports........................................................................................Passed
  vulture.................................................................................................Passed
  black...................................................................................................Passed
  codespell...............................................................................................Passed
  ...
  
  § python do.py pre-commit --upstream
  > pre-commit run --from-ref=upstream/main --to-ref=HEAD --all-files
  running pre-commit on all upstream-main files
  ...
  

• re-build C extensions
  • illustration:

  $ python do.py c-extesions
  building the C extensions
  running build_ext
  copying build/lib.macosx-11.0-arm64-cpython-38/pandas/_libs/algos.cpython-38-darwin.so -> pandas/_libs
  copying build/lib.macosx-11.0-arm64-cpython-38/pandas/_libs/arrays.cpython-38-darwin.so -> pandas/_libs
  ...
  building the C extensions is complete.
  

Clarifications:

• Is there anything else from the SciPy example you would like to have? opening shells/ipython, creating release notes, running benchmarks?
• Why do.it?
  • I can explore other tools if you would like. I saw that SciPy had a good experience developing with this one and considering the similar requirements, I figured there isn’t a need to spend too much time on tool selection. Also using similar tools as other folks in the echosystem might be usful in the long run if we need similar features, face similar bugs, etc.
• The do.it CLI support colors which will enable us to highlight important takeaways and keywords.

[phofl]

+1 on starting this and gathering some feedback through an initial version. We don’t have to be perfect from the beginning, especially since new contributors will probably be the best people to give us feedback on this

[jorisvandenbossche]

Any more feedback? If not, I assume the best would be for @noatamir to open a first PR with an initial version (with subset of proposed functionality) to have a more concrete feedback on the proposed approach.