Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Increase Per-File Backend Test Coverage

See original GitHub issue

Background

Currently, we have 100% test coverage of our backend code. However, for many backend code files, this coverage is partly from the file’s associated test files and partly from test files for other backend code. For example, consider the following files:

util.py

def parse_int(string_input):
    return int(string_input)

def int_to_str(number):
    return str(number)

util_test.py: Tests for parse_int()
main.py
```
def main():
  return int_to_str(5)
```
main_test.py: Tests for main()

Here we have 100% code coverage even though util_test.py only tests one of the functions in util.py because the other function is used by main.py, so it is tested indirectly.

Motivation

Indirect test coverage tends to be lower-quality because it’s covered accidentally rather than by a deliberate test.
Once we get full per-file backend test coverage, we can make the pre-push hook run backend tests for just the files that have been changed. This will let developers catch bugs earlier and speed up CI runs (since the load on CI will be lower).

Goal

The goal of this issue is to write tests so that every backend code file is fully covered by its associated test file. In other words, we want util_test.py to fully cover util.py. We call this “100% per-file coverage.” To meet this goal, we need to write tests for code that is currently tested indirectly (in the example above, we need to write tests for int_to_str() in util_test.py).

Coverage Tests

Our backend test runner (scripts/run_backend_tests.py) currently checks per-file coverage and will fail if that coverage is not 100% for any file associated with a test file not in scripts/backend_tests_incomplete_coverage.txt. There are a few important implications of this system:

The coverage checks can pass even if a backend code file has no associated test file at all. This is because adding entirely new test files is beyond the scope of the current work.
scripts/backend_tests_incomplete_coverage.txt functions as a list of test files that need more tests but are currently being ignored.

Procedure

To help us achieve 100% per-file backend test coverage, you can write tests in one of the files listed in scripts/backend_tests_incomplete_coverage.txt (copied as a task list below). Here’s how:

Assign yourself to one of the tasks by putting your username at the end. For example:
```
* [ ] core.controllers.base_test @your-username
```
If you don’t have permission to assign yourself, leave a comment mentioning @U8NWXD and asking to be assigned.
On your local computer, remove your assigned test module from scripts/backend_tests_incomplete_coverage.txt.

Run the backend tests. To speed up the tests, you can run only your test module. For example, if you were assigned scripts.docstrings_checker_test, you could run:

$ python -m scripts.run_backend_tests --test_target scripts.docstrings_checker_test --generate_coverage_report
...
INCOMPLETE COVERAGE (69.0%): scripts.docstrings_checker_test
Name                            Stmts   Miss  Cover   Missing
-------------------------------------------------------------
scripts/docstrings_checker.py     109     34    69%   61-67, 80-91, 104-109, 125-127, 133-142, 173-178
-------------------------------------------------------------
TOTAL                             109     34    69%
...

Notice that the output identifies the lines of the code file being tested that need to be covered. In this example, we need to write tests for lines 61-67, lines 80-91, etc.

Write tests! For help writing backend tests, see our backend tests wiki page. You may also find our debugging advice for backend tests helpful.
Once the coverage checks pass when you run your assigned test, you can make a pull request.

How to Request a Task

To request a task to work on, leave a comment specifying which available task you want to claim. @U8NWXD will assign you by putting your username next to the task in the list below.

Note: If you are working on writing tests for multiple files at the same time, please combine them into a single PR to ease the burden on our testing infrastructure and reviewers.

Note that some files may appear twice because some files that had full line coverage were added again once we started tracking branch coverage.

Tasks

Available

core.controllers.acl_decorators_test
core.controllers.base_test
core.controllers.contributor_dashboard_test
core.controllers.creator_dashboard_test
core.controllers.domain_objects_validator_test
core.controllers.editor_test
core.controllers.feedback_test
core.controllers.learner_playlist_test
core.controllers.library_test
core.controllers.payload_validator_test
core.controllers.practice_sessions_test
core.controllers.profile_test
core.controllers.questions_list_test
core.controllers.reader_test
core.controllers.skill_editor_test
core.controllers.story_editor_test
core.controllers.subtopic_viewer_test
core.controllers.suggestion_test
core.controllers.topic_editor_test
core.controllers.topics_and_skills_dashboard_test
core.domain.app_feedback_report_domain_test
core.domain.app_feedback_report_services_test
core.domain.blog_services_test
core.domain.calculation_registry_test
core.domain.classifier_services_test
core.domain.collection_domain_test
core.domain.collection_services_test
core.domain.config_domain_test
core.domain.draft_upgrade_services_test
core.domain.email_manager_test
core.domain.exp_domain_test
core.domain.exp_services_test
core.domain.feedback_services_test
core.domain.fs_services_test
core.domain.html_cleaner_test
core.domain.html_validation_service_test
core.domain.image_validation_services_test
core.domain.interaction_registry_test
core.domain.learner_goals_services_test
core.domain.learner_group_services_test
core.domain.learner_playlist_services_test
core.domain.learner_progress_services_test
core.domain.opportunity_services_test
core.domain.platform_parameter_domain_test
core.domain.platform_parameter_registry_test
core.domain.question_domain_test
core.domain.question_services_test
core.domain.rules_registry_test
core.domain.search_services_test
core.domain.skill_domain_test
core.domain.skill_services_test
core.domain.stats_domain_test
core.domain.story_fetchers_test
core.domain.subtopic_page_domain_test
core.domain.suggestion_registry_test
core.domain.summary_services_test
core.domain.topic_services_test
core.domain.translation_domain_test
core.domain.user_domain_test
core.domain.user_query_services_test
core.domain.user_services_test
core.domain.visualization_registry_test
core.domain.wipeout_domain_test
core.domain.wipeout_service_test
core.jobs.transforms.validation.base_validation_test
core.jobs.types.base_validation_errors_test
core.platform.auth.firebase_auth_services_test
core.platform.bulk_email.mailchimp_bulk_email_services_test
core.platform.email.dev_mode_email_services_test
core.platform.taskqueue.cloud_taskqueue_services_test
core.platform.taskqueue.cloud_tasks_emulator_test
core.schema_utils_test
core.storage.base_model.gae_models_test
core.storage.collection.gae_models_test
core.storage.exploration.gae_models_test
core.storage.question.gae_models_test
core.storage.topic.gae_models_test
core.utils_test
main_test
scripts.build_test
scripts.common_test
scripts.concurrent_task_utils_test
scripts.install_python_prod_dependencies_test
scripts.install_third_party_test
scripts.linters.html_linter_test
scripts.linters.js_ts_linter_test
scripts.linters.other_files_linter_test
scripts.linters.pylint_extensions_test
scripts.linters.pre_commit_linter_test
scripts.pre_push_hook_test
scripts.run_e2e_tests_test
scripts.run_frontend_tests_test
core.domain.rights_manager_test

Assigned

core.domain.action_registry_test @IamEzio #16406
core.domain.beam_job_services_test @IamEzio #16406
core.domain.event_services_test @IamEzio #16406
core.domain.learner_group_fetchers_test @IamEzio #16406
core.domain.playthrough_issue_registry_test @IamEzio #16471
core.domain.recommendations_services_test @IamEzio #16471
core.domain.rights_domain_test @IamEzio #16471
core.domain.rte_component_registry_test @IamEzio #16471
scripts.docstrings_checker_test @emil-bryniel #16723 #16724
core.domain.change_domain_test @gnurris
core.domain.auth_services_test @cyc0825 #16664 #16670 #16691
core.domain.blog_domain_test @Filip-Joh
core.domain.state_domain_test @ellahathaway #16695
core.domain.stats_services_test @agizoni
core.domain.story_domain_test @Kaicheng-Han #16681
core.domain.feedback_domain_test @anudeeps0306
core.controllers.blog_homepage_test @sukritsg #16708
core.domain.user_query_domain_test @kashishbhandula #16689 #16690
core.controllers.voice_artist_test @adamhalim (assigned 2022-12-12)
core.domain.story_services_test @sukritsg #16707

Complete

core.controllers.base_test @ayushjaink8 #14495
core.controllers.editor_test @ayushjaink8 #14594
core.controllers.resources_test @ananth243 #14541
extensions.rich_text_components.components_test @IamEzio #14528
core.controllers.profile_test @ananth243 #14529
core.controllers.story_editor_test @IamEzio #14641
core.controllers.questions_list_test @IamEzio #14538
core.controllers.library_test @IamEzio #14545
core.controllers.skill_editor_test @IamEzio #14641
core.platform.storage.cloud_storage_emulator_test @IamEzio #14548
core.domain.activity_domain_test @IamEzio #14558
core.domain.question_domain_test @ayushjaink8 #14759
core.domain.story_fetchers_test @ananth243 #14632
core.domain.rights_domain_test @IamEzio #14578
core.domain.param_domain_test @IamEzio #14581
core.domain.learner_goals_services_test @IamEzio #14582
scripts.common_test @U8NWXD
core.domain.event_services_test @ayushjaink8 #14682
core.domain.recommendations_services_test @IamEzio #14582
core.storage.config.gae_models_test @IamEzio #14582
core.storage.question.gae_models_test @IamEzio #14662
core.domain.taskqueue_services_test @paprajapati9 #14564
core.domain.html_validation_service_test @IamEzio #14589
core.domain.stats_domain_test @IamEzio #14582
core.domain.email_manager_test @IamEzio #14601
core.domain.action_registry_test @IamEzio #14601
core.domain.config_domain_test @IamEzio #14601
core.storage.auth.gae_models_test @Manan-Rathi #14665
core.jobs.job_utils_test @IamEzio #14662
core.storage.improvements.gae_models_test @Manan-Rathi #14580
core.storage.story.gae_models_test @Manan-Rathi #14586
core.storage.audit.gae_models_test @Manan-Rathi #14584
core.storage.classifier.gae_models_test @Manan-Rathi #14573
core.storage.blog.gae_models_test @Manan-Rathi #14585
core.storage.recommendations.gae_models_test @Manan-Rathi #14610
core.storage.subtopic.gae_models_test @Manan-Rathi #14610
core.storage.opportunity.gae_models_test @IamEzio #14641
core.storage.statistics.gae_models_test @IamEzio #14641
core.domain.feedback_services_test @Hatilar420 #14756
core.controllers.blog_homepage_test @tofu2323 #14834
core.domain.blog_services_test @Hatilar420 #14854
core.domain.summary_services_test @Hatilar420 #14890
core.domain.topic_services_test @Hatilar420 #14891
core.domain.exp_fetchers_test @Hatilar420 #14883
core.controllers.acl_decorators_test @ayushjaink8 #14892
core.domain.classifier_services_test @JayVivarekar #14803 #14812
core.domain.story_services_test @Hatilar420 #14942
core.domain.suggestion_services_test @ananth243 #14920
core.domain.skill_domain_test @ananth243 #14920
core.domain.topic_fetchers_test @Hatilar420 #14981
core.domain.classifier_domain_test @CohenAriel #14992 #14996
core.domain.playthrough_issue_registry_test @ashish-patwal #15026
core.domain.question_fetchers_test @Mohitbalwani26 #15040
core.domain.exp_services_test @Hatilar420 #14816
core.domain.skill_services_test @camperjett #14801
core.jobs.job_test_utils_test @shanjng #15272
scripts.install_third_party_libs_test @LevBernstein #15290
core.domain.interaction_registry_test @kevcch #15339
core.storage.user.gae_models_test @swethasekhar #15340 #15393
core.domain.role_services_test @kevcch #15384
core.constants_test @aashish-khub #15292
core.utils_test @Darsuu #15017 #15055 #15161
core.domain.user_services_test @snagas22 #15356
core.controllers.tasks_test @SawaTszm #15566
core.domain.value_generators_domain_test @jonlake5 #16198
core.domain.topic_domain_test @jonlake5 #16304
core.domain.param_domain_test @kashishbhandula https://github.com/oppia/oppia/pull/16659
core.domain.learner_group_domain_test @MattiasErlingson #16654