question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

CI fail on two tests

See original GitHub issue

The CI keeps failing on this test, I’m not sure why.

Example run: https://github.com/huggingface/huggingface_hub/runs/5628640361?check_suite_focus=true

Ping @LysandreJik @osanseviero


_______________________ HfApiPublicTest.test_model_info ________________________

self = <tests.test_hf_api.HfApiPublicTest testMethod=test_model_info>

    @with_production_testing
Downloading: 100%|██████████| 2.00/2.00 [00:00<00:00, 1.42kB/s]
    def test_model_info(self):
        _api = HfApi()
        model = _api.model_info(repo_id=DUMMY_MODEL_ID)
        self.assertIsInstance(model, ModelInfo)
        self.assertNotEqual(model.sha, DUMMY_MODEL_ID_REVISION_ONE_SPECIFIC_COMMIT)
        # One particular commit (not the top of `main`)
        model = _api.model_info(
            repo_id=DUMMY_MODEL_ID, revision=DUMMY_MODEL_ID_REVISION_ONE_SPECIFIC_COMMIT
        )
        self.assertIsInstance(model, ModelInfo)
        self.assertEqual(model.sha, DUMMY_MODEL_ID_REVISION_ONE_SPECIFIC_COMMIT)
        model = _api.model_info(
            repo_id=DUMMY_MODEL_ID,
            revision=DUMMY_MODEL_ID_REVISION_ONE_SPECIFIC_COMMIT,
            securityStatus=True,
        )
        self.assertEqual(
>           getattr(model, "securityStatus"),
            {"containsInfected": False, "infectionTypes": []},
        )
E       AttributeError: 'ModelInfo' object has no attribute 'securityStatus'

tests/test_hf_api.py:725: AttributeError
__________________ InferenceApiTest.test_inference_with_audio __________________

self = <tests.test_inference_api.InferenceApiTest testMethod=test_inference_with_audio>

    @with_production_testing
    def test_inference_with_audio(self):
        api = InferenceApi("facebook/wav2vec2-large-960h-lv60-self")
        dataset = datasets.load_dataset(
            "patrickvonplaten/librispeech_asr_dummy", "clean", split="validation"
        )
        data = self.read(dataset["file"][0])
        result = api(data=data)
        self.assertIsInstance(result, dict)
>       self.assertTrue("text" in result)
E       AssertionError: False is not true

tests/test_inference_api.py:73: AssertionError

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:10 (10 by maintainers)

github_iconTop GitHub Comments

1reaction
Pierrcicommented, Mar 25, 2022

@McPatate @Pierrci was the security status added to the staging endpoint? It seems as if setting securityStatus in the param does not do any change

It’s using the same security scanner endpoint/instance as in prod (meaning that if you test with a model also in prod you should see something), but the webhook isn’t plugged on it (meaning any random model added in staging will not be processed)

1reaction
osansevierocommented, Mar 23, 2022

Thanks @Narsil! The copy paste from the test went wrong 🤦‍♂️. I will be testing this out in https://github.com/huggingface/huggingface_hub/pull/788

Read more comments on GitHub >

github_iconTop Results From Across the Web

Fixing Tests in CI/CD: Why are Your Tests Failing?
Taking failing tests seriously is important. Tests are meant to help. Even so, most information about running tests is based on opinion.
Read more >
Tests in Gitab CI pipeline marked as passed when 2 tests ...
Two or three of our test cases in Google tests fail. But Gitlab consider that the test stage is successful. Is it because...
Read more >
CICD - How To Resolve Failing Tests - Undo.io
How to make software application test failures in CI go away using ... This video takes you through two demos that show exactly...
Read more >
Debugging Failing Tests and Test Pipelines - GitLab
Guidelines for investigating end-to-end test pipeline failures.
Read more >
Troubleshooting Continuous Integration, or How to Debug ...
If the tests failed once on the CI, they will probably fail again - unless you change the code. Or, if the tests...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found