Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support for "ignoring a list of specific fields"

See original GitHub issue

I was using this tool to generate our regression baselines, the result is a JSON string like:

"id": "4163B50C4DD9911804894F827078B218",
"instanceId": "020B24C54D891A89B44AAABDBCFCE057",
 "status": 1

The problem is that there are some fields such as the instanceId field that is using UUID that will always change, and I don’t want this false alert. One workaround would be:

  • We provide the ignores=["instanceId",] to the data_regression function, and the value of this field will be ignored when comparing the result.

  • Or, we can replace the value of specific field using special string, such as “{% ignore %}” or something else;

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:11 (5 by maintainers)

github_iconTop GitHub Comments

qweezecommented, Aug 12, 2020

As an idea, I think it would be nice to have a more generic solution that can handle not just ignoring specific fields, but also custom comparison behaviour.

For example, one may want to be able to

  • ignore the value of id but assert that id field is present
  • ignore the value but assert that it is a positive integer
  • assert that a float equals expected value with some tolerance
  • assert that a list contains expected items ignoring order
  • assert that a string matches a regex

One way to achive this could be by providing a way to override fields’ values with any object that defines an __eq__ method. Something like

from unittest.mock import ANY
from pytest import approx

data = {
    "orders": [{
        "id": "aksjsnj-10291",
    "product" : {
        "id": "215",
    "gravity": 9.81,
    # ...100500 other fields

    {'product': {'id': ANY}, 'gravity': approx(10, 0.1)}
igortgcommented, Aug 6, 2020

We had the same issue using data_regression for SQLAlchemy models.

Below is a recipe for a new fixture based on data_regression that would ignore some fields of the data given to the check method. The recipe uses regex patterns, but I think you can simplify it for your own purpose. You can write this code into your project, update the class member fields_to_ignore and use the db_model_regression.check instead.

def db_model_regression(data_regression):
    return DbModelRegression(data_regression)

class DbModelRegression:

    fields_to_ignore = ['updated_on', 'created_on', r'\w*_id', '^id$']

    def __init__(self, data_regression):
        self.data_regression = data_regression

    def check(self, data, **kwargs):
        return self.data_regression.check(self._prepare_for_regression(data), **kwargs)

    def _should_record(self, field_name):
        for pattern in self.fields_to_ignore:
            if re.match(pattern, field_name):
                return False
            return True

    def _prepare_for_regression(self, data):
        if isinstance(data, list):
            prepared_data = []
            for item in data:
            return prepared_data
        elif isinstance(data, dict):
            return self._prepare_dict(data)
            return data

    def _prepare_dict(self, data):
        prepared_data = {}
        for key in filter(self._should_record, data):
            value = data[key]
            if isinstance(value, list):
                prepared_list = []
                for item in value:
                prepared_data[key] = prepared_list
            elif isinstance(value, dict):
                prepared_data[key] = self._prepare_for_regression(value)
                prepared_data[key] = value
        return prepared_data

I agree with you that we must find a more elegant way of address that. But I don’t think that adding an ignores option is the answer. Maybe adding some highlevel fixtures that knows how to deal with objects instead of dicts. I’ll try to put some thought on it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

java - Testing content of list ignoring some of the fields
One way to do this would be to look for some detail in each element to see which expected element to compare with...
Read more >
Ignore fields - Db2 Object Comparison Tool 12.1 - IBM
By using ignore fields, you can compare DB2 catalog records while ignoring some fields. Ignore fields are used in situations where you are...
Read more >
Ignoring Fields - Oracle Help Center
Ignoring Fields ; Select Field, then Properties. ; In the Field Properties dialog box, select the Data Load Properties tab. ; Select Ignore...
Read more >
Ignore system fields, audit Fields when Extracting fields of an ...
One way to do this is by removing the fields which i dont want from the list. But i want a dynamic way...
Read more >
21454 (Ignoring certain fields on INSERT and UPDATE queries)
This could be done by allowing a field to tell django to ignore it on INSERT or UPDATE queries, the same way it...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found