question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

"unknown" option should apply to nested fields so that "exclude" works

See original GitHub issue

The fact that exclude takes dotted notation for nested fields, but unknown does not apply to nested fields, means that exclude doesn’t work for nested fields in some common use cases unless the nested schema(s) have the desired Meta.unknown setting, and then they are not runtime-flexible, which is the great benefit of Schema and load taking exclude. Somewhat new to marshmallow so maybe there’s a way to accomplish this (setting exclude on nested fields at load time) in a different way?

Happy to contribute a PR if you guys think this is a good idea.

Issue Analytics

  • State:open
  • Created 4 years ago
  • Comments:15 (11 by maintainers)

github_iconTop GitHub Comments

2reactions
mahenzoncommented, Oct 12, 2020

any news here?

1reaction
sirosencommented, May 15, 2020

Looking at this issue from the perspective of webargs, I see two (related) issues with #1575. I think I have a good idea of how to resolve them, but want to see if people like the results before I try to write it.

  • If unknown propagates automatically in any context (as in #1575), it will be backwards incompatible
  • If unknown behaves inconsistently between Meta, schema instantiation, and load, it will cause confusion

And then, as an ancillary concern, if someone is writing a library on top of marshmallow (👋 hi guys!), they’ll want to be able to control this or offer control to their users.

Maybe backwards incompatibility isn’t an issue in this case – if it’s considered a bug that’s being fixed – but I think it will be a problem for someone.


So my idea:

  • add a new parameter in all of the same places as unknown, propagate_unknown
  • propagate_unknown can be True or False (by default, for backwards compat) and can be removed (True everywhere) in marshmallow v4
  • if propagate_unknown is True, behave as in #1575 when unknown is passed to load, and if False, behave as today

I would take #1575 as a basis for this work. It looks solid, has nice tests, and credit where it’s due – I just don’t quite agree with the behavior.

The downsides are that we have a new attribute to control unknown, which might feel messy, and that we don’t get the desired behavior by default. To me, those are worthwhile in the first case and a necessary evil in the second.

You can do it all with just one attribute, but I promise that it's worse.

This is just for fun. Don’t take this suggestion seriously. Please. 😄

We could change the constants a bit…

RAISE = 1
INCLUDE = 2
EXCLUDE = 4
PROPAGATE = 8
...
schema.load(data, unknown=EXCLUDE|PROPAGATE)

but then we’re just asking for trouble because you have to check for INCLUDE|EXCLUDE, which makes no sense.

Read more comments on GitHub >

github_iconTop Results From Across the Web

"Unknown field" after updating to 3.5.0 marshmallow
from marshmallow import EXCLUDE class UserSchema(Schema): class Meta: ... The unknown option value set in load will override the value ...
Read more >
Upgrading to Newer Releases — marshmallow 3.19.0 ...
The unknown option can be passed as a Meta option, on Schema instantiation, or at load time. from marshmallow import Schema, fields, EXCLUDE,...
Read more >
Nested query | Elasticsearch Guide [8.5]
Wraps another query to search nested fields. The nested query searches nested field objects as if they were indexed as separate documents.
Read more >
How to Ignore Unknown Properties While Parsing JSON in ...
Jackson API provides two ways to ignore unknown fields, first at the class level using ... If you compile and run your program...
Read more >
EXCLUDE Level of Detail Expressions
Applies to: Tableau Cloud, Tableau Desktop, Tableau Public, Tableau Server ... the Analytics pane wouldn't work, because it would simply be a horizontal ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found