Validate ignored fields

Motivation Sometimes in a flat file, there are fields that are supposed to have constant values. You can just Ignored() them but maybe for additional safety, you want to check that they contain the expected value and that the file is completely valid.

Current solutions The only solution I can think of today is mapping them as normal fields and having code that throws if they aren’t what you’d expect.

mapper.Property(x => x.Version);

class Row 
{
  public string Version 
  { 
    set { if (value != "4") throw new BadDataException(); } 
  }
}

Drawbacks

• When you have multiple constant fields, it’s annoying to have Constant1, Constant2, Constant3 in your model, just like it’s annoying to declare Ignored1, Ignored2, Ignored3.
• A good exception to throw would be ColumnProcessingException and include context position. This exception is internal (so unavail. to user code) and the context is harder to get.

Proposal Add a new Ignored(string value) overload to Ignored(Window w).

This new overload would work like Ignored(value.Length) when the file is correct, but when the ignored field is different from value then it goes through the ColumnError process (column event, then row event, then global exception).

[jehugaleahsa]

OnParsing occurs before I try to interpret the text from the file, so before I check if it is null and try to convert it to the column type (int, double, DateTime, etc.). The OnParsed happens after I have determined it is null or converted it. In your case, since it’s for an ignored column, you can just use OnParsing and look at the raw value – I will have replaced it with null by the time OnParsed fires: https://github.com/jehugaleahsa/FlatFiles/blob/master/FlatFiles/IgnoredColumn.cs#L51

As for the exception, even something as simple as throw new Exception("Unexpected ignored column token") will cause the exception to get caught here: https://github.com/jehugaleahsa/FlatFiles/blob/master/FlatFiles/Schema.cs#L120. However, you may wish to use your own exception subclass if it helps you to distinguish between this specific error and something else, or you could simply look at the message contents.

Again, when you give this a try, if you see anything unexpected or need more guidance, just let me know.

[jehugaleahsa]

Yeah, curiously I had marked the Preprocess getter/setter on the IColumnDefinition interface as Obsolete but that did not result in any warnings on the ColumnDefinition class that implemented that interface, nor did it cause any using code to complain. With 4.9.0, I added the Obsolete onto the ColumnDefinition and the warnings showed up. I also went through each of the property mapping classes and marked the Preprocess methods as Obsolete, as well.

The property mapping classes (and ignored mapping) now have four new methods for registering OnParsing, OnParsed, OnFormatting and OnFormatted callbacks. I did have to modify the IgnoredColumn to fire these events whereas before I just returned null and called NullFormatter.FormatNull (I am not even sure I did that much, honestly).

This was actually a good exercise because it made me realize there were cases where these callbacks were not firing (when nulls were being handled) and it also made me realize that in the Schema class I was just skipping over ignored columns entirely, resulting in nulls getting written out instead of the desired value. 😬

Please give 4.9.0 a shot and let me know if you are able to handle everything you wanted to do in your original comment above. If you run into any issues or still cannot figure out how to do something, just ping me and I’ll see what I can do. Otherwise, please close the ticket.