Error in testing code examples from Combinators for Parsing

Testing code snippets from Combinators examples of documentation.

            // see Term, Number and Word classes in documentation...

            var spaces = either(
                eof,
                many1(satisfy(System.Char.IsWhiteSpace)).Map(_ => unit)
            );

            Parser<A> token<A>(Parser<A> parserA) =>
                from res in parserA
                from spc in spaces
                select res;

            var word = from w in token(asString(many1(letter)))
                        select new Word(w) as Term;

            var number = from d in token(asString(many1(digit)))
                          select new Number(Int32.Parse(d)) as Term;

            var term = either(word, number);

            var parser = from sp in spaces
                         from ws in many1(term)
                         select ws;

            var result = parse(parser, "   4 words are here");

Result is Error = {error at (line 1, column 20): unexpected end of stream, expecting letter, end of input or digit}.

Tried with appending space at the end of string:

  var result = parse(parser, "   4 words are here ");

Result is Error = {error at (line 1, column 21): unexpected end of stream, expecting letter, end of input or digit}.

Any thoughts?

[louthy]

But finding bit challening getting access to values inside containers and map further

It really depends on what you want to do, if you want to continue working with the results then you can just use LINQ:

var res1 = // Either<string, Seq<Term>>

var res2 = from terms in res1  // now terms == Seq<Term>
           let onlyWords = terms.Filter(t => t is Word).Map(t => t as Word)
           select onlyWords;

res2 will now be an Either<string, Seq<Word>>. You could also do it fluently:

var res2 = res1.Map(terms => terms.Filter(t => t is Word).Map(t => t as Word));

If you want to get at the raw Seq<Term> then you have to decide what to do if you get an error. For example, let’s say you ignored the error and decided that it represents zero terms:

var terms = res1.IfLeft(_ => Seq<Term>());

Then terms is just a Seq<Term> which is either empty or has the parsed terms in it. I find it’s usually best to stay in context (maintain the Either) for as long as possible, so if I had something that processed the terms then I would get that to return an Either too, and then that makes composition and error handling much easier. For example, let’s say we had a function that parsed the text and then validated the terms:

    Either<string, Unit> Validate(string text) =>
        from terms in parse(parser, text)
        from result in ValidateTerms(terms)
        select result;

    Either<string, Unit> ValidateTerms(Seq<Term> terms) =>
        term is Word word     ? ValidateWordTerm(word)
      : term is Number number ? ValidateNumberTerm(number)
      : Left("Unknown Term type");

    Either<string, Unit> ValidateWordTerm(Word word) =>
        word.Length > 1 && word.Length < 8
            ? Right<string, Unit>(unit)
            : Left("Invalid word");

    Either<string, Unit> ValidateNumberTerm(Number number) =>
        number > 100
            ? Right<string, Unit>(unit)
            : Left("Invalid number");

I think it’s often easy if you’re coming at the functional programming thing from the OO world to get into the ‘must abstract’ mindset. I think it’s very much the case that functional programming is more about concrete behaviour with known sets of types. Clearly you could add inherited methods to Term/Word/Number to do the Validation, and in this instance it may well be better - but then Term, Number, and Word are ‘infected’ with the decision on how to validate - whereas with my example above it’s contextual and the Word and Number types are just pure data.

Ultimately pick a solution that is appropriate for the problem you’re trying to solve. There are real benefits to separating data and functionality (the opposite of what we’re told is good in the OO world). Especially when the data is immutable - that’s when the systems become really powerful. It’s easy to move immutable blocks of data around a system, it’s easy to move immutable blocks of data between systems. Then each sub-system can have its own local functionality that deals with those data blocks.

The costs that come with this approach is well captured by the Expression Problem, which means it’s more difficult to extend a system built in the functional paradigm - for example, if I add a new type that derives from Term then I don’t immediately get a validator, or anything like that.

That’s the cost of functional - but my personal belief is (so, you can choose to ignore) that the OO world abstracts unnecessarily 99% of the time. And needs to extend existing types functionality rarely. It’s a myth that we need to build in abstraction layers by default “just in case”. And, so the few times that it’s needed (in the functional world) it does mean some additional diligence (especially as we haven’t got proper discriminated unions and pattern matching with exhaustiveness checking).

I can live with that because I believe that functional programs are significantly more robust, easier to test, and are cognitively easier to deal with when applications get to a super large scale.

[louthy]

@imranypatel Just to be clear. The code hasn’t changed the example or the outcome - I got the same results as @bender2k14 - it’s just the ToString() implementation was misleading. The results were correct and as stated in the Wiki documentation - IsFaulted was false, and Reply.Result did contain the correct result.

Am curious whether you’ve have seen my request on gitter on 6th April?

I did see it, I started writing a reply, but got pulled away on something else. Yes, the Free Monad approach would work well in that situation. Although if you’re just getting into this stuff then you may want to keep it simple and write some basic mapping functions from one type to another. That’s all the outcome will be: a series of operations which describe map computations. The benefit of the Free Monad approach will be that you can encapsulate the side-effecting code (printing), which will also make it easier to write unit tests.

Btw, it’s often better to ask those types of more in-depth questions on these Issues pages - it’s much easier to give code examples and can serve as future reference for others looking for similar solutions. So, feel free to open up a new issue and we can discuss it there.