got error if the xpath return nothing

Hi

I noticed that currently jsnapy raise an error if the Xpath return nothing. It’s an issue for Xpath that are tracking error messages for example. We want to check for error but it the XPAth return nothing, it mean there is no error.

In the SLAX version, this Xpath was valid, it was just returning a list of 0 member

Maybe we should allow a Xpath to return an empty list and add an option to define the minimum numbers of elements an XPATH should return. I can see many cases where it would be handy. · Check there are 2 RE · Check there are 4 Uplinks etc …

Tests Included: check_alarm
************************ Command: show chassis alarms ************************
ERROR!! Nodes are not present in given Xpath: <//alarm-detail>
FAIL | "alarm-class" is in list ['Major', 'Warning'] [ 0 matched / 1 failed ]

Thanks Damien

[eeishaan]

@maxdevyatov @dgarros The rational behind the current behaviour: Let’s say I run the following test:

test_fpc_version:
  - command: show chassis hardware clei-models
  - iterate:
      xpath: ./chassis/chassis-module
      id: name
      tests:
        - no-diff: chassis-sub-module/version

Now there are two xpaths to consider here:

1. The one mentioned in ‘xpath’
2. The one mentioned after ‘no-diff’

This is how the program works now:

1. Extract elements that match the first given xpath.
2. If nothing is found, then an error is raised because there is nothing that can be done on missing nodes.
3. If something is found, then it tries to find the second xpath given.
4. After searching for second xpath, happens something in the code that is operator dependent. If nothing is found in this second search, some operators (eg no-diff) treat it as null and continue with their work, while some complain and raise an error.

The program treats the absence of nodes for a given xpath as error because:

1. One must have expected some nodes to be there and thus wrote the tests in the first place. If not found the program should notify the user about the same.
2. No operations can be performed on a missing node. Either we can skip the test altogether or fail it. Passing it (however necessary it may seem) just doesn’t feel right.

What I propose:

Keeping in mind that there is demand for old behaviour, two solutions have been drawn up

1. Introduction of a new optional attribute : ignore-null. This can be used with every operator and brings back the old behaviour of passing the test in the absence of nodes for which xpath ( either of the above mentioned two ) is provided. One thing to note is that instead of classifying this as Error in the absence of nodes, the program would now just SKIP the test, not pass, just skip. So the above test now becomes:

   test_fpc_version:
  - command: show chassis hardware clei-models
  - iterate:
      xpath: ./chassis/chassis-module
      id: name
      tests:
        - no-diff: chassis-sub-module/version
           ignore-null: true

1. Use conditional test operators : #46 in the upcoming 1.1 version. If one wants the old behaviour of PASSING the test in the absence of nodes, one can do the following:

test_fpc_version:
  - command: show chassis hardware clei-models
  - iterate:
      xpath: ./chassis/chassis-module
      id: name
      tests:
        - OR:
                  - not-exists: ./chassis/chassis-module/chassis-sub-module/version
                  - no-diff: chassis-sub-module/version

In this, if the nodes along the xpath : ./chassis/chassis-module/chassis-sub-module/version are absent then the no-exists test would evaluate to true and skip the next operator. This way the whole test evaluates to True. If nodes are present, it works normally.

I think we all are expecting too much from each operator. It would be better to intuitively divide the work between different operators and eliminate hidden behaviors. This way it becomes easier and less frustrating for new users to pick up the tool.

Request you to kindly comment.

[dmontagner]

I really liked the idea of introducing the ignore-null attribute.

Both options are good from the test file readability perspective.

I think the discussion is more about if we introduce the new attribute (which require additional changes) or leave as it is (to be solved with the conditional test operators, hence no additional coding is required).

An important thing to note is (assuming we only go with the conditional test operators):

• if someone is using no-diff-in, the jsnap2py tool will have to convert no-diff-in in a conditional test. I think adding the test attribute is a simpler task than translating into a conditional test case.
• an explanation somewhere in the documentation/release notes that in order to achieve the old behaviour, a conditional test is required for this case.

Without de-valuing the conditional test operators, if I have to choose one of the options, I would go with the test attribute because this avoids me of using a conditional test operator, i.e., the test file is kept simple.

Thanks

On 31 Aug 2016, at 23:21, Ishaan Kumar [e-mail] wrote:

The rational behind the current behaviour: Let’s say I run the following test:

test_fpc_version:

• command: show chassis hardware clei-models
• iterate: xpath: ./chassis/chassis-module id: name tests: - no-diff: chassis-sub-module/version

Now there are two xpaths to consider here:

• The one mentioned in ‘xpath’ • The one mentioned after ‘no-diff’ This is how the program works now:

• Extract elements that match the first given xpath. • If nothing is found, then an error is raised because there is nothing that can be done on missing nodes. • If something is found, then it tries to find the second xpath given. • After searching for second xpath, happens something in the code that is operator dependent. If nothing is found in this second search, some operators (eg no-diff) treat it as null and continue with their work, while some complain and raise an error. The program treats the absence of nodes for a given xpath as error because:

• One must have expected some nodes to be there and thus wrote the tests in the first place. If not found the program should notify the user about the same. • No operations can be performed on a missing node. Either we can skip the test altogether or fail it. Passing it (however necessary it may seem) just doesn’t feel right. What I propose:

Keeping in mind that there is demand for old behaviour, two solutions have been drawn up

• Introduction of a new optional attribute : ignore-null. This can be used with every operator and brings back the old behaviour of passing the test in the absence of nodes for which xpath ( either of the above mentioned two ) is provided. One thing to note is that instead of classifying this as Error in the absence of nodes, the program would now just SKIP the test, not pass, just skip. So the above test now becomes: test_fpc_version:

• command: show chassis hardware clei-models

• iterate: xpath: ./chassis/chassis-module id: name tests: - no-diff: chassis-sub-module/version ignore-null: true

  • Use conditional test operators : #46 in the upcoming 1.1 version. If one wants the old behaviour of PASSING the test in the absence of nodes, one can do the following: test_fpc_version:

• command: show chassis hardware clei-models

• iterate: xpath: ./chassis/chassis-module id: name tests: OR: - not-exists: ./chassis/chassis-module/chassis-sub-module/version - no-diff: chassis-sub-module/version

In this, if the nodes along the xpath : ./chassis/chassis-module/chassis-sub-module/version are absent then the no-exists test would evaluate to true and skip the next operator. This way the whole test evaluates to True. If nodes are present, it works normally.

I think we all are expecting too much from each operator. It would be better to intuitively divide the work between different operators and eliminate hidden behaviors. This way it becomes easier and less frustrating for new users to pick up the tool.

Request you to kindly comment.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.