Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Input id/name with colon

See original GitHub issue

Version of the custom_component

v6.3.2

Configuration

multiscrape:
  - resource: 'https://czview.contazara.es/private/czview/meter_list.xhtml'
    scan_interval: 86400
    form_submit:
      submit_once: True
      resource: 'https://czview.contazara.es/login.xhtml'
      select: "#loginForm"
      input:
        loginForm:username: '*******'
        loginForm:password: '*************'
    sensor:
      - name: 'Consumo agua'
        select: '.ui-datatable-data > tr > td:nth-child(4)'
        value_template: '{{ value | float }}'
        unit_of_measurement: 'm³'

Describe the bug

Hello! I think the problem comes from the fact that the id/name of the form in the username input and password input has colon.

Debug log

2022-07-18 11:37:50 DEBUG (MainThread) [custom_components.multiscrape.sensor] Scraper_noname_0 # Consumo agua # Setting up sensor
2022-07-18 11:37:50 DEBUG (MainThread) [custom_components.multiscrape.sensor] Scraper_noname_0 # Consumo agua # Start scraping to update sensor
2022-07-18 11:37:50 DEBUG (MainThread) [custom_components.multiscrape.scraper] Scraper_noname_0 # Consumo agua # Tag selected: None
2022-07-18 11:37:50 DEBUG (MainThread) [custom_components.multiscrape.form] Scraper_noname_0 # Exception occurred while scraping, will try to resubmit the form next interval.
2022-07-18 11:37:50 ERROR (MainThread) [custom_components.multiscrape.sensor] Scraper_noname_0 # Consumo agua # Unable to scrape data: Could not find a tag for given selector.
Consider using debug logging and log_response for further investigation.
2022-07-18 11:37:50 DEBUG (MainThread) [custom_components.multiscrape.sensor] Scraper_noname_0 # Consumo agua # On-error, set value to None
2022-07-18 11:37:50 DEBUG (MainThread) [custom_components.multiscrape.entity] Scraper_noname_0 # Consumo agua # Updated sensor and attributes, now adding to HA

Thank you for your work!

Issue Analytics

State:
Created a year ago
Comments:7 (3 by maintainers)

Top GitHub Comments

1reaction

amaciasfcommented, Jul 22, 2022

Hello! I have been doing multiple tests these days and I have found that it is not really authenticating because the value of the submit button is not being sent.

You can see that this information is sent

HTML:

POST data:

Now, in HA I have installed tcpdump to capture the request made by multiscrape and it is seen that the value of the button is not sent.

Then I have created an HTML form without the button and the authentication has not worked. I have added the button to the form and the authentication has worked.

Therefore, I have changed the configuration by adding the corresponding input to the button:

multiscrape:
  - resource: 'https://czview.contazara.es/private/czview/meter_view.xhtml
    scan_interval: 10
    form_submit:
      #submit_once: True
      resource: 'https://czview.contazara.es/login.xhtml'
      select: "#loginForm"
      input:
        'loginForm:username': '***********'
        'loginForm:password': '***********'
        'loginForm:j_idt20': ''
    sensor:
      - name: 'Consumo agua'
        select: '.ui-datatable-data > tr > td:nth-child(4)'
        value_template: '{{ value | float }}'
        unit_of_measurement: 'm³'

And now it DOES work.

2022-07-22 12:07:17 DEBUG (MainThread) [custom_components.multiscrape.sensor] Scraper_noname_0 # Consumo agua # Start scraping to update sensor
2022-07-22 12:07:17 DEBUG (MainThread) [custom_components.multiscrape.scraper] Scraper_noname_0 # Consumo agua # Tag selected: <td>214.73</td>
2022-07-22 12:07:17 DEBUG (MainThread) [custom_components.multiscrape.scraper] Scraper_noname_0 # Consumo agua # Selector result: 214.73
2022-07-22 12:07:17 DEBUG (MainThread) [custom_components.multiscrape.scraper] Scraper_noname_0 # Consumo agua # Applying value_template on selector result
2022-07-22 12:07:17 DEBUG (MainThread) [custom_components.multiscrape.scraper] Scraper_noname_0 # Consumo agua # Final selector value: 214.73
2022-07-22 12:07:17 DEBUG (MainThread) [custom_components.multiscrape.sensor] Scraper_noname_0 # Consumo agua # Selected: 214.73
2022-07-22 12:07:17 DEBUG (MainThread) [custom_components.multiscrape.entity] Scraper_noname_0 # Consumo agua # Updated sensor and attributes, now adding to HA

Regards!

0reactions

danieldotnlcommented, Jul 25, 2022

Great, so I suppose this issue can be closed.