confusing error if wrong extension

ploomber infers the type of task with a combination of the source and product extension, but if there’s a typo a confusing error shows. example, if the user wants to dump data:

  
  - source: another.sql 
    product: another.parquets # typo here

error:

ploomber.exceptions.SourceInitializationError: Error initializing SQLScriptSource('SELECT *\nFROM nu...ams"], "param") ]]') Loaded from: /Users/Edu/Desktop/sql/another.sql: The {{product}} placeholder is required. Example: 'CREATE TABLE {{product}} AS (SELECT * FROM ...)'

Fix: see if the value in product has something similar to a .parquet or .csv extension and show an error

[edublancas]

right. the debug is not supported in SQL tasks.

As a recommendation for testing and debugging your implementation, add one test case here, you can use existing tasks as reference for building yours.

Essentially, you want to create a pipeline where one of the products has a wrong extension, then validate that the appropriate error is raised.

As a first step, write some sample test and ensure you get the original error:

ploomber.exceptions.SourceInitializationError: Error initializing SQLScriptSource('SELECT *\nFROM nu...ams"], "param") ]]') Loaded from: /Users/Edu/Desktop/sql/another.sql: The {{product}} placeholder is required. Example: 'CREATE TABLE {{product}} AS (SELECT * FROM ...)'

Then, take it from there and test your implementation, ensure you have your dev environment correctly configured: https://github.com/ploomber/ploomber/blob/master/CONTRIBUTING.md

Then work on the implementation until the test passes, you can run individual tests like this:

pytest path/to/test_file.py::test_name

I’ll add a few comments to the PR

[edublancas]

@neelasha23 as a pointer, you can use this to compute string distance to see if the user had a typo (e.g. parque instead of parquet)