question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[ENH] Allow updating UDFs

See original GitHub issue

Is your feature request related to a problem? Please describe. While developing UDF logic, it’s common to update or tweak the code then test it live inside a query.

Currently you can only register a function w/ the same name once. If you update the UDF code and re-register you get:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_1172564/3008080374.py in <module>
     22         return c * r
     23 
---> 24 c.register_function(haversine_dist, "haversine_dist", [
     25                     ("source_lat", np.float64),
     26                     ("source_lng", np.float64),

~/conda/envs/dsql/lib/python3.8/site-packages/dask_sql/context.py in register_function(self, f, name, parameters, return_type, replace, schema_name, row_udf)
    312 
    313         """
--> 314         self._register_callable(
    315             f,
    316             name,

~/conda/envs/dsql/lib/python3.8/site-packages/dask_sql/context.py in _register_callable(self, f, name, aggregation, parameters, return_type, replace, schema_name, row_udf)
    830 
    831             elif schema.functions[lower_name] != f:
--> 832                 raise ValueError(
    833                     "Registering different functions with the same name is not allowed"
    834                 )

ValueError: Registering different functions with the same name is not allowed

Describe the solution you’d like Ideally if you register a function w/ the same name a second time, Dask-SQL would “overwrite” the existing function and use the new code from then on.

Describe alternatives you’ve considered As a workaround, you can iterate by adding a numbered version to the function name, but that gets tedious.

You could re-create a new Context but that means re-loading all your tables.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
DaceTcommented, Oct 20, 2021

I can work on this.

1reaction
randerzandercommented, Oct 20, 2021

Probably linking to the API from the “Custom Functions and Aggregations” page is a good idea.

I can see why register_function doesn’t allow replace by default, but it might be nice for the error message to suggest using replace=True.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Allowing UDF Update in Document Rows
To allow this flexibility, SAP Business One has a new feature called “Allow Update of User Defined Fields When Document Rows Are Not...
Read more >
sql: support mutations within UDFs · Issue #87289 - GitHub
We do not currently allow mutations (INSERT, UPDATE, UPSERT, DELETE) within user-defined function bodies. We should lift this restriction.
Read more >
Getting Started with PySpark UDF(User Defined Function)
What is UDF ? A User Defined Function is a custom function defined to perform transformation operations on Pyspark dataframes.
Read more >
Workload Management with User Defined Functions and ...
A scalar UDF will update the TIP table on each AMP after every row is updated. Each time it finishes with its work...
Read more >
SOAP API: user-defined fields (UDFs) - AutoTask
IMPORTANT The SOAP API entered a limited enhancement phase in Q4 2020. ... You may also update UDFs for entity types that allow...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found