[ENH] Allow updating UDFs
See original GitHub issueIs your feature request related to a problem? Please describe. While developing UDF logic, it’s common to update or tweak the code then test it live inside a query.
Currently you can only register a function w/ the same name once. If you update the UDF code and re-register you get:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/tmp/ipykernel_1172564/3008080374.py in <module>
22 return c * r
23
---> 24 c.register_function(haversine_dist, "haversine_dist", [
25 ("source_lat", np.float64),
26 ("source_lng", np.float64),
~/conda/envs/dsql/lib/python3.8/site-packages/dask_sql/context.py in register_function(self, f, name, parameters, return_type, replace, schema_name, row_udf)
312
313 """
--> 314 self._register_callable(
315 f,
316 name,
~/conda/envs/dsql/lib/python3.8/site-packages/dask_sql/context.py in _register_callable(self, f, name, aggregation, parameters, return_type, replace, schema_name, row_udf)
830
831 elif schema.functions[lower_name] != f:
--> 832 raise ValueError(
833 "Registering different functions with the same name is not allowed"
834 )
ValueError: Registering different functions with the same name is not allowed
Describe the solution you’d like Ideally if you register a function w/ the same name a second time, Dask-SQL would “overwrite” the existing function and use the new code from then on.
Describe alternatives you’ve considered As a workaround, you can iterate by adding a numbered version to the function name, but that gets tedious.
You could re-create a new Context but that means re-loading all your tables.
Issue Analytics
- State:
- Created 2 years ago
- Comments:5 (1 by maintainers)
Top Results From Across the Web
Allowing UDF Update in Document Rows
To allow this flexibility, SAP Business One has a new feature called “Allow Update of User Defined Fields When Document Rows Are Not...
Read more >sql: support mutations within UDFs · Issue #87289 - GitHub
We do not currently allow mutations (INSERT, UPDATE, UPSERT, DELETE) within user-defined function bodies. We should lift this restriction.
Read more >Getting Started with PySpark UDF(User Defined Function)
What is UDF ? A User Defined Function is a custom function defined to perform transformation operations on Pyspark dataframes.
Read more >Workload Management with User Defined Functions and ...
A scalar UDF will update the TIP table on each AMP after every row is updated. Each time it finishes with its work...
Read more >SOAP API: user-defined fields (UDFs) - AutoTask
IMPORTANT The SOAP API entered a limited enhancement phase in Q4 2020. ... You may also update UDFs for entity types that allow...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I can work on this.
Probably linking to the API from the “Custom Functions and Aggregations” page is a good idea.
I can see why register_function doesn’t allow replace by default, but it might be nice for the error message to suggest using
replace=True
.