Support for Aggregations on Array Items

Use case: I’d like to be able to map functions (sum(), avg(), etc) across array values from multiple rows and return the results in the same order.

Feature description: There doesn’t seem to be a performant way to use aggregate functions (example: sum()) across an array field across each individual index from multiple rows while returning the results in the same order. I’ve tried playing with unnest and array_agg, but it jumbles all the rows together and seems like performance would suck if you had larger arrays. (1,000+ items in each array per row)

Example table:

event_time          | int_array_col
2021-10-19 18:12:26 | [1, 0, 1, 4, 2, 1, 0, 0, 1]
2021-10-19 18:22:41 | [1, 1, 1, 1, 0, 0, 0, 0, 1]
2021-10-19 18:32:13 | [0, 0, 0, 1, 1, 1, 0, 0, 1]
2021-10-19 18:43:17 | [2, 2, 2, 2, 0, 0, 0, 0, 1]

Desired query: SELECT array_map(sum(), int_array_col) from doc.my_table;

Output:

[4, 3, 4, 8, 3, 2, 0, 0, 4]

[proddata]

If it isn’t too many rows an UDF might be a (temporary) solution:

cr> CREATE TABLE arr_sum (arr ARRAY(INTEGER));                                                                                                                                                                                

cr> INSERT INTO arr_sum VALUES ([1,2,3]),([0,1,2]);      

CREATE OR REPLACE FUNCTION udf_arr_sum(arr ARRAY(ARRAY(INTEGER)))  
        RETURNS ARRAY(INTEGER)  
        LANGUAGE JAVASCRIPT  
        AS  
        'function udf_arr_sum(arr){  
            let res = [... arr[0]]; 
     
            for(let i = 1; i < arr.length; i++){ 
                for(let j = 0; j < res.length; j++){ 
                    res[j] = res[j] + arr[i][j]; 
                } 
            } 
          return res; 
          }'  
         ;                                                                                                                                                                                                                    
CREATE OK, 1 row affected  (0.127 sec)

 SELECT udf_arr_sum(array_agg(arr)) FROM arr_sum;                                                                                                                                                                          
+---------------------------------+
| doc.udf_arr_sum(array_agg(arr)) |
+---------------------------------+
| [1, 3, 5]                       |
+---------------------------------+ 

[proddata]

@proddata Would this still work at 100,000 rows?

I suppose. Quick test with 1 mio rows of 10 values takes a few seconds already. With 100.000 it is still instant.

Is there any way to profile UDF functions to know how much memory they’re using?

Not aware of any such functionality in place yet.