[BUG] Tuya data processing - ZCL frame multiple attributes

Describe the bug Lidl Parkside water valve (under development), which is as usual a Tuya device publishes one ZCL frame with two attributes+values. This structure cannot be deserialized correctly using any of two Tuya Manufucturing Clusters. In one case the second one is completely ignored (a warning is issued, see log) in other case both attributes will get value from the second one.

Here is the complete ZCL Frame: b'\x09\x7B\x02\x01\x0F\x01\x01\x00\x01\x01\x05\x02\x00\x04\x00\x00\x00\x07' first attribute is onoff \x01\x01\x00\x01\x01, second attribute is timer duration \x05\x02\x00\x04\x00\x00\x00\x07.

I could narrow the deserializing problem to class TuyaData https://github.com/zigpy/zha-device-handlers/blob/b802c1fb2cf2682f9a4722bfb57a1958cad9dad7/zhaquirks/tuya/__init__.py#L205-L210 (I would like to stick with TuyaNewManufCluster)

In theory the TuyaData class could during deserializing check if there is some more data to parse, but how to run this process again? I have to idea how to solve it, do any of you can point me to right direction?

Complete log

2022-05-15 07:00:23 DEBUG (MainThread) [zigpy.zcl] [0x2F34:1:0xef00] Received ZCL frame: b'\x09\x7B\x02\x01\x0F\x01\x01\x00\x01\x01\x05\x02\x00\x04\x00\x00\x00\x07'
2022-05-15 07:00:23 DEBUG (MainThread) [zigpy.zcl] [0x2F34:1:0xef00] Decoded ZCL frame header: ZCLHeader(frame_control=FrameControl(frame_type=<FrameType.CLUSTER_COMMAND: 1>, is_manufacturer_specific=0, is_reply=1, disable_default_response=0, reserved=0, *is_cluster=True, *is_general=False), tsn=123, command_id=2, *is_reply=True)
2022-05-15 07:00:23 DEBUG (MainThread) [zigpy.zcl] [0x2F34:1:0xef00] Decoded ZCL frame: TuyaWaterValveManufCluster:set_data_response(data=TuyaCommand(status=1, tsn=15, dp=1, data=TuyaData(dp_type=<TuyaDPType.BOOL: 1>, function=0, raw=b'\x01', *payload=<Bool.true: 1>)))
2022-05-15 07:00:23 WARNING (MainThread) [zigpy.zcl] [0x2F34:1:0xef00] Data remains after deserializing ZCL frame: b'\x05\x02\x00\x04\x00\x00\x00\x07'
2022-05-15 07:00:23 DEBUG (MainThread) [zigpy.zcl] [0x2F34:1:0xef00] Received command 0x02 (TSN 123): set_data_response(data=TuyaCommand(status=1, tsn=15, dp=1, data=TuyaData(dp_type=<TuyaDPType.BOOL: 1>, function=0, raw=b'\x01', *payload=<Bool.true: 1>)))

[puddly]

If I’m understanding the problem correctly, I think this might work (feel free to rename the struct, I’m not too familiar with the Tuya protocol):

diff --git a/zhaquirks/tuya/__init__.py b/zhaquirks/tuya/__init__.py
index 1164358..14679da 100644
--- a/zhaquirks/tuya/__init__.py
+++ b/zhaquirks/tuya/__init__.py
@@ -262,13 +262,18 @@ class Data(t.List, item_type=t.uint8_t):
         return value
 
 
+class TuyaDatapointAndData(t.Struct):
+    dp: t.uint8_t
+    data: TuyaData
+
+
 class TuyaCommand(t.Struct):
     """Tuya manufacturer cluster command."""
 
     status: t.uint8_t
     tsn: t.uint8_t
-    dp: t.uint8_t
-    data: TuyaData
+    datapoints: t.List[TuyaDatapointAndData]
 
 
 class TuyaManufCluster(CustomCluster):

Example:

In [1]: from unittest.mock import Mock

In [2]: from zhaquirks.tuya.mcu import TuyaNewManufCluster

In [3]: ep = Mock()  # fake endpoint object

In [4]: data = b'\x09\x7B\x02\x01\x0F\x01\x01\x00\x01\x01\x05\x02\x00\x04\x00\x00\x00\x07'

In [5]: TuyaNewManufCluster(ep).deserialize(data)
Out[5]:
(ZCLHeader(frame_control=FrameControl(frame_type=<FrameType.CLUSTER_COMMAND: 1>, is_manufacturer_specific=0, is_reply=True, disable_default_response=0, reserved=0, *is_cluster=True, *is_general=False), tsn=123, command_id=2, *is_reply=True),
 set_data_response(data=TuyaCommand(status=1, tsn=15, datapoints=[TuyaDatapointAndData(dp=1, data=TuyaData(dp_type=<TuyaDPType.BOOL: 1>, function=0, raw=b'\x01', *payload=<Bool.true: 1>)), TuyaDatapointAndData(dp=5, data=TuyaData(dp_type=<TuyaDPType.VALUE: 2>, function=0, raw=b'\x07\x00\x00\x00', *payload=7))])))

Unit tests will need to be updated for this to work.

[rforro]

Shortcut to how z2m solved it https://github.com/Koenkk/zigbee-herdsman/pull/483