ICloud integration gives invalid auth w/ app-specific pw

The problem

I’m attempting to setup the iCloud integration w/ an app-specific password, per the instructions here.

Upon hitting submit, I get a short spinner and then “Invalid authentication” — and I cannot proceed beyond this error to finish setting up the icloud integration. I’ve also tried the suggested troubleshooting step of removing the .storage/icloud directory, to no avail. (which is created upon attempting to setup the integration, but removing it has no affect on the issue)

I have confirmed that I’ve entered the app-specific password correct, and attempted several different passwords I’ve generated on appleid.apple.com.

What is version of Home Assistant Core has the issue?

core-2021.7.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

icloud

Link to integration documentation on our website

https://www.home-assistant.io/integrations/icloud/#app-specific-passwords

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2021-08-03 23:05:32 ERROR (SyncWorker_40) [pyicloud.base] Missing apple_id field
2021-08-03 23:05:32 ERROR (MainThread) [homeassistant.components.icloud.config_flow] Error logging into iCloud service: ('Invalid authentication token.', PyiCloudAPIResponseException('Missing apple_id field'))

Additional information

No response

[dsfaller]

I did more digging with the underlying pyicloud library (see comment in my issue there) - seems like pyicloud is emulating to be a web browser and not using any official API. Apparently, Apple changed the login flow or rejects app-specific passwords completely now - in any case the library no longer receives the necessary login tokens to work. My conclusion is that we need a rewrite of pyicloud for app-specific passwords or another library for the icloud plugin in Home Assistant… 😦

[magicus]

I’m currently in the process of replacing the standard icloud integration with icloud3. It is not really a smooth ride. Apparently icloud3 is much more capable, but that also means turning a lot more knobs to get it to work. Get me right, I love the kind of add-ons that gives you the full power to tweak anything, but for a default option in HA, there need to be a simple to use solution that solves most cases with good defaults.

I think icloud3 would need some work to get there, possibly splitting it in two, one simplified version to put in HA core, and one separate with full power. If the icloud3 developer(s) support this, I think it would be great, but it might not be painless for icloud3.

In fact, for me, the standard icloud solution was kind of perfect, except for the slightly annoying fact that it stopped working.

So an alternative is to just see what icloud3 does differently in terms of authentication, and copy that over to the standard icloud integration. Possibly it’s just a more up-to-date fork of pyicloud (pyicloud_ic3.py).