PSRAM is unusable due to broken ESP-IDF builder

According to documentation, adding some build flags is sufficient to enable PSRAM support and enable the PSRAM/SPI RAM cache workaround. For background, ESP32 chips have a silicon bug that causes memory inconsistency unless this workaround is enabled at compile & link time.

The documentation says to enable the workaround with the -DCONFIG_SPIRAM_CACHE_WORKAROUND build flag. However, this flag doesn’t actually do anything. The flag itself is handled by the ESP-IDF build system, which PlatformIO doesn’t use. So it has no actual effect.

Trying to use PSRAM results in memory inconsistencies and crashes. Random bytes in PSRAM will flip to zero and other weird stuff will happen. Overall, the system is completely unstable. This may also be an issue for Arduino users, but I’ve only tested with ESP-IDF.

To fix this problem, the build system needs to do a few different things when CONFIG_SPIRAM_CACHE_WORKAROUND is enabled:

1. Pass the -mfix-esp32-psram-cache-issue flag to the compiler to ensure only compatible code is emitted.
2. Ensure that libc-psram-workaround.a and libm-psram-workaround.a are linked instead of the default libc.a and libm.a.
3. Skip esp32.rom.spiram_incompatible_fns.ld during linking, since it is explicitly SPI RAM-incompatible. Instead, pass esp32-spiram-rom-functions.lf to ldgen.py.

Also, ldgen.py needs to be pointed to a valid sdkconfig file. Currently, generate_project_ld_script points it to sdkconfig.h, which it cannot process.

I’ve addressed these issues in a fork. I’m happy to open a pull request if this looks acceptable: https://github.com/tidbyt/platform-espressif32/commit/69359a84e88c4d603ab202ef0f884f087ae93805

Finally, if you want PSRAM to work, you need these fixes and also need, at minimum, the following config in sdkconfig:

CONFIG_SPIRAM_SUPPORT=y
CONFIG_SPIRAM_CACHE_WORKAROUND=y

// either of the following will work
CONFIG_SPIRAM_USE_MALLOC=y
CONFIG_SPIRAM_USE_CAPS_ALLOC=y

I think #175 tries to address this, but I’m not sure if build flags are sufficient, since ldgen actually looks at the sdkconfig file directly.

[valeros]

@davidjade

So I am curious, how do your changes work? Looking at the changes, I see stuff that looks like the old PIO fix, swapping out the broken newlib ROM stuff (but apparently not using the PSRAM libc & libm replacements?).

I didn’t expect to see this type of fix as needed as I thought that the new PIO was just using the ESP-IDF CMake system and the ESP-IDF build system would take care of everything? So what’s the interaction between the two systems now? Is it some sort of hybrid? I’d like to get a better understanding of this.

Yes, it’s somewhat hybrid, but not as much as it was with v3.x. In a nutshell, CMake doesn’t export (at least in the latest stable release) actual commands for targets added via the add_custom_target function, so we need to reproduce these targets in our build script manually. One of these custom targets is the command for generating the project linker script and the problem was that in our build script this command simply lacked an additional linker script fragment which takes into account the PSRAM (that’s why I need the actual SDK configuration to figure out whether the PSRAM is enabled).

And what about the libc-psram-workaround.a & libm-psram-workaround.a changes? Have these been depreciated in ESP-IDF 4.0? I had thought they would still be needed but I can’t seem to figure it out. Where do they come into play if still needed?

Looks like this has been fixed in the latest toolchain:

if(GCC_NOT_5_2_0)
    if(CONFIG_NEWLIB_NANO_FORMAT)
        set(LIBC c_nano)
    else()
        set(LIBC c)
    endif()

    set(LIBM m)
else()
    if(CONFIG_SPIRAM_CACHE_WORKAROUND)
        set(LIBC c-psram-workaround)
        set(LIBM m-psram-workaround)
    else()
        if(CONFIG_NEWLIB_NANO_FORMAT)
            set(LIBC c_nano)
        else()
            set(LIBC c)
        endif()

        set(LIBM m)
    endif()
endif()

[davidjade]

Thanks for the explanation and pointers - this helps.

From my reading of the various cmake files, it seems that when using GCC 8.2.0 now, these special libs are no longer being linked in. It seems that the way used to swap out the newlib ROM functions when using PSRAM has changed in ESP-IDF?

If that’s the case, then hopefully this discussion can help anyone else that sees this as there seems to be a lot of confusion on what steps are needed for PSRAM to work. Both good and bad (for those trying the old ways) that it has changed and maybe gotten simpler.