Defines added in extra_script are lost when using platform_packages

• PlatformIO Core.

Configuration

Operating system: Windows (VS code)

PlatformIO Version 4.1.1b3

Description of problem

I use a Python script to set/add defines before building the project.

When using the “old” way to define a platform, the defines set in the Python script are present at compile time. When using platform + platform_packages, they are lost.

This is the bug report of what I noticed here: https://github.com/platformio/platformio-core/issues/3239#issuecomment-558358080

Working:

platform = https://github.com/platformio/platform-espressif8266.git#feature/stage

Not working:

platform                  = espressif8266
platform_packages         =  
    framework-arduinoespressif8266 @ https://github.com/esp8266/Arduino.git#2.6.1

If problems with PlatformIO Build System:

The content of platformio.ini:

[platformio]

[esp82xx_defaults]
build_flags               = -D BUILD_GIT='"${sysenv.TRAVIS_TAG}"'
                            -D NDEBUG
                            -lstdc++ -lsupc++
                            -mtarget-align
                            -DPIO_FRAMEWORK_ARDUINO_LWIP2_LOW_MEMORY
                            -DVTABLES_IN_FLASH
                            -DPUYA_SUPPORT=1

[esp82xx_2_5_x]
build_flags               = -D BUILD_GIT='"${sysenv.TRAVIS_TAG}"'
                            -DNDEBUG
                            -mtarget-align
                            -DVTABLES_IN_FLASH
                            -fno-exceptions
                            -lstdc++
                            -DPIO_FRAMEWORK_ARDUINO_LWIP2_LOW_MEMORY_LOW_FLASH
                            -DPUYA_SUPPORT=1
                            -DCORE_POST_2_5_0

[esp82xx_2_6_x]
build_flags               = ${esp82xx_2_5_x.build_flags} 
                            -O2
                            -DBEARSSL_SSL_BASIC
                            -DCORE_POST_2_6_0 


[core_2_6_0]
platform                  = https://github.com/Jason2866/platform-espressif8266.git#core_2_6_0
build_flags               = ${esp82xx_2_6_x.build_flags} 
                            -DPIO_FRAMEWORK_ARDUINO_ESPRESSIF_SDK22x_190703

[core_2_6_1]
platform                  = https://github.com/Jason2866/platform-espressif8266.git#core_2_6_1
build_flags               = ${esp82xx_2_6_x.build_flags} 
                            -DPIO_FRAMEWORK_ARDUINO_ESPRESSIF_SDK22x_190703

[core_stage]
platform                  = https://github.com/platformio/platform-espressif8266.git#feature/stage
build_flags               = ${esp82xx_2_6_x.build_flags} -DARDUINO_ESP8266_RELEASE='"2.6.0-dev stage"'


[core_2_6_1_wifimesh]
platform                  = espressif8266
;platform_packages         = 
;    framework-arduinoespressif8266 @ https://github.com/TD-er/Arduino.git#wifi_mesh_update_2.2
platform_packages         =  
    framework-arduinoespressif8266 @ https://github.com/esp8266/Arduino.git#2.6.1
build_flags               = ${esp82xx_2_6_x.build_flags} 
                            -DPIO_FRAMEWORK_ARDUINO_ESPRESSIF_SDK22x_190703

[common]
board_build.f_cpu         = 80000000L
build_flags               = -DHTTPCLIENT_1_1_COMPATIBLE=0
build_unflags             = -DDEBUG_ESP_PORT
lib_deps                  = https://github.com/TD-er/ESPEasySerial.git
lib_ignore                = ESP32_ping, ESP32WebServer, IRremoteESP8266, HeatpumpIR, SD(esp8266), SDFS
;lib_ignore                = ESP32_ping, ESP32WebServer, IRremoteESP8266, HeatpumpIR
lib_ldf_mode              = chain
lib_archive               = false
upload_speed              = 115200
framework                 = arduino
board                     = esp12e
monitor_speed             = 115200
;targets                   = size, checkprogsize
targets                   =

[env]
framework                 = ${common.framework}
lib_ldf_mode              = ${common.lib_ldf_mode}
lib_archive               = ${common.lib_archive}
lib_deps                  = ${common.lib_deps}
lib_ignore                = ${common.lib_ignore}
upload_speed              = ${common.upload_speed}
monitor_speed             = ${common.monitor_speed}
targets                   = ${common.targets}


[regular_platform]
build_unflags             =
build_flags               = ${core_2_5_2.build_flags} ${common.build_flags}
platform                  = ${core_2_5_2.platform}

[beta_platform]
build_unflags             =
build_flags               = ${core_stage.build_flags} ${common.build_flags}
platform                  = ${core_stage.platform}


[esp8266_4M1M]
board                     = esp12e
board_build.flash_mode    = dio
board_upload.maximum_size = 1044464
board_build.f_cpu         = ${common.board_build.f_cpu}
build_unflags             = ${regular_platform.build_unflags}
build_flags               = -Wl,-Tesp8266.flash.4m1m.ld

; Environment for very simple build tests to assist in reporting issues.
[env:MCVE_ESP8266_4M1M]
platform                  = ${beta_platform.platform}
board                     = ${esp8266_4M1M.board}
board_build.f_cpu         = ${esp8266_4M1M.board_build.f_cpu}
board_build.flash_mode    = ${esp8266_4M1M.board_build.flash_mode}
build_unflags             = ${esp8266_4M1M.build_unflags}
build_flags               = ${esp8266_4M1M.build_flags} ${beta_platform.build_flags} -DPLUGIN_BUILD_CUSTOM
lib_ignore                = ESP32_ping, ESP32WebServer

extra_scripts             = pre:pre_extra_script_mcve.py

[env:spec_wifimesh_ESP8266_4M1M]
extends                   = esp8266_4M1M
framework                 = arduino
build_unflags             =
platform                  = ${core_2_6_1_wifimesh.platform}
platform_packages         = ${core_2_6_1_wifimesh.platform_packages}
build_flags               = ${core_2_6_1_wifimesh.build_flags} 
                            ${esp8266_4M1M.build_flags} 
                            -DPLUGIN_BUILD_CUSTOM
lib_ignore                = ESP32_ping, ESP32WebServer, IRremoteESP8266, HeatpumpIR, SD(esp8266), SDFS
extra_scripts             = pre:pre_extra_script_mcve.py

The content of pre_extra_script_mcve.py:

Import("env")
import os

env.Append(CPPDEFINES=[
  "PIO_FRAMEWORK_ARDUINO_ESPRESSIF_SDK22x_190703",
  "ESPEASY_MCVE_BUILD"
  # ,"NO_HTTP_UPDATER"
  # ,("WEBSERVER_RULES_DEBUG", "0")
])
if os.path.isfile('src/Custom.h'):
  env['CPPDEFINES'].append("USE_CUSTOM_H")
else:
  env['CPPDEFINES'].extend([
        "USES_P001",  # Switch
        "USE_SETTINGS_ARCHIVE"
  ])

print(env['CPPDEFINES'])

Source file to reproduce issue:

#include <Arduino.h>

void setup() {

  #ifdef USES_P001
  #warning "USES_P001 defined"
  #else
    #warning "USES_P001 NOT defined"
  #endif

}

void loop() {

}

Additional info

Build output: (first line is the print statement of the Python script)

['PIO_FRAMEWORK_ARDUINO_ESPRESSIF_SDK22x_190703', 'ESPEASY_MCVE_BUILD', 'USES_P001', 'USE_SETTINGS_ARCHIVE']
...
Compiling .pio\build\MCVE_ESP8266_4M1M\src\main.cpp.o
src\main.cpp:14:4: warning: #warning "USES_P001 defined" [-Wcpp]
   #warning "USES_P001 defined"
    ^

['PIO_FRAMEWORK_ARDUINO_ESPRESSIF_SDK22x_190703', 'ESPEASY_MCVE_BUILD', 'USES_P001', 'USE_SETTINGS_ARCHIVE']
...
Compiling .pio\build\spec_wifimesh_ESP8266_4M1M\FrameworkArduino\MD5Builder.cpp.o
src\main.cpp:16:6: warning: #warning "USES_P001 NOT defined" [-Wcpp]
     #warning "USES_P001 NOT defined"
      ^

[Jason2866]

platform = espressif8266@2.3.1 Will force to use core 2.6.2 and latest build chain

[ivankravets]

@TD-er could you try to debug this issue on your machine step-by-step? It’s very difficult to catch errors if you did a lot of changes.

1. Try using official dev/platform and framework from our registry

1. They try to use platform_packages = ...
2. etc, etc