A very serious error occurred during the compilation and burning of large programs on ESP32-S3 using PlatformIO

[qkqwork]

Project information: The UI was generated using eez studio and the lvgl framework. Previously developed using Arduino IDE, now attempting to switch to Vscode+PlatformIO development (where the problem occurred)
And I use ESP32-S3-WROOM-1-N16R8-Module to debelop.

Now I have encountered a very troublesome problem:
1.When I used Ardouino IDE to compile and upload before, everything was normal.
2.However, when I compiled and uploaded it on VSCODE, everything seemed normal and IDE did not output any errors or reports
3.But after the upload was successful, my machine did not respond at all. Even worse, when I tried to re upload:
The following is the error message generated by VSCode during the second burning process (when ESP32 has already encountered issues):
(Everything was normal earlier, and now we are burning it):

Writing at 0x000f5050... (82 %)
Writing at 0x000fa745... (85 %)
Writing at 0x000ffcba... (87 %)
Writing at 0x00105326... (89 %)

A serial exception error occurred: Cannot configure port, something went wrong. Original message: OSError(22, '指定不存在的设备。', None, 433)
Note: This error originates from pySerial. It is likely not a problem with esptool, but with the hardware connection or drivers.
For troubleshooting steps visit: https://docs.espressif.com/projects/esptool/en/latest/troubleshooting.html
*** [upload] Error 1

4. At the same time, the normal Arduino IDE cannot upload programs. In the end, I saved this by using LeXin's ESP flashing tool. After I was rescued, the Arduino IDE was functioning normally again, but Platformio was still unavailable
5. Afterwards, I made various attempts, such as modifying configurations, changing partition tables, and so on. It's useless
6. This issue only occurs when compiling and uploading this "large" project using platformIO. When I try to compile and upload small programs(such as "Blink") using platformIO, no problems occur

And this is platformio.ini in my project:

[env:esp32s3]
platform = espressif32
board = esp32-s3-devkitc-1
framework = arduino
monitor_speed = 115200
board_build.arduino.partitions = default_16MB.csv

I've also tried this one:

[env:esp32-s3-devkitc-1]
platform = espressif32
board = esp32-s3-devkitc-1
framework = arduino
monitor_speed = 115200
upload_speed = 921600
upload_protocol = esptool
board_build.partitions = large_spiffs_16MB.csv
board_build.memory_type = qio_opi
board_build.flash_mode = qio
board_build.psram_type = opi
board_upload.flash_size = 16MB
board_upload.maximum_size = 16777216

BUT none of them can prevent problems from happening : (

[Jason2866]

Caused from the old version of esptool.py used from Platformio. This version has a bug which fails to flash S3 with >=16MB flash correctly. The bug is fixed with an actual version of esptool.py

[valeros]

Hi @qkqwork, could you try to upload your binary using the latest esptoolpy? Just add these lines to your platformio.ini:

[env:esp32s3]
platform = espressif32
board = esp32-s3-devkitc-1
framework = arduino
...
; Switch to the latest esptoolpy package
platform_packages = 
  platformio/tool-esptoolpy@~1.40801.0

[qkqwork]

Thank U : )
I added this line to my platformio.ini,
but when I build again,an error occurs:

Traceback (most recent call last):
  File "C:\.platformio\packages\tool-esptoolpy\esptool.py", line 34, in <module>
    import esptool
  File "C:\.platformio\packages\tool-esptoolpy\esptool\__init__.py", line 41, in <module>
    from esptool.bin_image import intel_hex_to_bin
  File "C:\.platformio\packages\tool-esptoolpy\esptool\bin_image.py", line 16, in <module>
    from intelhex import HexRecordError, IntelHex
ModuleNotFoundError: No module named 'intelhex'

*** [.pio\build\esp32-s3-devkitc-1\bootloader.bin] Error 1

It seems that there is something wrong with esptool
How can I solve this problem？Shall I update esptool to the latest version manually?

[qkqwork]

OK，I installed intelhex to my python(C:.platformio\python3),and the error has been solved.
I successfully builded the program.
BUT unfortunately,the problem is still happening.
The first time when i upload it,nothing wrong happened
But the second time when i upload,a similar problem occured:

.......

Uploading .pio\build\esp32-s3-devkitc-1\firmware.bin
esptool.py v4.8.1
Serial port COM8
Connecting....

A serial exception error occurred: Write timeout
Note: This error originates from pySerial. It is likely not a problem with esptool, but with the hardware connection or drivers.
For troubleshooting steps visit: https://docs.espressif.com/projects/esptool/en/latest/troubleshooting.html
*** [upload] Error 1

[valeros]

@Jason2866 could you please shed some light on what version of the esptoolpy is needed? The latest stable 4.8.1 seems to be still broken.

[Jason2866]

@valeros @qkqwork Can you try this build? If this is still not working the issue is something else.

platform_packages = 
  tool-esptoolpy@https://github.com/Jason2866/esptool/releases/download/v4.8.7/esptool.zip

Need to search the commit where it is fixed. The dev version has it for sure included
https://pypi.org/project/esptool/4.9.dev5/

My build is based on commit espressif/esptool@8f1c206

[Jason2866]

Actual ArduinoIDE is using esptool.py v4.9dev3 https://github.com/espressif/arduino-esp32/blob/250c1abf781eba8cec7a524090939c30e05b690c/package/package_esp32_index.template.json#L463

[qkqwork]

Oh,no.
I've added this line
platform_packages = tool-esptoolpy@https://github.com/Jason2866/esptool/releases/download/v4.8.7/esptool.zip
to my platformio.ini.
But the problem is still happening like before:

.......

Uploading .pio\build\esp32-s3-devkitc-1\firmware.bin
esptool.py v4.8.7
Serial port COM8
Connecting....

A serial exception error occurred: Write timeout
Note: This error originates from pySerial. It is likely not a problem with esptool, but with the hardware connection or drivers.
For troubleshooting steps visit: https://docs.espressif.com/projects/esptool/en/latest/troubleshooting.html
*** [upload] Error 1

And this is the error information that the esp32 print in serial monitor:

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x4204ca1a
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x4bc
load:0x403c9700,len:0xbd8
load:0x403cc700,len:0x2a0c
entry 0x403c98d0
Guru Meditation Error: Core  1 panic'ed (StoreProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x42026b30  PS      : 0x00060c30  A0      : 0x82026bfc  A1      : 0x3fcebba0  
A2      : 0x00000010  A3      : 0x00000000  A4      : 0x60004000  A5      : 0x0000000b  
A6      : 0x000000ff  A7      : 0x3fca5718  A8      : 0x08000000  A9      : 0x3fcebb70  
A10     : 0x3fca6434  A11     : 0x00000001  A12     : 0xffffffff  A13     : 0x00008000  
A14     : 0x00000000  A15     : 0x3fc935a8  SAR     : 0x00000010  EXCCAUSE: 0x0000001d  
EXCVADDR: 0x00000010  LBEG    : 0x4203a900  LEND    : 0x4203a964  LCOUNT  : 0x00000000  


Backtrace: 0x42026b2d:0x3fcebba0 0x42026bf9:0x3fcebbd0 0x42026c04:0x3fcebbf0 0x42001db6:0x3fcebc10 0x42001e23:0x3fcebc30 0x42029082:0x3fcebc50




ELF file SHA256: e882915179a927cd

Rebooting...

Is my operation incorrect？

[Jason2866]

The com port serial stuff is not straight forward. It looks like the installed drivers for the USB serial chip are not working correctly or a not compatible (outdated?) driver is installed.
It is possible too, the Windows driver part is corrupted.

First step I would do. Disconnect the esp32. Deinstall all additional installed USB serial drivers. Restart Windows.
Get latest actual USB serial chipset drivers from Silab and WCH. Install them. Restart Windows.
Try again to flash.

[valeros]

@qkqwork Can you show a screenshot of your board settings used in Arduino IDE?

[qkqwork]

[qkqwork]

The Partition Scheme is not the same,but i've tried both of them : )

[valeros]

@qkqwork I noticed you enabled the USB CDC On Boot option in Arduino IDE, but it's not enabled in your platformio.ini, is that correct?

[qkqwork]

I can provide you with the project source code so that you can try to reproduce this issue.
MyProj.zip

[qkqwork]

I added

build_flags = 
    -D ARDUINO_USB_MODE=1
    -D ARDUINO_USB_CDC_ON_BOOT=1

in platformio.ini
But it doesn't solve the bug : (

[Jason2866]

@qkqwork Can you upload your complete project to github?

[qkqwork]

This just is my complete project of this part
But I'm not sure whether I packed all of the files and libs.You can tell me what file is missing and I'll upload again.

[qkqwork]

https://github.com/qkqwork/AnIssueInESP32S3/releases/download/1/MyProj.zip

[Jason2866]

Compiling and flashing does work perfectly fine on my Apple M1. The code itself crashes, but thats a different story.

[Jason2866]

Looks like your PC software setup is "broken" somewhere.

[qkqwork]

OK.Maybe I need to reinstall PlatformIO

[qkqwork]

Can you upload it again successfully after you upload it for the first time.