Misleading documentation for --input-file flag of upload command

[Kabouik]

Describe the problem

I need to upload precompiled binaries but I am facing the following issue:

To reproduce

$ arduino-cli upload --port /dev/ttyUSB0 --fqbn Heltec-esp32:esp32:heltec_wireless_stick_lite_V3 --input-file /path/to/my/precompiled/files//lora_lte_node_V2.15_v1_40MHz.bin`

usage: esptool write_flash [-h] [--erase-all]
                           [--flash_freq {keep,80m,60m,48m,40m,30m,26m,24m,20m,16m,15m,12m}]
                           [--flash_mode {keep,qio,qout,dio,dout}]
                           [--flash_size {detect,keep,256KB,512KB,1MB,2MB,2MB-c1,4MB,4MB-c1,8MB,16MB,32MB,64MB,128MB}]
                           [--spi-connection SPI_CONNECTION] [--no-progress]
                           [--verify] [--encrypt]
                           [--encrypt-files <address> <filename> [<address> <filename> ...]]
                           [--ignore-flash-encryption-efuse-setting] [--force]
                           [--compress | --no-compress]
                           <address> <filename> [<address> <filename> ...]
esptool write_flash: error: argument <address> <filename>: [Errno 2] No such file or directory: '/path/to/my/precompiled/files/lora_lte_node_V2.15_v1_40MHz.bootloader.bin'
Failed uploading: uploading error: exit status 2

Note how the filename differs in the error and in the command I used. If I copy the file to a new name to add the missing "bootloader" part that arduino-cli expects (and keep the file with its original name alongside the new one too), then it tries to add this suffix again (i.e., "bootloader.bootloader") when I try to upload with the new name, or keeps failing to find a suitable file even if I try to upload with the old name.

What am I doing wrong?

Expected behavior

The supplied binary filename should be used to upload to the board.

Arduino CLI version

Version: 0.35.3 Commit: 95cfd65 Date: 2024-02-19T13:24:24Z

Operating system

Linux

Operating system version

Debian 6.11.11 distrobox

Additional context

No response

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the nightly build
• My report contains all necessary details

[Kabouik]

Using esptool to flash just the .bin file I have without the bootloader, partition and boot_app0 files seems to do what I want:

python3 "$HOME/.arduino15/packages/Heltec-esp32/tools/esptool_py/4.6/esptool.py" --chip esp32s3 --port "/dev/ttyUSB0" --baud 921600  --before default_reset --after hard_reset write
_flash  -z --flash_mode keep --flash_freq keep --flash_size keep 0x10000 "./lora_lte_node_V2.15_v1_40MHz.bin"

arduino-cli upload's --input-file argument being singular, also in the help for this command, suggested that only one file was necessary, but maybe I am misusing it.

[per1234]

Thanks for your report @Kabouik.

arduino-cli upload's --input-file argument being singular, also in the help for this command, suggested that only one file was necessary, but maybe I am misusing it.

Yes, this is definitely misleading. The documentation needs to be corrected:

arduino-cli/internal/cli/upload/upload.go

Line 75 in a2eebcd

uploadCommand.Flags().StringVarP(&importFile, "input-file", "i", "", i18n.Tr("Binary file to upload."))

Despite what the documentation claims, Arduino CLI does not necessarily upload the file at the path passed to the flag. What it does instead is extract two separate things from that path, which are then provided to the boards platform for use in the upload (or program if the --programmer flag is also passed) pattern (the template from which the upload command line is generated):

• Parent folder -> build.path
• Filename without extension -> build.project_name

There is no provision in the Arduino boards platform framework for explicitly uploading a single binary file. The platform developers will configure their upload pattern with the expectation that it is used in combination with the files generated by arduino-cli compile, and thus the pattern might reference multiple binary files. That is the case with the esp32:esp32 platform:

https://github.com/espressif/arduino-esp32/blob/3.1.1/platform.txt#L288-L289

tools.esptool_py.upload.pattern_args=--chip {build.mcu} --port "{serial.port}" --baud {upload.speed} {upload.flags} --before default_reset --after hard_reset write_flash {upload.erase_cmd} -z --flash_mode keep --flash_freq keep --flash_size keep {build.bootloader_addr} "{build.path}/{build.project_name}.bootloader.bin" 0x8000 "{build.path}/{build.project_name}.partitions.bin" 0xe000 "{runtime.platform.path}/tools/partitions/boot_app0.bin" 0x10000 "{build.path}/{build.project_name}.bin" {upload.extra_flags}
tools.esptool_py.upload.pattern="{path}/{cmd}" {upload.pattern_args}

When you compile a sketch for a board of the esp32:esp32 platform, it will generate all the files the upload pattern references and place them in the {build.path} folder. So arduino-cli upload --input-file should work fine (even if not in the way the user expects) when passed the path of the primary binary file as generated by arduino-cli compile. But this may not be the case if you are trying to upload an arbitrary binary file that you obtained somewhere (which is likely to be a common expected use case for this flag).

Unless the provided binary (or group of binaries) were specifically intended for use with arduino-cli upload --input-file, you should instead use the upload tool directly instead of using Arduino CLI, as you did with esptool.

[Kabouik]

Thanks for the detailed answer and explanation.

Just to elaborate on my usecase and why I was trying to upload precompiled binaries:

The code was developed and tested with a text editor and plugins that manage everything Arduino-related, including compiling and uploading. This editor and plugins are not compatible with my OS, and I have my own habits with another editor anyway, so I ended up writing extra documentaiton in a fork for compiling and uploading using arduino-cli, also killing two birds with one stone as this should make it more reproducible (both for environment setup and automation).

I can compile the code with arduino-cli and upload right away just fine, but we've encountered unexpected behaviours of our devices and were wondering if the binaries compiled using either solutions were identical (one reason is I may be missing parameters to set the CPU frequency at compilation with the CLI method), and if this could be a potential cause of the issues. I guess the best would just be to compare the checksum of the files until I find all arduino-cli flags that allow me to set everything exactly like the binaries compiled by the developer..

[cmaglie]

@Kabouik maybe the --output-dir flag in compile and the --input-dir in upload may be better for your use case?

$ arduino-cli compile -h | grep output-
      --output-dir string                     Save build artifacts in this directory.
$ arduino-cli upload -h | grep input
      --input-dir string              Directory containing binaries to upload.
  -i, --input-file string             Binary file to upload.

If you compile with the --output-dir PATH flag set, the artifacts needed for the upload will be saved in the given PATH, you can provide the same PATH with --input-dir to upload.