Merge branch 'master' into separate-platform-definiton

Author: hathach

.gitignore

@@ -0,0 +1,2 @@
+.DS_Store
+.idea

README.md

@@ -1,56 +1,78 @@
 # Arduino CI Scripts
 
-This repos contains various scripts and tools related to running
-continuous integration (CI) checks on Arduino Library Repos.
+This repos contains various scripts and tools related to running continuous integration (CI) checks on Arduino Library Repos. The operations include:
+
+* checking formatting using using [clang-format](https://clang.llvm.org/docs/ClangFormat.html),
+* generating documentation from source comments using using [Doxygen](https://www.doxygen.nl/), and
+* building each example in the library for selected targets.
 
 There is an associated guide available here:
 https://learn.adafruit.com/the-well-automated-arduino-library/
 
 ## Adding GitHub Actions to Repo
 
+To run these continuous integration checks on each push, pull-request or [repository dispatch](https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#create-a-repository-dispatch-event) using [GitHub actions](https://github.com/features/actions):  
+
 * Create a folder named `.github/worflows` in the root of the repo.
 * Copy `example_actions.yml` into the above directory and rename it `githubci.yml`.
-* Edit `githubci.yml` and change `PRETTYNAME` to the library repo name.
+* Edit `githubci.yml` and change `PRETTYNAME` to the library repo name. Optionally, delete or comment out steps (using the `#` character), you don't want to include. 
 * Here's an example: [Adafruit_BME280_Library](https://github.com/adafruit/Adafruit_BME280_Library/blob/master/.github/workflows/githubci.yml)
-* These actions will now run automatically on any pull, push, or dispatch.
 
 ## Controlling Test Behavior
 
-The `build_platform.py` script is used to test each `.ino` example in the repo for the
-selected build platforms. The `ALL_PLATFORMS` dictionary contains a listing of all
-available platforms. By default, `main_platforms` is used. Additionally, UF2 files
-of the compiled sketches can be generated for supported platforms. The behavior
-can be controlled using special hidden filenames. These are just empty files
-placed in the root folder:
+The `build_platform.py` script is used to test each `.ino` example in the repo for selected build platforms. The [`ALL_PLATFORMS`](ci-arduino/blob/master/build_platform.py#L54) dictionary contains a listing of all available platforms and selected platform groups. By default, `main_platforms` is used. To select a specific platform or group, replace `main_platforms` in [`githubci.yml`](`example_actions.yml`) with the group or platform name.
+
+Additionally, [UF2 files](https://github.com/microsoft/uf2) of the compiled sketches can be generated for supported platforms. 
+
+### Fine tuning test selection
 
-* `.YOUR_PLATFORM_HERE.test.skip` - Skip the specified platform. All others are tested.
-* `.YOUR_PLATFORM_HERE.test.only` - Test only the specfied platform. All others are skipped.
-* `.YOUR_PLATFORM_HERE.generate` - Generate UF2 of sketch for specified platform (if supported).
+The script behavior can be controlled using special filenames: 
 
-Replace `YOUR_PLATFORM_HERE` in the name with exact text from `ALL_PLATFORMS`.
+* `.PLATFORM_ID.test.skip` - Skip the specified platform. All others are tested.
+* `.PLATFORM_ID.test.only` - Test the specified platform. All others are skipped.
+* `.PLATFORM_ID.generate` - Generate UF2 of sketch for specified platform (if supported).
+
+These are just empty files placed in an example folder. Replace `PLATFORM_ID` in the name with the key from [`ALL_PLATFORMS`](ci-arduino/blob/master/build_platform.py#L54). `metro_m0` from the following line in `build_platform.py`, for example:
+
+```python
+"metro_m0" : ["adafruit:samd:adafruit_metro_m0", "0x68ed2b88", None],
+```
 
-### Examples
+You can use several `.PLATFORM_ID.test.skip` or `.PLATFORM_ID.test.only` to exclude or include multiple platforms. For example:
 
 * To **skip** testing on ESP8266, add a file named `.esp8266.test.skip`
 * To test **only** the Arduino UNO, add a file named `.uno.test.only`
 * To skip all and test **nothing**, add a file named `.none.test.only`
 * To generate UF2s for PyPortal, add a file named `.pyportal.generate`
 
+### Dependencies
+
+Any library dependencies included in the [`library.properties`](https://arduino.github.io/arduino-cli/0.19/library-specification/#libraryproperties-file-format) are automatically installed before the tests are started. To install additional dependencies (e.g., those required for some examples but not the library itself) using [`arduino-cli`](https://arduino.github.io/arduino-cli/0.19/commands/arduino-cli_lib_install/), you could add additional steps to the `githubci.yml` file. For example:
+
+```yaml
+- name: Set configuration
+  run: arduino-cli config set library.enable_unsafe_install true 
+
+- name: Install test dependencies
+  run: arduino-cli lib install --git-url https://github.com/arduino-libraries/Servo --git-url https://github.com/arduino-libraries/Ethernet
+```
+
+Note: you'll only need to enable the [`enable_unsafe_install`](https://arduino.github.io/arduino-cli/0.32/configuration/#configuration-keys) option if you want to identify libraries using urls. This isn't necessary when using the library name. 
+
 ## Formatting Check with Clang
 
-The `run-clang-format.py` script is used to run ClangFormat and check file formatting.
+The `run-clang-format.py` script is used to run [clang-format](https://clang.llvm.org/docs/ClangFormat.html) and check file formatting.
 See [the guide](https://learn.adafruit.com/the-well-automated-arduino-library/formatting-with-clang-format) for details on installing `clang-format` to run formatting locally.
 Even a single extra white space can cause the CI to fail on formatting.
 You can typically just let clang do its thing and edit files in place using:
+
 ```
 clang-format -i File_To_Format.cpp
 ```
 
 ## Documentation with Doxygen
 
-The `doxy_gen_and_deploy.sh` script uses Doxygen to generate and deploy documentation
+The `doxy_gen_and_deploy.sh` script uses [Doxygen](https://www.doxygen.nl/) to generate and deploy documentation
 for the library. Any issues, like missing documentation, will cause the CI to fail.
-See the [the guide](https://learn.adafruit.com/the-well-automated-arduino-library/doxygen)
-for details on installing and running doxygen locally. The guide also has some
-[tips](https://learn.adafruit.com/the-well-automated-arduino-library/doxygen-tips)
-on basic usage of doxygen markup within your code.
+See the [the guide](https://learn.adafruit.com/the-well-automated-arduino-library/doxygen) for details on installing and running doxygen locally. The guide also has some
+[tips](https://learn.adafruit.com/the-well-automated-arduino-library/doxygen-tips) on basic usage of doxygen markup within your code.

all_platforms.py

@@ -18,10 +18,14 @@
     "qtpy_esp32s2" : ["esp32:esp32:adafruit_qtpy_esp32s2", "0xbfdd4eee", None],
     "feather_esp32s2" : ["esp32:esp32:adafruit_feather_esp32s2", "0xbfdd4eee", None],
     "feather_esp32s2_tft" : ["esp32:esp32:adafruit_feather_esp32s2_tft", "0xbfdd4eee", None],
+    "feather_esp32s2_reverse_tft" : ["esp32:esp32:adafruit_feather_esp32s2_reversetft", "0xbfdd4eee", None],
     "feather_esp32s3" : ["esp32:esp32:adafruit_feather_esp32s3_nopsram", "0xc47e5767", None],
     "feather_esp32s3_4mbflash_2mbpsram" : ["esp32:esp32:adafruit_feather_esp32s3", "0xc47e5767", None],
     "feather_esp32s3_tft" : ["esp32:esp32:adafruit_feather_esp32s3_tft", "0xc47e5767", None],
+    "feather_esp32s3_reverse_tft" : ["esp32:esp32:adafruit_feather_esp32s3_reversetft", "0xc47e5767", None],
+    "matrixportal_s3" : ["esp32:esp32:adafruit_matrixportal_esp32s3", "0xc47e5767", None],
     "qtpy_esp32s3" : ["esp32:esp32:adafruit_qtpy_esp32s3_nopsram", "0xc47e5767", None],
+    "qtpy_esp32s3_n4r2" : ["esp32:esp32:adafruit_qtpy_esp32s3_n4r2", "0xc47e5767", None],
     "qtpy_esp32" : ["esp32:esp32:adafruit_qtpy_esp32_pico", None, None],
     "qtpy_esp32c3" : ["esp32:esp32:adafruit_qtpy_esp32c3:FlashMode=qio", None, None],
     # Adafruit AVR
@@ -89,11 +93,19 @@
     "pico_rp2040" : ["rp2040:rp2040:rpipico:freq=125,flash=2097152_0", "0xe48bff56", None],
     "pico_rp2040_tinyusb" : ["rp2040:rp2040:rpipico:flash=2097152_0,freq=125,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
     "picow_rp2040" : ["rp2040:rp2040:rpipicow:flash=2097152_0,freq=125", "0xe48bff56", None],
-    "picow_rp2040_tinyusb" : ["rp2040:rp2040:rpipicow:flash=2097152_0,freq=133,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "picow_rp2040_tinyusb" : ["rp2040:rp2040:rpipicow:flash=2097152_131072,freq=133,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
     "feather_rp2040" : ["rp2040:rp2040:adafruit_feather:freq=125,flash=8388608_0", "0xe48bff56", None],
     "feather_rp2040_tinyusb" : ["rp2040:rp2040:adafruit_feather:flash=8388608_0,freq=125,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "feather_rp2040_rfm" : ["rp2040:rp2040:adafruit_feather_rfm:freq=125,flash=8388608_0", "0xe48bff56", None],
+    "feather_rp2040_rfm_tinyusb" : ["rp2040:rp2040:adafruit_feather_rfm:flash=8388608_0,freq=125,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "feather_rp2040_dvi" : ["rp2040:rp2040:adafruit_feather_dvi:freq=125,flash=8388608_0", "0xe48bff56", None],
+    "feather_rp2040_dvi_tinyusb" : ["rp2040:rp2040:adafruit_feather_dvi:flash=8388608_0,freq=125,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "feather_rp2040_usbhost_tinyusb" : ["rp2040:rp2040:adafruit_feather_usb_host:flash=8388608_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
     "qt2040_trinkey" : ["rp2040:rp2040:adafruit_trinkeyrp2040qt:freq=125,flash=8388608_0", "0xe48bff56", None],
     "qt2040_trinkey_tinyusb" : ["rp2040:rp2040:adafruit_trinkeyrp2040qt:flash=8388608_0,freq=125,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "qt_py_rp2040": ["rp2040:rp2040:adafruit_qtpy:freq=125,flash=8388608_0", "0xe48bff56", None],
+    "qt_py_rp2040_tinyusb": ["rp2040:rp2040:adafruit_qtpy:flash=8388608_0,freq=125,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+
     # Attiny8xy, 16xy, 32xy (SpenceKonde)
     "attiny3217" : ["megaTinyCore:megaavr:atxy7:chip=3217", None, None],
     "attiny3216" : ["megaTinyCore:megaavr:atxy6:chip=3216", None, None],

build_platform.py

@@ -19,6 +19,15 @@
     BUILD_WARN = False
     sys.argv.remove("--no_warn")
 
+# optional timeout argument to extend build time
+# for larger sketches or firmware builds
+BUILD_TIMEOUT = False
+if "--build_timeout" in sys.argv:
+    BUILD_TIMEOUT = True
+    popen_timeout = int(sys.argv[sys.argv.index("--build_timeout") + 1])
+    sys.argv.pop(sys.argv.index("--build_timeout") + 1)
+    sys.argv.remove("--build_timeout")
+
 # add user bin to path!
 BUILD_DIR = ''
 # add user bin to path!
@@ -150,6 +159,12 @@ def run_or_die(cmd, error):
 
 run_or_die("arduino-cli core update-index --additional-urls "+BSP_URLS+
            " > /dev/null", "FAILED to update core indices")
+##### HACK !!!!!!!!!!!!!!!!!!!
+# manual fix for megatinycore URL truncation issue
+# see: https://github.com/arduino/arduino-cli/issues/2345
+#      https://github.com/SpenceKonde/megaTinyCore/issues/1005
+os.system("mv ~/.arduino15/package_drazzy.json ~/.arduino15/package_drazzy.com_index.json")
+##### HACK !!!!!!!!!!!!!!!!!!!!
 print()
 
 ################################ Install dependencies
@@ -249,7 +264,10 @@ def generate_uf2(example_path):
         cmd = ['python3', 'uf2conv.py', input_file, '-c', '-f', family_id, '-b', "0x0000", '-o', output_file]
 
     proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-    r = proc.wait(timeout=60)
+    if BUILD_TIMEOUT:
+        r = proc.wait(timeout=popen_timeout)
+    else:
+        r = proc.wait(timeout=60)
     out = proc.stdout.read()
     err = proc.stderr.read()
     if r == 0 and not err:
@@ -335,7 +353,10 @@ def test_examples_in_folder(folderpath):
         proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,
                                 stderr=subprocess.PIPE)
         try:
-            out, err = proc.communicate(timeout=120)
+            if BUILD_TIMEOUT:
+                out, err = proc.communicate(timeout=popen_timeout)
+            else:
+                out, err = proc.communicate(timeout=120)
             r = proc.returncode
         except:
             proc.kill()

example_actions.yml

@@ -7,11 +7,11 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-    - uses: actions/setup-python@v1
+    - uses: actions/setup-python@v4
       with:
         python-version: '3.x'
-    - uses: actions/checkout@v2
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
+    - uses: actions/checkout@v3
       with:
          repository: adafruit/ci-arduino
          path: ci