Demo http S3 generate presigned url - generate not only GET but also PUT (for http S3 upload demo) (#1933)

presigned URL demo - generate not only GET but also PUT (for S3 upload demo)

updated documentation

Update demos/http/http_demo_s3_generate_presigned_url/http_demo_s3_generate_presigned_url.c

Author: giuspen

.github/links_allowlist.txt

@@ -4,3 +4,4 @@ https://www.somewebsite.com/path/to/item.txt?optionalquery=stuff
 https://dummy-url.com/ota.bin
 file://template.json
 file://policy.json
+https://www.gnu.org/software/complexity/manual/complexity.html

.github/workflows/ci.yml

@@ -197,30 +197,10 @@ jobs:
     runs-on: ubuntu-20.04
     steps:
       - uses: actions/checkout@v2
-      - name: Install Uncrustify
-        run: sudo apt-get install uncrustify
-      - name: Check Formatting With Uncrustify
-        run: find . -iname "*.[hc]" -exec uncrustify --check -c tools/uncrustify.cfg {} +
-      - name: Check For Trailing Whitespace
-        run: |
-          set +e
-          grep --exclude="README.md" -rnI -e "[[:blank:]]$" .
-          if [ "$?" = "0" ]; then
-            echo "Files have trailing whitespace."
-            exit 1
-          else
-            exit 0
-          fi
-      - name: Check for CRLF
-        run: |
-          set +e
-          find . -path ./.git -prune -o -exec file {} + |  grep "CRLF"
-          if [ "$?" = "0" ]; then
-            echo "Files have CRLF line endings."
-            exit 1
-          else
-            exit 0
-          fi
+      - name: Check formatting
+        uses: FreeRTOS/CI-CD-Github-Actions/formatting@main
+        with:
+          path: ./
   doxygen:
     runs-on: ubuntu-latest
     steps:
@@ -230,7 +210,7 @@ jobs:
       - name: Install Doxygen
         run: |
           wget -qO- "https://sourceforge.net/projects/doxygen/files/rel-1.9.6/doxygen-1.9.6.linux.bin.tar.gz/download" | sudo tar --strip-components=1 -xz -C /usr/local
-          sudo apt-get install -y libclang-11-dev libclang-cpp11 graphviz
+          sudo apt-get install -y libclang-18-dev libclang-cpp18 graphviz
       - name: Install Python3
         uses: actions/setup-python@v2
         with:

README.md

@@ -470,7 +470,7 @@ In order to set these configurations manually, edit `demo_config.h` in `demos/ht
 * Set `S3_PRESIGNED_GET_URL` to a S3 presigned URL with GET access.
 * Set `S3_PRESIGNED_PUT_URL` to a S3 presigned URL with PUT access.
 
-You can generate the presigned urls using [demos/http/common/src/presigned_urls_gen.py](demos/http/common/src/presigned_urls_gen.py). More info can be found [here](demos/http/common/src/README.md).
+You can generate the presigned urls using [demos/http/common/src/presigned_urls_gen.py](demos/http/common/src/presigned_urls_gen.py) (more info can be found [here](demos/http/common/src/README.md)) or [demos/http/http_demo_s3_generate_presigned_url/README.md](demos/http/http_demo_s3_generate_presigned_url/README.md).
 
 ####  Configure S3 Download HTTP Demo using SigV4 Library:
 
@@ -503,13 +503,13 @@ The following creates a job that specifies a Linux Kernel link for downloading.
         --document '{"url":"https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.8.5.tar.xz"}'
 ```
 3. Run the demo using this command:
-```    
+```
 ./build/bin/jobs_demo_mosquitto \
 -n <thing-name> \
 -h <aws-iot endpoint> \
 --certfile <device certificate of the thing> \
 --keyfile <private key of the thing>
-``` 
+```
 Note: Replace the placeholders in angle brackets with your specific information.
 
 
@@ -670,7 +670,7 @@ First, install Tunnelmole. On Linux, Mac and Windows Subsystem for Linux, use
 ```sh
 curl -O https://tunnelmole.com/sh/install.sh && sudo bash install.sh
 ```
-The above may not function correctly on ARM machines like mac. As an alternative, you can clone the repository and build the application from its source code. 
+The above may not function correctly on ARM machines like mac. As an alternative, you can clone the repository and build the application from its source code.
 For Windows without WSL, [download tmole.exe](https://tunnelmole.com/downloads/tmole.exe) and add it to your [PATH](https://www.wikihow.com/Change-the-PATH-Environment-Variable-on-Windows).
 
 Then run `tmole 80`

demos/fleet_provisioning/fleet_provisioning_keys_cert/fleet_provisioning_keys_cert_demo.c

@@ -772,57 +772,57 @@ int main( int argc,
         LogInfo( ( "Demo completed successfully." ) );
 
         #if defined( DOWNLOADED_CERT_WRITE_PATH )
+        {
+            int fd = open( DOWNLOADED_CERT_WRITE_PATH, O_CREAT | O_WRONLY | O_TRUNC, S_IRUSR | S_IWUSR );
+
+            if( -1 != fd )
             {
-                int fd = open( DOWNLOADED_CERT_WRITE_PATH, O_CREAT | O_WRONLY | O_TRUNC, S_IRUSR | S_IWUSR );
+                const ssize_t writtenBytes = write( fd, certificate, certificateLength );
 
-                if( -1 != fd )
+                if( writtenBytes == certificateLength )
                 {
-                    const ssize_t writtenBytes = write( fd, certificate, certificateLength );
-
-                    if( writtenBytes == certificateLength )
-                    {
-                        LogInfo( ( "Written %s successfully.", DOWNLOADED_CERT_WRITE_PATH ) );
-                    }
-                    else
-                    {
-                        LogError( ( "Could not write to %s. Error: %s.", DOWNLOADED_CERT_WRITE_PATH, strerror( errno ) ) );
-                    }
-
-                    close( fd );
+                    LogInfo( ( "Written %s successfully.", DOWNLOADED_CERT_WRITE_PATH ) );
                 }
                 else
                 {
-                    LogError( ( "Could not open %s. Error: %s.", DOWNLOADED_CERT_WRITE_PATH, strerror( errno ) ) );
+                    LogError( ( "Could not write to %s. Error: %s.", DOWNLOADED_CERT_WRITE_PATH, strerror( errno ) ) );
                 }
+
+                close( fd );
             }
+            else
+            {
+                LogError( ( "Could not open %s. Error: %s.", DOWNLOADED_CERT_WRITE_PATH, strerror( errno ) ) );
+            }
+        }
         #else /* if defined( DOWNLOADED_CERT_WRITE_PATH ) */
             LogInfo( ( "NOTE: define DOWNLOADED_CERT_WRITE_PATH in order to have the certificate written to disk." ) );
         #endif // DOWNLOADED_CERT_WRITE_PATH
 
         #if defined( DOWNLOADED_PRIVATE_KEY_WRITE_PATH )
+        {
+            int fd = open( DOWNLOADED_PRIVATE_KEY_WRITE_PATH, O_CREAT | O_WRONLY | O_TRUNC, S_IRUSR | S_IWUSR );
+
+            if( -1 != fd )
             {
-                int fd = open( DOWNLOADED_PRIVATE_KEY_WRITE_PATH, O_CREAT | O_WRONLY | O_TRUNC, S_IRUSR | S_IWUSR );
+                const ssize_t writtenBytes = write( fd, privatekey, privatekeyLength );
 
-                if( -1 != fd )
+                if( writtenBytes == privatekeyLength )
                 {
-                    const ssize_t writtenBytes = write( fd, privatekey, privatekeyLength );
-
-                    if( writtenBytes == privatekeyLength )
-                    {
-                        LogInfo( ( "Written %s successfully.", DOWNLOADED_PRIVATE_KEY_WRITE_PATH ) );
-                    }
-                    else
-                    {
-                        LogError( ( "Could not write to %s. Error: %s.", DOWNLOADED_PRIVATE_KEY_WRITE_PATH, strerror( errno ) ) );
-                    }
-
-                    close( fd );
+                    LogInfo( ( "Written %s successfully.", DOWNLOADED_PRIVATE_KEY_WRITE_PATH ) );
                 }
                 else
                 {
-                    LogError( ( "Could not open %s. Error: %s.", DOWNLOADED_PRIVATE_KEY_WRITE_PATH, strerror( errno ) ) );
+                    LogError( ( "Could not write to %s. Error: %s.", DOWNLOADED_PRIVATE_KEY_WRITE_PATH, strerror( errno ) ) );
                 }
+
+                close( fd );
             }
+            else
+            {
+                LogError( ( "Could not open %s. Error: %s.", DOWNLOADED_PRIVATE_KEY_WRITE_PATH, strerror( errno ) ) );
+            }
+        }
         #else /* if defined( DOWNLOADED_PRIVATE_KEY_WRITE_PATH ) */
             LogInfo( ( "NOTE: define DOWNLOADED_PRIVATE_KEY_WRITE_PATH in order to have the private key written to disk." ) );
         #endif // DOWNLOADED_PRIVATE_KEY_WRITE_PATH

demos/fleet_provisioning/fleet_provisioning_with_csr/fleet_provisioning_with_csr_demo.c

@@ -783,29 +783,29 @@ int main( int argc,
         LogInfo( ( "Demo completed successfully." ) );
 
         #if defined( DOWNLOADED_CERT_WRITE_PATH )
+        {
+            int fd = open( DOWNLOADED_CERT_WRITE_PATH, O_CREAT | O_WRONLY | O_TRUNC, S_IRUSR | S_IWUSR );
+
+            if( -1 != fd )
             {
-                int fd = open( DOWNLOADED_CERT_WRITE_PATH, O_CREAT | O_WRONLY | O_TRUNC, S_IRUSR | S_IWUSR );
+                const ssize_t writtenBytes = write( fd, certificate, certificateLength );
 
-                if( -1 != fd )
+                if( writtenBytes == certificateLength )
                 {
-                    const ssize_t writtenBytes = write( fd, certificate, certificateLength );
-
-                    if( writtenBytes == certificateLength )
-                    {
-                        LogInfo( ( "Written %s successfully.", DOWNLOADED_CERT_WRITE_PATH ) );
-                    }
-                    else
-                    {
-                        LogError( ( "Could not write to %s. Error: %s.", DOWNLOADED_CERT_WRITE_PATH, strerror( errno ) ) );
-                    }
-
-                    close( fd );
+                    LogInfo( ( "Written %s successfully.", DOWNLOADED_CERT_WRITE_PATH ) );
                 }
                 else
                 {
-                    LogError( ( "Could not open %s. Error: %s.", DOWNLOADED_CERT_WRITE_PATH, strerror( errno ) ) );
+                    LogError( ( "Could not write to %s. Error: %s.", DOWNLOADED_CERT_WRITE_PATH, strerror( errno ) ) );
                 }
+
+                close( fd );
             }
+            else
+            {
+                LogError( ( "Could not open %s. Error: %s.", DOWNLOADED_CERT_WRITE_PATH, strerror( errno ) ) );
+            }
+        }
         #else /* if defined( DOWNLOADED_CERT_WRITE_PATH ) */
             LogInfo( ( "NOTE: define DOWNLOADED_CERT_WRITE_PATH in order to have the certificate written to disk." ) );
         #endif // DOWNLOADED_CERT_WRITE_PATH

demos/greengrass/greengrass_demo_local_auth/README.md

@@ -52,7 +52,7 @@ Next you will need to set up a Root CA for your Greengrass device.
 
 On the Greengrass core, run the following:
 
-1. Create private key for the CA certificate 
+1. Create private key for the CA certificate
 ```sh
 openssl genrsa -out ca.key 2048
 ```

demos/http/common/src/README.md

@@ -16,9 +16,9 @@
    ```
 
 1. You will need an AWS Account with S3 access before beginning. You must install and configure the AWS CLI in order to
-   use this script.  
-   For information on AWS S3 please see: https://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html  
-   For AWS CLI installation information please see: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html  
+   use this script.
+   For information on AWS S3 please see: https://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html
+   For AWS CLI installation information please see: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
    For AWS CLI configuration information please see: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html
 
    ```sh

demos/http/http_demo_s3_download/README.md

@@ -4,9 +4,9 @@ Following steps needs to be followed to configure HTTP S3 Download Demo to use S
 
 ### Prerequisites
 
-1. You will need an AWS Account with S3 access before beginning. You must be familiar with AWS IoT and IAM to perform steps using the AWS CLI. You must install and configure the AWS CLI in order to follow the steps.  
-   For information on AWS S3 please see: https://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html  
-   For AWS CLI installation information please see: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html  
+1. You will need an AWS Account with S3 access before beginning. You must be familiar with AWS IoT and IAM to perform steps using the AWS CLI. You must install and configure the AWS CLI in order to follow the steps.
+   For information on AWS S3 please see: https://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html
+   For AWS CLI installation information please see: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
    For AWS CLI configuration information please see: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html
 
    ```sh
@@ -15,7 +15,7 @@ Following steps needs to be followed to configure HTTP S3 Download Demo to use S
 
 ### Detailed Steps
 
-#### 1. Create an AWS IoT thing: 
+#### 1. Create an AWS IoT thing:
 
 You may utilize an already existing AWS IoT Thing or create a new one in the IoT Core section of the AWS Management Console UI.
 
@@ -37,8 +37,8 @@ Run the following command in the AWS CLI to attach the device certificate to you
 ```sh
 aws iot attach-thing-principal --thing-name device_thing_name --principal <certificate-arn>
 ```
-    
-#### 3. Configure an IAM role: 
+
+#### 3. Configure an IAM role:
 
 Next, configure an IAM role in your AWS account that will be assumed by the credentials provider on behalf of your device. You are required to associate two policies with the role: a trust policy that controls who can assume the role, and an access policy that controls which actions can be performed on which resources by assuming the role.
 
@@ -108,8 +108,8 @@ Now, run the following command to attach the policy to the IAM user.
 aws iam attach-user-policy --policy-arn arn:aws:iam::<your_aws_account_id>:policy/passrolepermission --user-name <user_name>
 ```
 
-#### 4. Create a role alias: 
-         
+#### 4. Create a role alias:
+
 Now that you have configured the IAM role, you will create a role alias with AWS IoT. You must provide the following pieces of information when creating a role alias:
 
 RoleAlias: This is the primary key of the role alias data model and hence a mandatory attribute. It is a string; the minimum length is 1 character, and the maximum length is 128 characters.
@@ -120,7 +120,7 @@ Run the following command in the AWS CLI to create a role alias. Use the credent
 aws iot create-role-alias --role-alias name-s3-access-role-alias --role-arn arn:aws:iam::<your_aws_account_id>:role/s3-access-role --credential-duration-seconds 3600
 ```
 
-#### 5. Attach a policy: 
+#### 5. Attach a policy:
 You created and registered a certificate with AWS IoT earlier for successful authentication of your device. Now, you need to create and attach a policy to the certificate to authorize the request for the security token.
 ```
 {
@@ -143,8 +143,8 @@ Use the following command to attach the policy with the certificate you register
 aws iot attach-policy --policy-name Thing_Policy_Name --target <certificate-arn>
 ```
 
-#### 6. Request a security token: 
-        
+#### 6. Request a security token:
+
 Make an HTTPS request to the credentials provider to fetch a security token. You have to supply the following information:
 
 Certificate and key pair: Because this is an HTTP request over TLS mutual authentication, you have to provide the certificate and the corresponding key pair to your client while making the request. Use the same certificate and key pair that you used during certificate registration with AWS IoT.
@@ -174,13 +174,13 @@ The following is sample output of the describe-endpoint command. It contains the
 
 #### 8. After the following the above steps, configure the below macros in `demo_config.h`.
 ```c
-#define AWS_IOT_THING_NAME                      "Name of IOT Thing that you provided in STEP 1" 
+#define AWS_IOT_THING_NAME                      "Name of IOT Thing that you provided in STEP 1"
 #define AWS_IOT_CREDENTIAL_PROVIDER_ROLE        "Name of ROLE ALIAS that you provided in STEP 4"
 #define AWS_S3_BUCKET_NAME                      "Name of Bucket that contains the object that needs to be downloaded"
 #define AWS_S3_BUCKET_REGION                    "Region where Bucket is located"
 #define AWS_S3_OBJECT_NAME                      "Name of object that needs to be downloaded from AWS S3"
 ```
-         
+
 ### Parameters
 
 #### device_thing_name