Merge branch 'main' into fix_for_transformers_4_37

.github/ISSUE_TEMPLATE/bug-report.yml

@@ -0,0 +1,73 @@
+name: "\U0001F41B Bug Report"
+description: Submit a bug report to help us improve Optimum Neuron
+labels: [ "bug" ]
+body:
+  - type: textarea
+    id: system-info
+    attributes:
+      label: System Info
+      description: Please share your system info with us. You can run the command optimum-cli env and copy-paste its output below.
+      render: shell
+      placeholder: optimum-neuron version, optimum version, platform, python version, ...
+    validations:
+      required: true
+
+  - type: textarea
+    id: who-can-help
+    attributes:
+      label: Who can help?
+      description: |
+        Your issue will be replied to more quickly if you can figure out the right person to tag with @
+        If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
+        Please tag fewer than 3 people.
+
+        - Neuron Training: `@michaelbenayoun`
+        - Export models to Neuron format: `@JingyaHuang`
+        - Inference: `@dacorvo`, `@JingyaHuang`
+        - TGI: `@dacorvo`
+        - Pipelines: `@philschmid`, `@JingyaHuang`
+
+      placeholder: "@Username ..."
+
+  - type: checkboxes
+    id: information-scripts-examples
+    attributes:
+      label: Information
+      description: 'The problem arises when using:'
+      options:
+        - label: "The official example scripts"
+        - label: "My own modified scripts"
+
+  - type: checkboxes
+    id: information-tasks
+    attributes:
+      label: Tasks
+      description: "The tasks I am working on are:"
+      options:
+        - label: "An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)"
+        - label: "My own task or dataset (give details below)"
+
+  - type: textarea
+    id: reproduction
+    validations:
+      required: true
+    attributes:
+      label: Reproduction (minimal, reproducible, runnable)
+      description: |
+        Please provide a code sample that reproduces the problem you ran into. It can be a Colab link or just a code snippet.
+        If you have code snippets, error messages, stack traces please provide them here as well.
+        Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
+        Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.
+        Providing a **minimal**, **reproducible** reproduction using a **publicly available model** significantly increase the chances of a fix in a timely manner.
+
+      placeholder: |
+        Providing a minimal, reproducible reproduction using a publicly available model significantly increase the chances of a fix in a timely manner.
+
+
+  - type: textarea
+    id: expected-behavior
+    validations:
+      required: true
+    attributes:
+      label: Expected behavior
+      description: "A clear and concise description of what you would expect to happen."

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,12 @@
+blank_issues_enabled: true
+version: 2.1
+contact_links:
+  - name: Model checkpoints on the Hugging Face Hub
+    url: https://huggingface.co/models
+    about: Open a Pull request / Discussion related to a specific model checkpoint directly on the Hugging Face Hub
+  - name: Website Related
+    url: https://github.com/huggingface/hub-docs/issues
+    about: Feature requests and bug reports related to the website
+  - name: Forum
+    url: https://discuss.huggingface.co/
+    about: General usage questions and community discussions

.github/ISSUE_TEMPLATE/feature-request.yml

@@ -0,0 +1,31 @@
+name: "\U0001F680 Feature request"
+description: Submit a proposal/request a new feature on Optimum Neuron
+labels: [ "feature" ]
+body:
+  - type: textarea
+    id: feature-request
+    validations:
+      required: true
+    attributes:
+      label: Feature request
+      description: |
+        A clear and concise description of the feature proposal. Please provide a link to the paper and code in case they exist.
+
+  - type: textarea
+    id: motivation
+    validations:
+      required: true
+    attributes:
+      label: Motivation
+      description: |
+        Please outline the motivation for the proposal. Is your feature request related to a problem? e.g., I'm always frustrated when [...]. If this is related to another GitHub issue, please link here too.
+        
+
+  - type: textarea
+    id: contribution
+    validations:
+      required: true
+    attributes:
+      label: Your contribution
+      description: |
+        Is there any way that you could help, e.g. by submitting a PR?

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,21 @@
+# What does this PR do?
+
+<!--
+Congratulations! You've made it this far! You're not quite done yet though.
+
+Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.
+
+Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.
+
+Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
+-->
+
+<!-- Remove if not applicable -->
+
+Fixes # (issue)
+
+
+## Before submitting
+- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
+- [ ] Did you make sure to update the documentation with your changes?
+- [ ] Did you write any new necessary tests?

.github/workflows/build-ami.yml

@@ -0,0 +1,83 @@
+# The workflow file for building the AWS Neuron AMI using Packer
+# It can be triggered by push and pull request to main when changes made to infrastructure/ami folder, manually and scheduler.
+name: Build AWS Neuron AMI
+on:
+    push:
+      branches:
+        - main
+      paths:
+        - 'infrastructure/ami/**'
+    pull_request:
+      branches:
+        - main
+      paths:
+        - 'infrastructure/ami/**'
+    workflow_dispatch:
+      inputs: 
+        tag: 
+          description: 'Tag to use for the AMI build'
+          default: 'main'
+    schedule:
+      # Schedule the workflow to run every second day at midnight UTC
+      - cron: '0 0 */2 * *'
+
+jobs:
+    build-ami:
+        defaults:
+          run:
+            working-directory: infrastructure/ami
+        runs-on: ubuntu-latest
+        env:
+          AWS_REGION: us-east-1
+        steps:
+          - name: Checkout
+            uses: actions/checkout@v4
+
+          - name: Setup Packer
+            uses: hashicorp/setup-packer@main
+            id: setup-packer
+            with:
+              version: "1.10.1"
+
+          - name: configure aws credentials
+            uses: aws-actions/configure-aws-credentials@v4
+            with:
+              aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID_BUILD_AMI }}
+              aws-secret-access-key: ${{ secrets.AWS_ACCESS_KEY_SECRET_BUILD_AMI }}
+              aws-region: ${{ env.AWS_REGION }}
+
+          - name: Packer format
+            id: format
+            run: packer fmt hcl2-files
+            continue-on-error: true
+
+          - name: Packer Init
+            id: init
+            run: packer init hcl2-files
+            continue-on-error: true
+
+
+          - name: Packer Validate
+            id: validate
+            # If the workflow is triggered manually or scheduled, uses the tag, otherwise uses the name of branch that triggered workflow for building the AMI  
+            run: packer validate -var "optimum_version=${{ github.event.inputs.tag || github.head_ref || github.ref_name }}" -var "region=${{ env.AWS_REGION }}" hcl2-files
+            continue-on-error: true
+
+          - name: Packer Build
+            id: build
+            # If the workflow is triggered manually or scheduled, uses the tag, otherwise uses the name of branch that triggered workflow for building the AMI 
+            run:  | 
+              packer build -var "optimum_version=${{ github.event.inputs.tag || github.head_ref || github.ref_name }}" -var "region=${{ env.AWS_REGION }}" hcl2-files              
+
+          - name: Slack Notification on Failure
+            id: slack
+            uses: slackapi/[email protected]
+            if: ${{ failure() && github.event_name == 'schedule' }}
+            with:
+              channel-id: 'C06GAEQJLNN' #copied from slack channel
+              payload: |
+                {
+                  "text": "GitHub Action HuggingFace Neuron AMI Build result: ${{job.status}}"
+                }
+            env:
+              SLACK_BOT_TOKEN: ${{ secrets.SLACK_CIFEEDBACK_BOT_TOKEN }}

.github/workflows/test_inf1_export.yml

@@ -0,0 +1,49 @@
+name: Optimum neuron / Test INF1 export
+
+on:
+  push:
+    branches: [ main ]
+    paths:
+      - "setup.py"
+      - "optimum/**.py"
+  pull_request:
+    branches: [ main ]
+    paths:
+      - "setup.py"
+      - "optimum/**.py"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  do-the-job:
+    name: Run INF1 tests
+    runs-on: [self-hosted, 4-aws-inf1, 24-cpu, ci]
+    env:
+      AWS_REGION: us-east-1
+    steps:
+      - name: Check AMI
+        run: dpkg -l | grep neuron
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Install system packages
+        run: |
+          sudo apt install python3.8-venv -y
+      - name: Install python packages
+        run: |
+          python3 -m venv aws_neuron_venv_pytorch
+          source aws_neuron_venv_pytorch/bin/activate
+          python -m pip install -U pip
+          python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com
+          python -m pip install .[neuron,tests]
+          python -m pip uninstall optimum -y
+          python -m pip install optimum
+      - name: Run CLI tests
+        run: |
+          source aws_neuron_venv_pytorch/bin/activate
+          HF_TOKEN=${{ secrets.HF_TOKEN_OPTIMUM_NEURON_CI }} pytest -m is_inferentia_test tests/cli
+      - name: Run export tests
+        run: |
+          source aws_neuron_venv_pytorch/bin/activate
+          HF_TOKEN=${{ secrets.HF_TOKEN_OPTIMUM_NEURON_CI }} pytest -m is_inferentia_test tests/exporters

.github/workflows/test_inf1.yml → .github/workflows/test_inf1_inference.yml

@@ -1,4 +1,4 @@
-name: Optimum neuron / Test INF1
+name: Optimum neuron / Test INF1 inference & pipelines
 
 on:
   push:
@@ -39,14 +39,6 @@ jobs:
           python -m pip install .[neuron,tests]
           python -m pip uninstall optimum -y
           python -m pip install optimum
-      - name: Run CLI tests
-        run: |
-          source aws_neuron_venv_pytorch/bin/activate
-          HF_TOKEN=${{ secrets.HF_TOKEN_OPTIMUM_NEURON_CI }} pytest -m is_inferentia_test tests/cli
-      - name: Run export tests
-        run: |
-          source aws_neuron_venv_pytorch/bin/activate
-          HF_TOKEN=${{ secrets.HF_TOKEN_OPTIMUM_NEURON_CI }} pytest -m is_inferentia_test tests/exporters
       - name: Run inference tests
         run: |
           source aws_neuron_venv_pytorch/bin/activate

.github/workflows/test_inf2.yml

@@ -47,15 +47,3 @@ jobs:
         run: |
           source aws_neuron_venv_pytorch/bin/activate
           HF_TOKEN=${{ secrets.HF_TOKEN_OPTIMUM_NEURON_CI }} pytest -m is_inferentia_test tests/generation
-      - name: Run exporters tests
-        run: |
-          source aws_neuron_venv_pytorch/bin/activate
-          HF_TOKEN=${{ secrets.HF_TOKEN_OPTIMUM_NEURON_CI }} pytest -m is_inferentia_test tests/exporters
-      - name: Run inference tests
-        run: |
-          source aws_neuron_venv_pytorch/bin/activate
-          HF_TOKEN=${{ secrets.HF_TOKEN_OPTIMUM_NEURON_CI }} pytest -m is_inferentia_test tests/inference
-      - name: Run pipelines tests
-        run: |
-          source aws_neuron_venv_pytorch/bin/activate
-          HF_TOKEN=${{ secrets.HF_TOKEN_OPTIMUM_NEURON_CI }} pytest -m is_inferentia_test tests/pipelines

.github/workflows/test_inf2_export.yml

@@ -0,0 +1,41 @@
+name: Optimum neuron / Test INF2 export
+
+on:
+  push:
+    branches: [ main ]
+    paths:
+      - "setup.py"
+      - "optimum/**.py"
+  pull_request:
+    branches: [ main ]
+    paths:
+      - "setup.py"
+      - "optimum/**.py"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  do-the-job:
+    name: Run INF2 tests
+    runs-on: [self-hosted, 1-aws-inf2, 32-cpu, ci] # run the job on the newly created runner
+    env:
+      AWS_REGION: us-east-1
+    steps:
+      - name: Check AMI
+        run: dpkg -l | grep neuron
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Install python dependencies
+        run: |
+          sudo apt install python3.8-venv -y
+          python3 -m venv aws_neuron_venv_pytorch
+          source aws_neuron_venv_pytorch/bin/activate
+          python -m pip install -U pip
+          python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com
+          python -m pip install .[neuronx,tests]
+      - name: Run exporters tests
+        run: |
+          source aws_neuron_venv_pytorch/bin/activate
+          HF_TOKEN=${{ secrets.HF_TOKEN_OPTIMUM_NEURON_CI }} pytest -m is_inferentia_test tests/exporters