Skip to content

Add examples #4

Description

@AlexKoff88

The API of the repo allows benchmarking accuracy of many optimization backends including:

  • HuggingFace (4-bit via Bytesandbits)
  • GPTQ
  • LLama.cpp (via bigdl-llm)
  • OpenVINO (via optimum)

I suggest creating examples for all four backends and demonstrating capabilities. We can also do a comparison and publish in readme.
You can use optimized versions of llama-7B from here: https://huggingface.co/TheBloke

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions