File tree Expand file tree Collapse file tree 1 file changed +8
-9
lines changed Expand file tree Collapse file tree 1 file changed +8
-9
lines changed Original file line number Diff line number Diff line change 66
77-  State-of-the-art LLMs support
88-  Continuous Batching via [ vLLM] ( https://github.com/vllm-project/vllm ) 
9- -  Quantization([ issue #37 ] ( https://github.com/InftyAI/llmlite/issues/37 ) )
10- -  Adapter support ([ issue #51 ] ( https://github.com/InftyAI/llmlite/issues/51 ) )
11- -  Streaming support ([ issue #52 ] ( https://github.com/InftyAI/llmlite/issues/52 ) )
9+ -  Quantization([ issue #37 ]  ( < https://github.com/InftyAI/llmlite/issues/37 > ))
10+ -  Loading specific adapters  ([ issue #51 ] ( https://github.com/InftyAI/llmlite/issues/51 ) )
11+ -  Streaming ([ issue #52 ] ( https://github.com/InftyAI/llmlite/issues/52 ) )
1212
1313### Model Support  
1414
1818|  Llama-2 |  Done ✅ |  Yes |  | 
1919|  CodeLlama |  Done ✅ |  Yes |  | 
2020|  ChatGLM2 |  Done ✅ |  No |  | 
21- |  ChatGLM3 |  WIP ⏳ |  Yes |  | 
2221|  Baichuan2 |  Done ✅ |  Yes |  | 
22+ |  ChatGLM3 |  WIP ⏳ |  Yes |  | 
2323| Claude-2 | RoadMap 📋 | | [ issue #7 ] ( https://github.com/InftyAI/ChatLLM/issues/7 ) 
2424| Falcon | RoadMap 📋 | | [ issue #8 ] ( https://github.com/InftyAI/ChatLLM/issues/8 ) 
2525|  StableLM |  RoadMap 📋 |  |  [ issue #11 ] ( https://github.com/InftyAI/ChatLLM/issues/11 )  | 
26- | Baichuan2 | RoadMap 📋 | | [ issue #34 ] ( https://github.com/InftyAI/llmlite/issues/34 ) 
2726
2827### Backend Support  
2928
30- |  backend |  State |  Note  | 
31- |  ---- |  ---- |  ----  | 
32- |  [ huggingface] ( https://github.com/huggingface )  |  Done ✅ |  Support by huggingface pipeline  | 
33- |  [ vLLM] ( https://github.com/vllm-project/vllm )  |  Done ✅ |   | 
29+ |  backend |  State | 
30+ |  ---- |  ---- | 
31+ |  [ huggingface] ( https://github.com/huggingface )  |  Done ✅ | 
32+ |  [ vLLM] ( https://github.com/vllm-project/vllm )  |  Done ✅ | 
3433
3534## How to install  
3635
    
 
   
 
     
   
   
          
     
  
    
     
 
    
      
     
 
     
    You can’t perform that action at this time.
  
 
    
  
     
    
      
        
     
 
       
      
     
   
 
    
    
  
 
  
 
     
    
0 commit comments