feat: AI Proxy Wasm Plugin Integration with GitHub Models #1304

[hackerYHT]

Ⅰ. Describe what this PR did

AI Proxy Wasm Plugin Integration with GitHub Models #1304

API documentation: https://github.com/marketplace/models/azure-openai/gpt-4o

Ⅱ. Does this pull request fix one issue?

fixes #1304

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

openai协议，非流式

curl -X POST "http://127.0.0.1:10000/v1/chat/completions" \ -H "Content-Type: application/json" \ -d '{ "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "What is the capital of France?" } ], "temperature": 1.0, "top_p": 1.0, "max_tokens": 1000, "model": "gpt-4o" }'

回复

{ "choices": [ { "finish_reason": "stop", "index": 0, "logprobs": null, "message": { "content": "The capital of France is Paris.", "role": "assistant" } } ], "created": 1728131051, "id": "chatcmpl-AEy7PU2JImdsD1W6Jw8GigZSEnM2u", "model": "gpt-4o-2024-08-06", "object": "chat.completion", "system_fingerprint": "fp_67802d9a6d", "usage": { "completion_tokens": 7, "prompt_tokens": 24, "total_tokens": 31 } }

openai协议，流式

curl -X POST "http://127.0.0.1:10000/v1/chat/completions" \ -H "Content-Type: application/json" \ -d '{ "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "What is the capital of France?" } ], "stream": true, "temperature": 1.0, "top_p": 1.0, "max_tokens": 1000, "model": "gpt-4o" }'

回复

`data: {"choices":[{"delta":{"role":"assistant"},"finish_reason":null,"index":0,"logprobs":null}],"created":1728132827,"id":"chatcmpl-AEya3ZysGvKnIsYoPwE1lJNmDXQqZ","model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","system_fingerprint":"fp_67802d9a6d"}

data: {"choices":[{"delta":{"content":"The"},"finish_reason":null,"index":0,"logprobs":null}],"created":1728132827,"id":"chatcmpl-AEya3ZysGvKnIsYoPwE1lJNmDXQqZ","model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","system_fingerprint":"fp_67802d9a6d"}

data: {"choices":[{"delta":{"content":" capital"},"finish_reason":null,"index":0,"logprobs":null}],"created":1728132827,"id":"chatcmpl-AEya3ZysGvKnIsYoPwE1lJNmDXQqZ","model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","system_fingerprint":"fp_67802d9a6d"}

data: {"choices":[{"delta":{"content":" of"},"finish_reason":null,"index":0,"logprobs":null}],"created":1728132827,"id":"chatcmpl-AEya3ZysGvKnIsYoPwE1lJNmDXQqZ","model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","system_fingerprint":"fp_67802d9a6d"}

data: {"choices":[{"delta":{"content":" France"},"finish_reason":null,"index":0,"logprobs":null}],"created":1728132827,"id":"chatcmpl-AEya3ZysGvKnIsYoPwE1lJNmDXQqZ","model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","system_fingerprint":"fp_67802d9a6d"}

data: {"choices":[{"delta":{"content":" is"},"finish_reason":null,"index":0,"logprobs":null}],"created":1728132827,"id":"chatcmpl-AEya3ZysGvKnIsYoPwE1lJNmDXQqZ","model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","system_fingerprint":"fp_67802d9a6d"}

data: {"choices":[{"delta":{"content":" Paris"},"finish_reason":null,"index":0,"logprobs":null}],"created":1728132827,"id":"chatcmpl-AEya3ZysGvKnIsYoPwE1lJNmDXQqZ","model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","system_fingerprint":"fp_67802d9a6d"}

data: {"choices":[{"delta":{"content":"."},"finish_reason":null,"index":0,"logprobs":null}],"created":1728132827,"id":"chatcmpl-AEya3ZysGvKnIsYoPwE1lJNmDXQqZ","model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","system_fingerprint":"fp_67802d9a6d"}

data: {"choices":[{"delta":{},"finish_reason":"stop","index":0,"logprobs":null}],"created":1728132827,"id":"chatcmpl-AEya3ZysGvKnIsYoPwE1lJNmDXQqZ","model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","system_fingerprint":"fp_67802d9a6d"}

data: [DONE]`

openai协议，请求embedding模型

curl -X POST "http://127.0.0.1:10000/v1/embeddings"
-H "Content-Type: application/json"
-d '{
"input": ["first phrase", "second phrase", "third phrase"],
"model": "text-embedding-3-large"
}'

回复

{ "object": "list", "data": [ { "object": "embedding", "index": 0, "embedding": [ -0.0012583479, 0.0020349282, ... 0.012051377, -0.0053306012, 0.0060688322 ] } ], "model": "text-embedding-3-large", "usage": { "prompt_tokens": 6, "total_tokens": 6 } }

Ⅴ. Special notes for reviews

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 44.23%. Comparing base (ef31e09) to head (c613f72).
Report is 124 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1362      +/-   ##
==========================================
+ Coverage   35.91%   44.23%   +8.32%     
==========================================
  Files          69       76       +7     
  Lines       11576     9895    -1681     
==========================================
+ Hits         4157     4377     +220     
+ Misses       7104     5183    -1921     
- Partials      315      335      +20     

see 91 files with indirect coverage changes

[CH3CHO]

这里整合的其实是 Azure OpenAI 模型，叫 GitHub 模型是不是言过其实了？用 OpenAI 加自定义 URL 能完成对接吗？

[hackerYHT]

这里整合的其实是 Azure OpenAI 模型，叫 GitHub 模型是不是言过其实了？用 OpenAI 加自定义 URL 能完成对接吗？

github的model market的模型都是请求https://models.inference.ai.azure.com 这个域名，只是在请求体当中指定“ model”。这里是可以请求github model market提供的所有模型，不单单是指 Azure OpenAI 模型。

[CH3CHO]

这里整合的其实是 Azure OpenAI 模型，叫 GitHub 模型是不是言过其实了？用 OpenAI 加自定义 URL 能完成对接吗？

github的model market的模型都是请求https://models.inference.ai.azure.com 这个域名，只是在请求体当中指定“ model”。这里是可以请求github model market提供的所有模型，不单单是指 Azure OpenAI 模型。

OK

[CH3CHO]

LGTM