Feat: Add Deepl support for plugins/ai-proxy #1147
Conversation
|
@CH3CHO @urlyy
但是根据我的测试，我发现不管加不加 context 好像都不影响翻译的结果。如果是这样的话，那么 Deepl 就相当于是一个普通的翻译 API，在我看来就没用兼容 openai 协议的必要了。 # 不加 context
curl -H "Authorization: DeepL-Auth-Key <API KEY>" https://api-free.deepl.com/v2/translate -H "Content-Type: application/json" -d '
{
"source_lang": "EN",
"target_lang": "ZH",
"text": ["Higress is a next-generation cloud-native gateway built on the core of open-source Istio + Envoy based on Alibaba’s internal Envoy Gateway practice. It realizes the high integration capability of three-in-one traffic gateway + microservice gateway + security gateway, and deeply integrates Dubbo and Nacos , Sentinel and other micro-service technology stacks can help users greatly reduce the cost of gateway deployment and operation and maintenance without compromising capabilities; fully support Ingress and Gateway APIs in terms of standards, and actively embrace the standard API specifications under cloud native; at the same time, Higress The Controller also supports smooth migration of Nginx Ingress, helping users quickly migrate to Higress at zero cost."]
}'
{"translations":[{"detected_source_language":"EN","text":"Higress是基于阿里巴巴内部Envoy网关实践，以开源Istio+Envoy为核心构建的下一代云原生网关。它实现了流量网关+微服务网关+安全网关三位一体的高集成度能力，深度集成了Dubbo和Nacos、Sentinel等微服务技术栈，可以帮助用户在不影响能力的前提下大大降低网关部署和运维成本；在标准方面全面支持Ingress和Gateway API，积极拥抱云原生下的标准API规范；同时，Higress The Controller还支持Nginx Ingress的平滑迁移，帮助用户零成本快速迁移到Higress。"}]}
# 加 context
curl -H "Authorization: DeepL-Auth-Key <API KEY>" https://api-free.deepl.com/v2/translate -H "Content-Type: application/json" -d '
{
"source_lang": "EN",
"target_lang": "ZH",
"text": ["Higress is a next-generation cloud-native gateway built on the core of open-source Istio + Envoy based on Alibaba’s internal Envoy Gateway practice. It realizes the high integration capability of three-in-one traffic gateway + microservice gateway + security gateway, and deeply integrates Dubbo and Nacos , Sentinel and other micro-service technology stacks can help users greatly reduce the cost of gateway deployment and operation and maintenance without compromising capabilities; fully support Ingress and Gateway APIs in terms of standards, and actively embrace the standard API specifications under cloud native; at the same time, Higress The Controller also supports smooth migration of Nginx Ingress, helping users quickly migrate to Higress at zero cost."],
"context": "翻译的时候不要逐字翻译，可以意译，使用一些高级的词汇"
}'
{"translations":[{"detected_source_language":"EN","text":"Higress是基于阿里巴巴内部Envoy网关实践，以开源Istio+Envoy为核心构建的下一代云原生网关。它实现了流量网关+微服务网关+安全网关三位一体的高集成度能力，深度集成了Dubbo和Nacos、Sentinel等微服务技术栈，可以帮助用户在不影响能力的前提下大大降低网关部署和运维成本；在标准方面全面支持Ingress和Gateway API，积极拥抱云原生下的标准API规范；同时，Higress The Controller还支持Nginx Ingress的平滑迁移，帮助用户零成本快速迁移到Higress。"}]}% |
|
@cr7258 看了几篇博客和官方描述，context起到的引导作用没有prompt那么大，只是缩小一下语境、避免一词多义导致翻译错误的情况。 curl -X POST 'https://api-free.deepl.com/v2/translate' \
--header 'Authorization: DeepL-Auth-Key <API KEY>' \
--header 'Content-Type: application/json' \
--data '{
"text": [
"sit by the bank"
],
"target_lang": "ZH"
}'
curl -X POST 'https://api-free.deepl.com/v2/translate' \
--header 'Authorization: DeepL-Auth-Key <API KEY>' \
--header 'Content-Type: application/json' \
--data '{
"text": [
"sit by the bank"
],
"target_lang": "ZH",
"context": "river"
}'
curl -X POST 'https://api-free.deepl.com/v2/translate' \
--header 'Authorization: DeepL-Auth-Key <API KEY>' \
--header 'Content-Type: application/json' \
--data '{
"text": [
"sit by the bank"
],
"target_lang": "ZH",
"context": "money"
}'
官方也说了 |
|
@urlyy Ok, 那至少 context 目前还是有一定作用的，可以在请求的时候把 system prompt 拿到设置到 context 里。 |
model 可以是 pro 和 free，根据 pro 和 free 设置请求的 API URL，这样就不需要额外的参数 deeplVersion 了。 |
增加两个配置参数来表示 target_lang 和 source_lang 是不是更好点？ |
|
@cr7258 所以目前就是我们除了text数组外还需要3或4个参数 {
"model" : "freee', // 这个不属于DeepL的请求参数
"text": [
"Hello, world!",
"你好"
],
"target_lang": "DE",
"source_lang": "EN", //这个不是必要的，DeepL有一定的识别待翻译文本的语种的能力
"context": "river",
}但是对于openai协议的request，最外层只有 type chatCompletionRequest struct {
Model string `json:"model"`
Messages []chatMessage `json:"messages"`
MaxTokens int `json:"max_tokens,omitempty"`
FrequencyPenalty float64 `json:"frequency_penalty,omitempty"`
N int `json:"n,omitempty"`
PresencePenalty float64 `json:"presence_penalty,omitempty"`
Seed int `json:"seed,omitempty"`
Stream bool `json:"stream,omitempty"`
StreamOptions *streamOptions `json:"stream_options,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
TopP float64 `json:"top_p,omitempty"`
Tools []tool `json:"tools,omitempty"`
ToolChoice *toolChoice `json:"tool_choice,omitempty"`
User string `json:"user,omitempty"`
Stop []string `json:"stop,omitempty"`
}
type chatMessage struct {
Name string `json:"name,omitempty"`
Role string `json:"role,omitempty"`
Content string `json:"content,omitempty"`
ToolCalls []toolCall `json:"tool_calls,omitempty"`
}
type streamOptions struct {
IncludeUsage bool `json:"include_usage,omitempty"`
}
type tool struct {
Type string `json:"type"`
Function function `json:"function"`
}
type function struct {
Description string `json:"description,omitempty"`
Name string `json:"name"`
Parameters map[string]interface{} `json:"parameters,omitempty"`
}
type toolChoice struct {
Type string `json:"type"`
Function function `json:"function"`
}如果不加入新参数，可能就复用 我是不太想代理一个不太像LLM的服务反而影响其他正常代理，包括还有我对 |
|
@urlyy 我的意思是把配置参数加在这里 higress/plugins/wasm-go/extensions/ai-proxy/provider/provider.go Lines 117 to 172 in d79129b |
这样不支持用户自己更改翻译语种吧？ |
是的，这样就只能根据配置来设置翻译语种了。 但是这样感觉起码比把 target_lang 和 source_lang 设置到不相干的 model 和 user 字段里好点。。。 |
|
@cr7258 这次将版本放在Model字段上了，然后添加了 问题就是
|
详情参考：#974 (comment) |
|
@urlyy Please resolve the conflicts |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1147 +/- ##
==========================================
+ Coverage 35.91% 35.96% +0.05%
==========================================
Files 69 69
Lines 11576 9500 -2076
==========================================
- Hits 4157 3417 -740
+ Misses 7104 5767 -1337
- Partials 315 316 +1 |
|
@cr7258 我根据这个方法：#974 (comment) 来弄的话，还是没有进入onRequestBody，但我感觉代码没问题了，能麻烦您能帮我测试一下吗？apikey是 |
可能是你编译的命令不对？如果用了 HeaderStopIteration，编译的时候要带上 EXTRA_TAGS=proxy_wasm_version_0_2_100。 curl --location 'http://127.0.0.1:10000/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
"model":"Free",
"messages": [
{
"role": "system",
"content": "money"
},
{
"content": "sit by the bank"
}
]
}'
{"choices":[{"index":0,"message":{"name":"EN","role":"assistant","content":"坐庄"}}],"created":1723639642,"model":"Free","object":"chat.completion","usage":{}}另外我重跑了 ci 测试，通过了。 |
Ⅰ. Describe what this PR did
Add Deepl support for plugins/ai-proxy
Ⅱ. Does this pull request fix one issue?
#963
Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
ai-proxy/docker-compose.yamlai-proxy/envoy.yamlai-proxy/build.shgo mod tidy tinygo build -o plugin.wasm -scheduler=none -target=wasi -gc=custom -tags="custommalloc nottinygc_finalizer" ./main.go sudo docker compose up此处
model表示deepl的服务类型，必须为"Free"或"Pro"，messages[i].content表示deepl的text[i]，messages[i].role为system时表示deepl的context。响应值
此处
choices[i].message.name表示deepl的translations[i].detected_source_language，choices[i].message.content表示deepl的translations[i].text。usage不能从响应中拿到，需要调另外的接口获得。"stream":true时
Ⅴ. Special notes for reviews
"stream" : true做了兼容，会进入OnStreamingResponseBody中。但发现ai-proxy/main.go中对于响应头的处理是 "先根据响应头判断是否按流式处理，再根据插件的OnResponseHeaders方法修改响应头"，为了根据stream字段判断进入哪个处理响应的方法，我将这两个流程的顺序交换了一下。不知道会不会对其他插件造成影响。{ message: "error reason" }，具体可参考 https://developers.deepl.com/docs/v/zh/best-practices/error-handling 。但似乎因为这样就不会进入onResponseBody，我就不能把他转成OpenAI协议的响应值了。所以这里还有待处理。go mod tidy的话，会有依赖"github.com/higress-group/proxy-wasm-go-sdk/proxywasm/types"爆红。我把变更后的go.sum也提交上来了，如果不应该更改的话，我会改回去。本次改动用到了通义灵码辅助编程