网站建设拾金手指下拉二十,网站开发保障合同,wordpress增加底部导航栏,海南响应式网站建设方案最近在做大模型对话相关功能#xff0c;需要将对话内容流式返回给前端页面#xff08;类似GPT的效果#xff09;。下面直接说下如何实现#xff1a;
1.首先导入fastapi和sse流式返回所需要的包
from fastapi import APIRouter, Response, status
from sse_starlette.sse …最近在做大模型对话相关功能需要将对话内容流式返回给前端页面类似GPT的效果。下面直接说下如何实现
1.首先导入fastapi和sse流式返回所需要的包
from fastapi import APIRouter, Response, status
from sse_starlette.sse import EventSourceResponse
2.用EventSourceResponse来调用生成方法在对话方法用yield推送对话内容
api_router.post(/stream-chat)
def stream_chat(ask_form: ChatParam, response: Response):# 设置响应头部信息response.headers[Content-Type] text/event-streamresponse.headers[Cache-Control] no-cachereturn EventSourceResponse(stream_generate_text(ask_form, streamTrue))
def stream_generate_text(ask_form: ChatParam, stream: bool False):for res in model.chat(queryask_form.prompt, streamstream, historyshistory, temperaturetemperature):yield json.dumps({answer: res},ensure_asciiFalse,)
3.返回效果
