购物网站开发教程视频,工程公司经营范围,新手怎么做外贸,喷码机营销型网站背景
llama-index在实现RAG方案的时候多是用的llama等英文大模型#xff0c;对于国内的诸多模型案例较少#xff0c;本次将使用qwen大模型实现llama-index的RAG方案。
环境配置
#xff08;1#xff09;pip包
llamaindex需要预装很多包#xff0c;这里先把我成功的案例…背景
llama-index在实现RAG方案的时候多是用的llama等英文大模型对于国内的诸多模型案例较少本次将使用qwen大模型实现llama-index的RAG方案。
环境配置
1pip包
llamaindex需要预装很多包这里先把我成功的案例里面的pip包配置发出来在requirements.txt里面。
absl-py1.4.0
accelerate0.27.2
aiohttp3.9.3
aiosignal1.3.1
aliyun-python-sdk-core2.13.36
aliyun-python-sdk-kms2.16.1
annotated-types0.6.0
anyio3.7.1
apphub file:///environment/apps/apphub/dist/apphub-1.0.0.tar.gz#sha256260f99c0de4c575b19ab913aa134877e9efd81b820b97511fc8379674643c253
argon2-cffi21.3.0
argon2-cffi-bindings21.2.0
asgiref3.7.2
asttokens2.2.1
astunparse1.6.3
async-timeout4.0.3
attrs23.1.0
Babel2.12.1
backcall0.2.0
backoff2.2.1
bcrypt4.1.2
beautifulsoup44.12.3
bleach6.0.0
boltons file:///croot/boltons_1677628692245/work
brotlipy0.7.0
bs40.0.2
build1.1.1
cachetools5.3.1
certifi file:///croot/certifi_1690232220950/work/certifi
cffi file:///croot/cffi_1670423208954/work
chardet3.0.4
charset-normalizer file:///tmp/build/80754af9/charset-normalizer_1630003229654/work
chroma-hnswlib0.7.3
chromadb0.4.24
click7.1.2
cmake3.25.0
coloredlogs15.0.1
comm0.1.4
conda file:///croot/conda_1690494963117/work
conda-content-trust file:///tmp/abs_5952f1c8-355c-4855-ad2e-538535021ba5h26t22e5/croots/recipe/conda-content-trust_1658126371814/work
conda-libmamba-solver file:///croot/conda-libmamba-solver_1685032319139/work/src
conda-package-handling file:///croot/conda-package-handling_1685024767917/work
conda_package_streaming file:///croot/conda-package-streaming_1685019673878/work
contourpy1.2.0
crcmod1.7
cryptography file:///croot/cryptography_1686613057838/work
cycler0.12.1
dataclasses-json0.6.4
debugpy1.6.7
decorator5.1.1
defusedxml0.7.1
Deprecated1.2.14
dirtyjson1.0.8
distro1.9.0
ecdsa0.18.0
exceptiongroup1.1.2
executing1.2.0
fastapi0.104.1
fastjsonschema2.18.0
featurize0.0.24
filelock3.9.0
flatbuffers23.5.26
fonttools4.44.0
frozenlist1.4.1
fsspec2024.2.0
gast0.4.0
google-auth2.22.0
google-auth-oauthlib1.0.0
google-pasta0.2.0
googleapis-common-protos1.62.0
greenlet3.0.3
grpcio1.62.0
gunicorn21.2.0
h110.14.0
h5py3.9.0
httpcore0.17.3
httptools0.6.1
httpx0.24.1
huggingface-hub0.20.3
humanfriendly10.0
idna2.10
imageio2.32.0
importlib-metadata6.11.0
importlib_resources6.1.3
ipykernel6.25.0
ipython8.14.0
ipython-genutils0.2.0
ipywidgets8.1.2
jedi0.19.0
Jinja23.1.2
jmespath0.10.0
joblib1.3.2
json50.9.14
jsonpatch file:///tmp/build/80754af9/jsonpatch_1615747632069/work
jsonpointer2.1
jsonschema4.18.6
jsonschema-specifications2023.7.1
jupyter-server1.24.0
jupyter_client8.3.0
jupyter_core5.3.1
jupyterlab3.2.9
jupyterlab-pygments0.2.2
jupyterlab_server2.24.0
jupyterlab_widgets3.0.10
keras2.13.1
kiwisolver1.4.5
kubernetes29.0.0
lazy_loader0.3
libclang16.0.6
libmambapy file:///croot/mamba-split_1685993156657/work/libmambapy
lit15.0.7
llama-index0.10.17
llama-index-agent-openai0.1.5
llama-index-cli0.1.8
llama-index-core0.10.17
llama-index-embeddings-huggingface0.1.4
llama-index-embeddings-openai0.1.6
llama-index-indices-managed-llama-cloud0.1.3
llama-index-legacy0.9.48
llama-index-llms-huggingface0.1.3
llama-index-llms-openai0.1.7
llama-index-multi-modal-llms-openai0.1.4
llama-index-program-openai0.1.4
llama-index-question-gen-openai0.1.3
llama-index-readers-file0.1.8
llama-index-readers-llama-parse0.1.3
llama-index-vector-stores-chroma0.1.5
llama-parse0.3.8
llamaindex-py-client0.1.13
Markdown3.4.4
MarkupSafe2.1.2
marshmallow3.21.1
matplotlib3.8.1
matplotlib-inline0.1.6
mistune3.0.1
mmh34.1.0
monotonic1.6
mpmath1.2.1
multidict6.0.4
mypy-extensions1.0.0
nbclassic0.2.8
nbclient0.8.0
nbconvert7.7.3
nbformat5.9.2
nest-asyncio1.6.0
networkx3.0
nltk3.8.1
notebook6.4.12
numpy1.24.1
nvidia-cublas-cu1212.1.3.1
nvidia-cuda-cupti-cu1212.1.105
nvidia-cuda-nvrtc-cu1212.1.105
nvidia-cuda-runtime-cu1212.1.105
nvidia-cudnn-cu128.9.2.26
nvidia-cufft-cu1211.0.2.54
nvidia-curand-cu1210.3.2.106
nvidia-cusolver-cu1211.4.5.107
nvidia-cusparse-cu1212.1.0.106
nvidia-nccl-cu122.19.3
nvidia-nvjitlink-cu1212.4.99
nvidia-nvtx-cu1212.1.105
oauthlib3.2.2
onnxruntime1.17.1
openai1.13.3
opencv-python4.8.1.78
opentelemetry-api1.23.0
opentelemetry-exporter-otlp-proto-common1.23.0
opentelemetry-exporter-otlp-proto-grpc1.23.0
opentelemetry-instrumentation0.44b0
opentelemetry-instrumentation-asgi0.44b0
opentelemetry-instrumentation-fastapi0.44b0
opentelemetry-proto1.23.0
opentelemetry-sdk1.23.0
opentelemetry-semantic-conventions0.44b0
opentelemetry-util-http0.44b0
opt-einsum3.3.0
orjson3.9.15
oss22.18.1
overrides7.7.0
packaging file:///croot/packaging_1678965309396/work
pandas2.1.2
pandocfilters1.5.0
parso0.8.3
pexpect4.8.0
pickleshare0.7.5
Pillow9.3.0
platformdirs3.10.0
pluggy file:///tmp/build/80754af9/pluggy_1648024709248/work
posthog3.5.0
prometheus-client0.17.1
prompt-toolkit3.0.39
protobuf4.23.4
psutil5.9.5
ptyprocess0.7.0
pulsar-client3.4.0
pure-eval0.2.2
pyasn10.5.0
pyasn1-modules0.3.0
pycosat file:///croot/pycosat_1666805502580/work
pycparser file:///tmp/build/80754af9/pycparser_1636541352034/work
pycryptodome3.18.0
pydantic2.4.2
pydantic_core2.10.1
Pygments2.15.1
PyMuPDF1.23.26
PyMuPDFb1.23.22
pyOpenSSL file:///croot/pyopenssl_1677607685877/work
pyparsing3.1.1
pypdf4.1.0
PyPika0.48.9
pyproject_hooks1.0.0
PySocks file:///home/builder/ci_310/pysocks_1640793678128/work
python-dateutil2.8.2
python-dotenv1.0.0
pytz2023.3.post1
PyYAML6.0.1
pyzmq25.1.0
referencing0.30.0
regex2023.12.25
requests2.31.0
requests-oauthlib1.3.1
rpds-py0.9.2
rsa4.9
ruamel.yaml file:///croot/ruamel.yaml_1666304550667/work
ruamel.yaml.clib file:///croot/ruamel.yaml.clib_1666302247304/work
safetensors0.4.2
scikit-image0.22.0
scikit-learn1.3.2
scipy1.11.3
seaborn0.13.0
Send2Trash1.8.2
six file:///tmp/build/80754af9/six_1644875935023/work
sniffio1.3.0
socksio1.0.0
soupsieve2.4.1
SQLAlchemy2.0.28
sshpubkeys3.3.1
stack-data0.6.2
starlette0.27.0
sympy1.11.1
tabulate0.8.7
tenacity8.2.3
tensorboard2.13.0
tensorboard-data-server0.7.1
tensorflow2.13.0
tensorflow-estimator2.13.0
tensorflow-io-gcs-filesystem0.33.0
termcolor2.3.0
terminado0.17.1
threadpoolctl3.2.0
tifffile2023.9.26
tiktoken0.6.0
tinycss21.2.1
tokenizers0.15.2
tomli2.0.1
toolz file:///croot/toolz_1667464077321/work
torch2.2.1
torchaudio2.0.2cu118
torchvision0.15.2cu118
tornado6.3.2
tqdm4.66.2
traitlets5.9.0
transformers4.38.2
triton2.2.0
typer0.9.0
typing-inspect0.9.0
typing_extensions4.8.0
tzdata2023.3
urllib31.25.11
uvicorn0.23.2
uvloop0.19.0
watchfiles0.21.0
wcwidth0.2.5
webencodings0.5.1
websocket-client1.2.1
websockets12.0
Werkzeug2.3.6
widgetsnbextension4.0.10
workspace file:///home/featurize/work/workspace/dist/workspace-0.1.0.tar.gz#sha256b292beb3599f79d3791771eff9dc422cc37c58c1fc8daadeafbf025a2e7ea986
wrapt1.15.0
yarl1.9.2
zipp3.17.0
zstandard file:///croot/zstandard_1677013143055/work2python 环境 3安装命令
!pip install llama-index
!pip install llama-index-llms-huggingface
!pip install llama-index-embeddings-huggingface
!pip install llama-index ipywidgets
!pip install torch
!git clone https://www.modelscope.cn/AI-ModelScope/bge-small-zh-v1.5.git
!git clone https://www.modelscope.cn/qwen/Qwen1.5-4B-Chat.git
4目录结构 代码
1加载模型
import torch
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.core import PromptTemplate
import os
os.environ[KMP_DUPLICATE_LIB_OK]True
# Model names (make sure you have access on HF)
LLAMA2_7B /home/featurize/Qwen1.5-4B-Chat
# LLAMA2_7B_CHAT meta-llama/Llama-2-7b-chat-hf
# LLAMA2_13B meta-llama/Llama-2-13b-hf
LLAMA2_13B_CHAT /home/featurize/Qwen1.5-4B-Chat
# LLAMA2_70B meta-llama/Llama-2-70b-hf
# LLAMA2_70B_CHAT meta-llama/Llama-2-70b-chat-hfselected_model LLAMA2_13B_CHATSYSTEM_PROMPT You are an AI assistant that answers questions in a friendly manner, based on the given source documents. Here are some rules you always follow:
- Generate human readable output, avoid creating output with gibberish text.
- Generate only the requested output, dont include any other language before or after the requested output.
- Never say thank you, that you are happy to help, that you are an AI agent, etc. Just answer directly.
- Generate professional language typically used in business documents in North America.
- Never generate offensive or foul language.
query_wrapper_prompt PromptTemplate([INST]SYS\n SYSTEM_PROMPT /SYS\n\n{query_str}[/INST]
)llm HuggingFaceLLM(context_window4096,max_new_tokens2048,generate_kwargs{temperature: 0.0, do_sample: False},query_wrapper_promptquery_wrapper_prompt,tokenizer_nameselected_model,model_nameselected_model,device_mapauto
)2加载词嵌入向量
from llama_index.embeddings.huggingface import HuggingFaceEmbeddingembed_model HuggingFaceEmbedding(model_name/home/featurize/bge-small-zh-v1.5)
from llama_index.core import SettingsSettings.llm llm
Settings.embed_model embed_model
from llama_index.core import SimpleDirectoryReader# load documents
documents SimpleDirectoryReader(./data/).load_data()
from llama_index.core import VectorStoreIndex
index VectorStoreIndex.from_documents(documents) index # set Logging to DEBUG for more detailed outputs
query_engine index.as_query_engine()
response query_engine.query(小额贷款咋规定的?)
print(response) 知识库
llamaindex实现RAG中很关键的一环就是知识库知识库主要是各种类型的文档这里给的文档是一个pdf文件文件内容如下。 总结
从上面的代码可以看出我们使用qwen和bge-zh模型可以实现本地下载模型的RAG方案知识库里面的内容也可以实现中文问答这非常有利于我们进行私有化部署方案从而扩展我们的功能。
