中文网站建设中模板下载,网站什么时候做SEO优化最合适,如何申请网上商城,陕西省建设厅网站证件查询1. MindNLP ChatGLM-6B StreamChat
本案例基于MindNLP和ChatGLM-6B实现一个聊天应用。 ChatGLM-6B应该是国内第一个发布的可以在消费级显卡上进行推理部署的国产开源大模型#xff0c;2023年3月就发布了。我在23年6月份的时候就在自己的笔记本电脑上部署测试过#xff0c;当…1. MindNLP ChatGLM-6B StreamChat
本案例基于MindNLP和ChatGLM-6B实现一个聊天应用。 ChatGLM-6B应该是国内第一个发布的可以在消费级显卡上进行推理部署的国产开源大模型2023年3月就发布了。我在23年6月份的时候就在自己的笔记本电脑上部署测试过当时的1代6B模型已经能解鸡兔同笼的数学问题感觉上是真正“理解”了人类语言的语义。我认为和chatgpt相比也并没有非常明显的差距。当然存在的问题也有不少。 1.1 环境配置
安装mindnlp
!pip install mindnlp 安装过程:
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting mindnlpDownloading https://pypi.tuna.tsinghua.edu.cn/packages/72/37/ef313c23fd587c3d1f46b0741c98235aecdfd93b4d6d446376f3db6a552c/mindnlp-0.3.1-py3-none-any.whl (5.7 MB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 16.7 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: mindspore in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from mindnlp) (2.2.14)
Requirement already satisfied: tqdm in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from mindnlp) (4.66.4)
Requirement already satisfied: requests in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from mindnlp) (2.32.3)
Collecting datasets (from mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/60/2d/963b266bb8f88492d5ab4232d74292af8beb5b6fdae97902df9e284d4c32/datasets-2.20.0-py3-none-any.whl (547 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 547.8/547.8 kB 16.4 MB/s eta 0:00:00
Collecting evaluate (from mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c2/d6/ff9baefc8fc679dcd9eb21b29da3ef10c81aa36be630a7ae78e4611588e1/evaluate-0.4.2-py3-none-any.whl (84 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.1/84.1 kB 25.4 MB/s eta 0:00:00
Collecting tokenizers (from mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ba/26/139bd2371228a0e203da7b3e3eddcb02f45b2b7edd91df00e342e4b55e13/tokenizers-0.19.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 19.6 MB/s eta 0:00:0000:0100:01
Collecting safetensors (from mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c6/02/28e6280ed0f1bde89eed644b80f2ece4e5ae212dc9ee70d7f56fadc93602/safetensors-0.4.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.2 MB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 20.2 MB/s eta 0:00:00a 0:00:01
Collecting sentencepiece (from mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a3/69/e96ef68261fa5b82379fdedb325ceaf1d353c6e839ec346d8244e0da5f2f/sentencepiece-0.2.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.3 MB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 15.5 MB/s eta 0:00:00a 0:00:01
Collecting regex (from mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/70/70/fea4865c89a841432497d1abbfd53878513b55c6543245fabe31cf8df0b8/regex-2024.5.15-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (774 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 774.7/774.7 kB 17.0 MB/s eta 0:00:00a 0:00:01
Collecting addict (from mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/6a/00/b08f23b7d7e1e14ce01419a467b583edbb93c6cdb8654e54a9cc579cd61f/addict-2.4.0-py3-none-any.whl (3.8 kB)
Collecting ml-dtypes (from mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/50/96/13d7c3cc82d5ef597279216cf56ff461f8b57e7096a3ef10246a83ca80c0/ml_dtypes-0.4.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.2 MB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 12.6 MB/s eta 0:00:00a 0:00:01
Collecting pyctcdecode (from mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a5/8a/93e2118411ae5e861d4f4ce65578c62e85d0f1d9cb389bd63bd57130604e/pyctcdecode-0.5.0-py2.py3-none-any.whl (39 kB)
Collecting jieba (from mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c6/cb/18eeb235f833b726522d7ebed54f2278ce28ba9438e3135ab0278d9792a2/jieba-0.42.1.tar.gz (19.2 MB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.2/19.2 MB 19.3 MB/s eta 0:00:0000:0100:01Preparing metadata (setup.py) ... done
Collecting pytest7.2.0 (from mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/67/68/a5eb36c3a8540594b6035e6cdae40c1ef1b6a2bfacbecc3d1a544583c078/pytest-7.2.0-py3-none-any.whl (316 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 316.8/316.8 kB 17.8 MB/s eta 0:00:00
Requirement already satisfied: attrs19.2.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pytest7.2.0-mindnlp) (23.2.0)
Requirement already satisfied: iniconfig in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pytest7.2.0-mindnlp) (2.0.0)
Requirement already satisfied: packaging in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pytest7.2.0-mindnlp) (23.2)
Requirement already satisfied: pluggy2.0,0.12 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pytest7.2.0-mindnlp) (1.5.0)
Requirement already satisfied: exceptiongroup1.0.0rc8 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pytest7.2.0-mindnlp) (1.2.0)
Requirement already satisfied: tomli1.0.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pytest7.2.0-mindnlp) (2.0.1)
Requirement already satisfied: filelock in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from datasets-mindnlp) (3.15.3)
Requirement already satisfied: numpy1.17 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from datasets-mindnlp) (1.26.4)
Collecting pyarrow15.0.0 (from datasets-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/87/60/cc0645eb4ef73f88847e40a7f9d238bae6b7409d6c1f6a5d200d8ade1f09/pyarrow-16.1.0-cp39-cp39-manylinux_2_28_aarch64.whl (38.1 MB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 38.1/38.1 MB 18.3 MB/s eta 0:00:0000:0100:01
Collecting pyarrow-hotfix (from datasets-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/e4/f4/9ec2222f5f5f8ea04f66f184caafd991a39c8782e31f5b0266f101cb68ca/pyarrow_hotfix-0.6-py3-none-any.whl (7.9 kB)
Requirement already satisfied: dill0.3.9,0.3.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from datasets-mindnlp) (0.3.8)
Requirement already satisfied: pandas in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from datasets-mindnlp) (2.2.2)
Collecting xxhash (from datasets-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7c/b9/93f860969093d5d1c4fa60c75ca351b212560de68f33dc0da04c89b7dc1b/xxhash-3.4.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (220 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 220.6/220.6 kB 17.8 MB/s eta 0:00:00
Collecting multiprocess (from datasets-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/da/d9/f7f9379981e39b8c2511c9e0326d212accacb82f12fbfdc1aa2ce2a7b2b6/multiprocess-0.70.16-py39-none-any.whl (133 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.4/133.4 kB 17.5 MB/s eta 0:00:00
Collecting fsspec2024.5.0,2023.1.0 (from fsspec[http]2024.5.0,2023.1.0-datasets-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ba/a3/16e9fe32187e9c8bc7f9b7bcd9728529faa725231a0c96f2f98714ff2fc5/fsspec-2024.5.0-py3-none-any.whl (316 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 316.1/316.1 kB 19.4 MB/s eta 0:00:00
Collecting aiohttp (from datasets-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/eb/45/eebe8d2215328434f33ccb44a05d2741ff7ed4b96b56ca507e2ecf598b73/aiohttp-3.9.5-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.2 MB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 18.2 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: huggingface-hub0.21.2 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from datasets-mindnlp) (0.23.4)
Requirement already satisfied: pyyaml5.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from datasets-mindnlp) (6.0.1)
Requirement already satisfied: charset-normalizer4,2 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from requests-mindnlp) (3.3.2)
Requirement already satisfied: idna4,2.5 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from requests-mindnlp) (3.7)
Requirement already satisfied: urllib33,1.21.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from requests-mindnlp) (2.2.2)
Requirement already satisfied: certifi2017.4.17 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from requests-mindnlp) (2024.6.2)
Requirement already satisfied: protobuf3.13.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from mindspore-mindnlp) (5.27.1)
Requirement already satisfied: asttokens2.0.4 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from mindspore-mindnlp) (2.0.5)
Requirement already satisfied: pillow6.2.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from mindspore-mindnlp) (10.3.0)
Requirement already satisfied: scipy1.5.4 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from mindspore-mindnlp) (1.13.1)
Requirement already satisfied: psutil5.6.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from mindspore-mindnlp) (5.9.0)
Requirement already satisfied: astunparse1.6.3 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from mindspore-mindnlp) (1.6.3)
Collecting pygtrie3.0,2.1 (from pyctcdecode-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ec/cd/bd196b2cf014afb1009de8b0f05ecd54011d881944e62763f3c1b1e8ef37/pygtrie-2.5.0-py3-none-any.whl (25 kB)
Collecting hypothesis7,6.14 (from pyctcdecode-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/58/14/a4c621cb713f6053f37afa78ab3809f9d879182422071ca9d4af61c6d1d9/hypothesis-6.105.0-py3-none-any.whl (462 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 462.2/462.2 kB 21.2 MB/s eta 0:00:00
Requirement already satisfied: six in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from asttokens2.0.4-mindspore-mindnlp) (1.16.0)
Requirement already satisfied: wheel1.0,0.23.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from astunparse1.6.3-mindspore-mindnlp) (0.43.0)
Collecting aiosignal1.1.2 (from aiohttp-datasets-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/76/ac/a7305707cb852b7e16ff80eaf5692309bde30e2b1100a1fcacdc8f731d97/aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting frozenlist1.1.1 (from aiohttp-datasets-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/57/15/172af60c7e150a1d88ecc832f2590721166ae41eab582172fe1e9844eab4/frozenlist-1.4.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (239 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 239.4/239.4 kB 19.6 MB/s eta 0:00:00
Collecting multidict7.0,4.5 (from aiohttp-datasets-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d0/10/2ff646c471e84af25fe8111985ffb8ec85a3f6e1ade8643bfcfcc0f4d2b1/multidict-6.0.5-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (125 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.9/125.9 kB 16.9 MB/s eta 0:00:00
Collecting yarl2.0,1.0 (from aiohttp-datasets-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c6/d6/5b30ae1d8a13104ee2ceb649f28f2db5ad42afbd5697fd0fc61528bb112c/yarl-1.9.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (300 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 300.9/300.9 kB 14.5 MB/s eta 0:00:00
Collecting async-timeout5.0,4.0 (from aiohttp-datasets-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a7/fa/e01228c2938de91d47b307831c62ab9e4001e747789d0b05baf779a6488c/async_timeout-4.0.3-py3-none-any.whl (5.7 kB)
Requirement already satisfied: typing-extensions3.7.4.3 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from huggingface-hub0.21.2-datasets-mindnlp) (4.11.0)
Collecting sortedcontainers3.0.0,2.1.0 (from hypothesis7,6.14-pyctcdecode-mindnlp)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/32/46/9cb0e58b2deb7f82b84065f37f3bffeb12413f947f9388e4cac22c4621ce/sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Requirement already satisfied: python-dateutil2.8.2 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pandas-datasets-mindnlp) (2.9.0.post0)
Requirement already satisfied: pytz2020.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pandas-datasets-mindnlp) (2024.1)
Requirement already satisfied: tzdata2022.7 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pandas-datasets-mindnlp) (2024.1)
Building wheels for collected packages: jiebaBuilding wheel for jieba (setup.py) ... doneCreated wheel for jieba: filenamejieba-0.42.1-py3-none-any.whl size19314459 sha256b34d50dd74d300723e8f42cfea23a680ef40525b2f94488be07da842e838a41cStored in directory: /home/nginx/.cache/pip/wheels/1a/76/68/b6d79c4db704bb18d54f6a73ab551185f4711f9730c0c15d97
Successfully built jieba
Installing collected packages: sortedcontainers, sentencepiece, pygtrie, jieba, addict, xxhash, safetensors, regex, pytest, pyarrow-hotfix, pyarrow, multiprocess, multidict, ml-dtypes, hypothesis, fsspec, frozenlist, async-timeout, yarl, pyctcdecode, aiosignal, tokenizers, aiohttp, datasets, evaluate, mindnlpAttempting uninstall: pytestFound existing installation: pytest 8.0.0Uninstalling pytest-8.0.0:Successfully uninstalled pytest-8.0.0Attempting uninstall: fsspecFound existing installation: fsspec 2024.6.0Uninstalling fsspec-2024.6.0:Successfully uninstalled fsspec-2024.6.0
Successfully installed addict-2.4.0 aiohttp-3.9.5 aiosignal-1.3.1 async-timeout-4.0.3 datasets-2.20.0 evaluate-0.4.2 frozenlist-1.4.1 fsspec-2024.5.0 hypothesis-6.105.0 jieba-0.42.1 mindnlp-0.3.1 ml-dtypes-0.4.0 multidict-6.0.5 multiprocess-0.70.16 pyarrow-16.1.0 pyarrow-hotfix-0.6 pyctcdecode-0.5.0 pygtrie-2.5.0 pytest-7.2.0 regex-2024.5.15 safetensors-0.4.3 sentencepiece-0.2.0 sortedcontainers-2.4.0 tokenizers-0.19.1 xxhash-3.4.1 yarl-1.9.4[notice] A new release of pip is available: 24.1 - 24.1.1
[notice] To update, run: python -m pip install --upgrade pip安装mdtex2html
!pip install mdtex2htmlLooking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting mdtex2htmlDownloading https://pypi.tuna.tsinghua.edu.cn/packages/ff/e8/c5fab9aa5d9254ad7c7e37d33a3c32fd49d82b4c6b54da337bbca378eb5c/mdtex2html-1.3.0-py3-none-any.whl (13 kB)
Requirement already satisfied: gradio in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (4.26.0)
Collecting markdown (from mdtex2html)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/fc/b3/0c0c994fe49cd661084f8d5dc06562af53818cc0abefaca35bdc894577c3/Markdown-3.6-py3-none-any.whl (105 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 105.4/105.4 kB 11.6 MB/s eta 0:00:00
Collecting latex2mathml (from mdtex2html)Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f2/0a/181ed55562ce90179aedf33b09fcd79db31c868a5d480f3cb71a31d19692/latex2mathml-3.77.0-py3-none-any.whl (73 kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 73.7/73.7 kB 22.7 MB/s eta 0:00:00
Requirement already satisfied: aiofiles24.0,22.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (22.1.0)
Requirement already satisfied: altair6.0,4.2.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (5.3.0)
Requirement already satisfied: fastapi in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (0.111.0)
Requirement already satisfied: ffmpy in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (0.3.2)
Requirement already satisfied: gradio-client0.15.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (0.15.1)
Requirement already satisfied: httpx0.24.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (0.27.0)
Requirement already satisfied: huggingface-hub0.19.3 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (0.23.4)
Requirement already satisfied: importlib-resources7.0,1.3 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (6.4.0)
Requirement already satisfied: jinja24.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (3.1.4)
Requirement already satisfied: markupsafe~2.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (2.1.5)
Requirement already satisfied: matplotlib~3.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (3.9.0)
Requirement already satisfied: numpy~1.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (1.26.4)
Requirement already satisfied: orjson~3.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (3.10.5)
Requirement already satisfied: packaging in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (23.2)
Requirement already satisfied: pandas3.0,1.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (2.2.2)
Requirement already satisfied: pillow11.0,8.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (10.3.0)
Requirement already satisfied: pydantic2.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (2.7.4)
Requirement already satisfied: pydub in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (0.25.1)
Requirement already satisfied: python-multipart0.0.9 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (0.0.9)
Requirement already satisfied: pyyaml7.0,5.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (6.0.1)
Requirement already satisfied: ruff0.2.2 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (0.4.10)
Requirement already satisfied: semantic-version~2.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (2.10.0)
Requirement already satisfied: tomlkit0.12.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (0.12.0)
Requirement already satisfied: typer1.0,0.9 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from typer[all]1.0,0.9; sys_platform ! emscripten-gradio) (0.12.3)
Requirement already satisfied: typing-extensions~4.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (4.11.0)
Requirement already satisfied: uvicorn0.14.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio) (0.30.1)
Requirement already satisfied: fsspec in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio-client0.15.1-gradio) (2024.5.0)
Requirement already satisfied: websockets12.0,10.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from gradio-client0.15.1-gradio) (11.0.3)
Requirement already satisfied: jsonschema3.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from altair6.0,4.2.0-gradio) (4.22.0)
Requirement already satisfied: toolz in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from altair6.0,4.2.0-gradio) (0.12.1)
Requirement already satisfied: anyio in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from httpx0.24.1-gradio) (4.4.0)
Requirement already satisfied: certifi in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from httpx0.24.1-gradio) (2024.6.2)
Requirement already satisfied: httpcore1.* in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from httpx0.24.1-gradio) (1.0.5)
Requirement already satisfied: idna in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from httpx0.24.1-gradio) (3.7)
Requirement already satisfied: sniffio in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from httpx0.24.1-gradio) (1.3.1)
Requirement already satisfied: h110.15,0.13 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from httpcore1.*-httpx0.24.1-gradio) (0.14.0)
Requirement already satisfied: filelock in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from huggingface-hub0.19.3-gradio) (3.15.3)
Requirement already satisfied: requests in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from huggingface-hub0.19.3-gradio) (2.32.3)
Requirement already satisfied: tqdm4.42.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from huggingface-hub0.19.3-gradio) (4.66.4)
Requirement already satisfied: zipp3.1.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from importlib-resources7.0,1.3-gradio) (3.17.0)
Requirement already satisfied: contourpy1.0.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from matplotlib~3.0-gradio) (1.2.1)
Requirement already satisfied: cycler0.10 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from matplotlib~3.0-gradio) (0.12.1)
Requirement already satisfied: fonttools4.22.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from matplotlib~3.0-gradio) (4.53.0)
Requirement already satisfied: kiwisolver1.3.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from matplotlib~3.0-gradio) (1.4.5)
Requirement already satisfied: pyparsing2.3.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from matplotlib~3.0-gradio) (3.1.2)
Requirement already satisfied: python-dateutil2.7 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from matplotlib~3.0-gradio) (2.9.0.post0)
Requirement already satisfied: pytz2020.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pandas3.0,1.0-gradio) (2024.1)
Requirement already satisfied: tzdata2022.7 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pandas3.0,1.0-gradio) (2024.1)
Requirement already satisfied: annotated-types0.4.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pydantic2.0-gradio) (0.7.0)
Requirement already satisfied: pydantic-core2.18.4 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from pydantic2.0-gradio) (2.18.4)
Requirement already satisfied: click8.0.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from typer1.0,0.9-typer[all]1.0,0.9; sys_platform ! emscripten-gradio) (8.1.7)
Requirement already satisfied: shellingham1.3.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from typer1.0,0.9-typer[all]1.0,0.9; sys_platform ! emscripten-gradio) (1.5.4)
Requirement already satisfied: rich10.11.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from typer1.0,0.9-typer[all]1.0,0.9; sys_platform ! emscripten-gradio) (13.7.1)
WARNING: typer 0.12.3 does not provide the extra all
Requirement already satisfied: starlette0.38.0,0.37.2 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from fastapi-gradio) (0.37.2)
Requirement already satisfied: fastapi-cli0.0.2 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from fastapi-gradio) (0.0.4)
Requirement already satisfied: ujson!4.0.2,!4.1.0,!4.2.0,!4.3.0,!5.0.0,!5.1.0,4.0.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from fastapi-gradio) (5.10.0)
Requirement already satisfied: email_validator2.0.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from fastapi-gradio) (2.2.0)
Requirement already satisfied: importlib-metadata4.4 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from markdown-mdtex2html) (7.0.1)
Requirement already satisfied: dnspython2.0.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from email_validator2.0.0-fastapi-gradio) (2.6.1)
Requirement already satisfied: attrs22.2.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from jsonschema3.0-altair6.0,4.2.0-gradio) (23.2.0)
Requirement already satisfied: jsonschema-specifications2023.03.6 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from jsonschema3.0-altair6.0,4.2.0-gradio) (2023.12.1)
Requirement already satisfied: referencing0.28.4 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from jsonschema3.0-altair6.0,4.2.0-gradio) (0.35.1)
Requirement already satisfied: rpds-py0.7.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from jsonschema3.0-altair6.0,4.2.0-gradio) (0.18.1)
Requirement already satisfied: six1.5 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from python-dateutil2.7-matplotlib~3.0-gradio) (1.16.0)
Requirement already satisfied: markdown-it-py2.2.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from rich10.11.0-typer1.0,0.9-typer[all]1.0,0.9; sys_platform ! emscripten-gradio) (3.0.0)
Requirement already satisfied: pygments3.0.0,2.13.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from rich10.11.0-typer1.0,0.9-typer[all]1.0,0.9; sys_platform ! emscripten-gradio) (2.15.1)
Requirement already satisfied: exceptiongroup1.0.2 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from anyio-httpx0.24.1-gradio) (1.2.0)
Requirement already satisfied: httptools0.5.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from uvicorn[standard]0.12.0-fastapi-gradio) (0.6.1)
Requirement already satisfied: python-dotenv0.13 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from uvicorn[standard]0.12.0-fastapi-gradio) (1.0.1)
Requirement already satisfied: uvloop!0.15.0,!0.15.1,0.14.0 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from uvicorn[standard]0.12.0-fastapi-gradio) (0.19.0)
Requirement already satisfied: watchfiles0.13 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from uvicorn[standard]0.12.0-fastapi-gradio) (0.22.0)
Requirement already satisfied: charset-normalizer4,2 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from requests-huggingface-hub0.19.3-gradio) (3.3.2)
Requirement already satisfied: urllib33,1.21.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from requests-huggingface-hub0.19.3-gradio) (2.2.2)
Requirement already satisfied: mdurl~0.1 in /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages (from markdown-it-py2.2.0-rich10.11.0-typer1.0,0.9-typer[all]1.0,0.9; sys_platform ! emscripten-gradio) (0.1.2)
Installing collected packages: latex2mathml, markdown, mdtex2html
Successfully installed latex2mathml-3.77.0 markdown-3.6 mdtex2html-1.3.0[notice] A new release of pip is available: 24.1 - 24.1.1
[notice] To update, run: python -m pip install --upgrade pip安装gradio
!pip install gradio配置环境变量
# 设置环境变量 HF_ENDPOINT其值为 https://hf-mirror.com
# 这个环境变量通常用于指定 Hugging Face Transformers 模型位于国内的镜像站点
# 以便从该镜像站点下载模型和相关资源提高下载速度和稳定性,不需要代理
export HF_ENDPOINThttps://hf-mirror.com1.2 代码开发
模型参数量为6B, 磁盘空间大小占用约12G, 下载权重产加载大约需要20分钟
# 导入 MindNLP 库中的 AutoModelForSeq2SeqLM 类和 AutoTokenizer 类
# MindNLP 是一个基于 PyTorch 的自然语言处理库提供了许多预训练模型和工具
from mindnlp.transformers import AutoModelForSeq2SeqLM, AutoTokenizer# 导入 Gradio 库用于创建一个交互式的 Web 界面
import gradio as gr# 导入 mdtex2html 库用于将 Markdown 格式的文本转换为 HTML 格式
import mdtex2html# 使用 AutoModelForSeq2SeqLM 类从预训练模型 ZhipuAI/ChatGLM-6B 创建一个模型实例
# ZhipuAI/ChatGLM-6B 是一个序列到序列的语言模型用于文本生成任务
# mirrormodelscope 参数指定了模型的镜像站点为 modelscope以提高下载速度和稳定性.half()是将float32转为float16.
model AutoModelForSeq2SeqLM.from_pretrained(ZhipuAI/ChatGLM-6B, mirrormodelscope).half()# 将模型设置为评估模式即不进行训练
model.set_train(False)# 使用 AutoTokenizer 类从预训练模型 ZhipuAI/ChatGLM-6B 创建一个分词器实例
# 分词器用于将输入文本转换为模型可以理解的 tokens
tokenizer AutoTokenizer.from_pretrained(ZhipuAI/ChatGLM-6B, mirrormodelscope)
输出:
100%773/773 [00:0000:00, 50.6kB/s]
100%32.6k/32.6k [00:0000:00, 2.78MB/s]
Downloading shards: 100%8/8 [16:4300:00, 107.69s/it]
100%1.62G/1.62G [02:1400:00, 14.2MB/s]
100%1.75G/1.75G [02:1900:00, 16.8MB/s]
100%1.84G/1.84G [02:2700:00, 17.9MB/s]
100%1.78G/1.78G [02:2100:00, 17.5MB/s]
100%1.75G/1.75G [02:2000:00, 14.6MB/s]
100%1.75G/1.75G [02:2000:00, 18.7MB/s]
100%1.00G/1.00G [01:1800:00, 16.7MB/s]
100%1.00G/1.00G [01:1900:00, 8.88MB/s]
Loading checkpoint shards: 100%8/8 [00:5100:00, 5.84s/it]
100%441/441 [00:0000:00, 38.4kB/s]
100%2.58M/2.58M [00:0000:00, 6.99MB/s]1.3 进行推理
# 定义一个字符串变量 prompt内容为 你好表示用户的提问或对话输入
prompt 你好# 定义一个列表变量 history用于存储对话历史
# 在这个例子中对话历史为空因为这是新的对话
history []# 调用 model 的 chat 方法传入 tokenizer、prompt、history 和 max_length 参数
# chat 方法是模型的一个函数用于生成对话响应
# tokenizer 是之前加载的分词器用于处理输入文本
# prompt 是用户的输入文本
# history 是对话历史用于提供上下文信息
# max_length 是生成响应的最大长度
response, _ model.chat(tokenizer, prompt, historyhistory, max_length20)# 打印出模型的对话响应
print(response)
输出:
\
The dtype of attention mask (Float32) is not bool
|
你好我是人工智能助手 ChatGLM-6B响应非常地慢, 不知是什么原因. 根据npu监控信息来看, 一开始完全没有利用到ai core, 而是一直在跑内存. 直到开始调用ai core之后,很快就给出了响应. prompt 房间里有7只鸡和兔子,共20只脚.问有几只鸡,几只兔子?
history []
response, _ model.chat(tokenizer, prompt, historyhistory, max_length512)
response输出: 可以看到,他有正确的解题思路. 虽然数学稀烂,二元一次方程组都能解错. 耗时约200s, 输出250多个字符.如果算token的话,可能就是每秒1个token, 这个性能是真的拉垮… 当然,这是免费的虚拟资源并且很大概率是共享的可能并不能和实体的昇腾芯片相提并论。 2. 小结
本文主要介绍了使用mindnlp下载chatglm-6B预训练模型并基于此模型通过输入提示词完成回答文本生成的模型推理任务从而实现了一个简单的聊天应用。
