做淘客网站需要什么,网站空间商,临沂酒店建设信息网站,校园二手网站的建设方案Chatgpt的出现给NLP领域带来了让人振奋的消息#xff0c;可以很逼真的模拟人的对话#xff0c;回答人们提出的问题#xff0c;不过Chatgpt参数量#xff0c;规模#xff0c;训练代价都很昂贵。
幸运的是#xff0c;出现了开源的一些相对小的模型#xff0c;可以在本地或…Chatgpt的出现给NLP领域带来了让人振奋的消息可以很逼真的模拟人的对话回答人们提出的问题不过Chatgpt参数量规模训练代价都很昂贵。
幸运的是出现了开源的一些相对小的模型可以在本地或者云端部署体验
动手体验了下Vicuna-7b翻译过来是小羊驼」骆马拥有70亿参数据作者实验能达到GPT-4的90%性能。在作者官网发布了三个版本其中3个月前发布了v1.117天前发布了v1.3。
官网lmsys (Large Model Systems Organization)
环境
ubuntu18.04
九天毕昇8核32G内存仅有cpu
时间2023年7月5号
llama-7B
python3 -m cli --model-path /root/transformer_model_7b --device cpu 文件cli是fastchat/serve目录下的cli.py文件
cli.py内容 Chat with a model with command line interface. Usage: python3 -m fastchat.serve.cli --model lmsys/vicuna-7b-v1.3 python3 -m fastchat.serve.cli --model lmsys/fastchat-t5-3b-v1.0 Other commands: - Type !!exit or an empty line to exit. - Type !!reset to start a new conversation. import argparse import os import re import sys from prompt_toolkit import PromptSession from prompt_toolkit.auto_suggest import AutoSuggestFromHistory from prompt_toolkit.completion import WordCompleter from prompt_toolkit.history import InMemoryHistory from prompt_toolkit.key_binding import KeyBindings from rich.console import Console from rich.live import Live from rich.markdown import Markdown from fastchat.model.model_adapter import add_model_args from fastchat.modules.gptq import GptqConfig from fastchat.serve.inference import ChatIO, chat_loop class SimpleChatIO(ChatIO): def prompt_for_input(self, role) - str: return input(f{role}: )
参数 --model-path 是指模型权重存放路径这个目录下包含tokenizer.model文件请仔细核对
参数 --device cpu 指定在cpu上运行
开始推理 输入 who is Musk? what is baidu? china
要等待几分钟看到几个词几个词的蹦出来
看回答字数还不少内容只能说是相关
看下cpu使用率
top 可以看到python3进程推理时cpu使用率在81.1%一直在波动。
换用v100有时候会到98%甚至更高速度会快很多一次输出一小段文字
vicuna-7b-delta-v1.1
python3 -m cli --model-path /root/vicuna-7b/vicuna-7b-delta-v11 --device cpu 加载模型中间过程要几分钟
python3 -m cli --model-path /root/vicuna-7b/vicuna-7b-delta-v11 --device cpu Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████| 2/2 [02:5800:00, 89.16s/it]
开始推理实现一个python斐波那契函数可以看到能够写出函数还简单介绍了怎么用
USER: hello ASSISTANT: Hello! How can I help you today? Is there something you would like to know or talk about? Im here to help answer any questions you may have. USER: implement a python function to compute fibonacci numbers ASSISTANT: Sure! Heres an implementation of a Python function that computes the Fibonacci sequence: python def fibonacci(n): if n 1: return n else: return fibonacci(n-1) fibonacci(n-2) This function takes an integer n as input and returns the n-th Fibonacci number. The first two numbers in the sequence are 0 and 1, so the function handles these cases separately: if n is less than or equal to 1, it returns n, and otherwise it computes the sum of the previous two Fibonacci numbers and returns that.
To use the function, you can call it with an integer n and it will return the n-th Fibonacci number. For example: python fibonacci(5) 13 This would compute the 5-th Fibonacci number, which is 13.
USER: china ASSISTANT: China is a country located in East Asia, bordered by several other countries, including Russia, Mongolia, North Korea, Vietnam, Laos, Myanmar (Burma), Bhutan, India, Nepal, and Pakistan. It is the worlds most populous country, with over 1.4 billion people.
China has a long and complex history, dating back over 5,000 years. It is known for its rich cultural heritage, including its cuisine, literature, and arts. The country has a diverse range of landscapes, from the high plateaus in the west to the fertile river valleys in the east, and from the tropical beaches in the south to the frigid deserts in the north.
vicuna-7b-v1.3 python3 -m cli --model-path /root//vicuna-7b-v13 --device cpu 参数-m 后面指定cli.py文件注意【.py】省略。python3 -m cli是python3执行cli.py文件具体内容要看cli.py写了什么
加载模型等待几分钟
python3 -m cli --model-path /root//vicuna-7b-v13 --device cpu Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████| 2/2 [03:1700:00, 98.73s/it]
推理
USER: Introduce China
ASSISTANT: China, officially known as the Peoples Republic of China (PRC), is a sovereign state located in East Asia. It is the worlds most populous country, with a population of over 1.4 billion people, and the third-largest country by land area, covering approximately 9.6 million square kilometers.
China has a long and complex history, dating back more than 5,000 years. It has been ruled by various dynasties, including the Zhou, Qin, Han, Tang, Song, Yuan, Ming, and Qing dynasties, among others.
The Peoples Republic of China is a single-party socialist state with a planned economy, guided by the CPC. The country is governed according to the Constitution of the Peoples Republic of China, which was adopted in 1982. The government is divided into various branches, including the National Peoples Congress (NPC), the State Council, the Supreme Peoples Court, and the Central Military Commission.
China has a diverse landscape and culture, encompassing various ethnic groups, including the Han, Tibetans, Uighurs, Mongols, and Manchus, among others. The official language is Standard Chinese (Mandarin), although there are numerous regional languages and dialects spoken throughout the country.
The Peoples Republic of China is a single-party socialist state with a planned economy, guided by the CPC. The country is governed according to the Constitution of the Peoples Republic of China, which was adopted in 1982. The government is divided into various branches, including the National Peoples Congress (NPC), the State Council, the Supreme Peoples Court, and the Central Military Commission.
China has a diverse landscape and culture, encompassing various ethnic groups, including the Han, Tibetans, Uighurs, Mongols, and Manchus, among others. The official language is Standard Chinese (Mandarin), although there are numerous regional languages and dialects spoken throughout the country.
The economy of China has experienced rapid growth in recent decades, making it the worlds second-largest economy by nominal GDP and the largest by purchasing power parity (PPP). The country is a leading global exporter of goods and services, with industries such as electronics, textiles, automobiles, and telecommunications playing a significant role in its economic development.
China has been actively involved in international affairs, participating as a permanent member of the United Nations Security Council and a member of various multilateral organizations, including the World Trade Organization (WTO), the Asia-Pacific Economic Cooperation (APEC), and the Group of 20 (G20) major economies.
USER: china ASSISTANT: China is a country located in East Asia, covering approximately 9.6 million square kilometers. With a population of over 1.4 billion people, it is the most populous country in the world. The official language is Mandarin Chinese, and the capital city is Beijing. China has a rich history and a diverse cultural heritage, with many different ethnic groups and over 56 recognized ethnic minorities.
目前尚不能支持中文提问和中文回答
提问方式很重要输入china回答的只能说是相关文字就少很多。输入 Introduce China输出大段的介绍
