在 OpenBayes 运行通义千问Qwen-1.8B-Chat模型并进行微调

本教程主要演示在 OpenBayes 平台运行通义千问Qwen-1.8B-Chat模型并进行微调的主要流程。

在Jupyter工作空间中，打开第一个终端窗口

导航到Qwen源码路径，输入命令 cd Qwen

安装依赖，输入命令 pip install -r requirements.txt

开始微调，输入命令 bash finetune/finetune_lora_single_gpu.sh -m /input0/Qwen-1_8B-Chat -d finetune/train.txt

(base) root@cf9yunid7u5-y0a3nauzttqj-main:/openbayes/home/Qwen# bash finetune/finetune_lora_single_gpu.sh -m /input0/Qwen-1_8B-Chat -d finetune/train.txt
[2024-03-04 09:56:03,631] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to cuda (auto detect)
The model is automatically converting to bf16 for faster inference. If you want to disable the automatic precision, please manually add bf16/fp16/fp32=True to "AutoModelForCausalLM.from_pretrained".
Try importing flash-attention for faster inference...
Warning: import flash_attn rotary fail, please install FlashAttention rotary to get higher efficiency https://github.com/Dao-AILab/flash-attention/tree/main/csrc/rotary
Warning: import flash_attn rms_norm fail, please install FlashAttention layer_norm to get higher efficiency https://github.com/Dao-AILab/flash-attention/tree/main/csrc/layer_norm
Warning: import flash_attn fail, please install FlashAttention to get higher efficiency https://github.com/Dao-AILab/flash-attention
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  9.86it/s]
trainable params: 53,673,984 || all params: 1,890,502,656 || trainable%: 2.83913824874309
Loading data...
Formatting inputs...Skip in lazy mode
  0%|

启动api调用环境，输入命令 python openai_api_1_0_finetune.py

(base) root@cf9yunid7u5-y0a3nauzttqj-main:/openbayes/home/Qwen# python openai_api_1_0_finetune.py 
The model is automatically converting to bf16 for faster inference. If you want to disable the automatic precision, please manually add bf16/fp16/fp32=True to "AutoModelForCausalLM.from_pretrained".
Try importing flash-attention for faster inference...
Warning: import flash_attn rotary fail, please install FlashAttention rotary to get higher efficiency https://github.com/Dao-AILab/flash-attention/tree/main/csrc/rotary
Warning: import flash_attn rms_norm fail, please install FlashAttention layer_norm to get higher efficiency https://github.com/Dao-AILab/flash-attention/tree/main/csrc/layer_norm
Warning: import flash_attn fail, please install FlashAttention to get higher efficiency https://github.com/Dao-AILab/flash-attention
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00,  1.33it/s]
You are resizing the embedding layer without providing a `pad_to_multiple_of` parameter. This means that the new embeding dimension will be 151851. This might induce some performance reduction as *Tensor Cores* will not be available. For more details  about this, or help on choosing the correct value for resizing, refer to this guide: https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html#requirements-tc
API地址: https://cf9yunid7u5-y0a3nauzttqj.gear-c1.openbayes.net/v1/
INFO:     Started server process [526]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)

在Jupyter工作空间中，打开第二个终端窗口

导航到Home路径，输入命令：cd /output/

验证微调效果，输入命令：python validate_lora_ft.py

(base) root@cf9yunid7u5-y0a3nauzttqj-main:/openbayes/home# python validate_lora_ft.py 
Q:2020年4月16号三亚下雨么？
A:{"city": "三亚", "date": "2020-04-16"}

Q:青岛3-15号天气预报
A:{"city": "青岛", "date": "03-15"}

Q:5月6号下雪么，城市是威海
A:{"city": "威海", "date": "05-06"}

Q:重庆2023年12月30号有雾霾么?
A:{"city": "重庆", "date": "2023-12-30"}

相关链接：

通义千问 GitHub

public.tutorials/通义千问 Qwen-1.8B-Chat-API-FT Democlone

准备体验 OpenBayes？

public.tutorials
/
通义千问 Qwen-1.8B-Chat-API-FT Demo