上云无忧 > 文档中心 > 第八章:精通深度学习的高级内容 - 飞桨场景应用开发套件-PaddleX
飞桨PaddlePaddle开源深度学习平台
第八章:精通深度学习的高级内容 - 飞桨场景应用开发套件-PaddleX

文档简介:
飞桨场景应用开发套件-PaddleX: PaddleX是飞桨场景应用开发套件,它集成飞桨智能视觉领域图像分类、目标检测、语义分割、实例分割任务能力,将深度学习开发全流程从数据准备、模型训练与优化到多端部署端到端打通,并提供统一任务API接口及图形化开发界面Demo。开发者无需分别安装不同套件,以低代码的形式即可快速完成飞桨全流程开发。
*此产品及展示信息均由百度智能云官方提供。免费试用 咨询热线:400-826-7010,为您提供专业的售前咨询,让您快速了解云产品,助您轻松上云! 微信咨询
  免费试用、价格特惠

飞桨场景应用开发套件-PaddleX

PaddleX是飞桨场景应用开发套件,它集成飞桨智能视觉领域图像分类、目标检测、语义分割、实例分割任务能力,将深度学习开发全流程从数据准备、模型训练与优化到多端部署端到端打通,并提供统一任务API接口及图形化开发界面Demo。开发者无需分别安装不同套件,以低代码的形式即可快速完成飞桨全流程开发。

PaddleX 经过质检、安防、巡检、遥感、零售、医疗等十多个行业实际应用场景验证,沉淀产业实际经验,并提供丰富的案例实践教程,全程助力开发者产业实践落地。

PaddleX提供可视化界面GUI和Python API两种使用模式,通过可视化界面,即使是不熟悉编码的用户也可以快速上手飞桨的使用。通过简明易懂的Python API,方便用户根据实际生产需求直接调用或二次开发,为开发者提供飞桨全流程开发的最佳实践。同时,PaddleX提供的Restful API可以让用户通过简单集成即可生产所在行业的专属AI工具。


PaddleX具备以下特点:

  1. 全流程打通:打通从数据接入到推理部署的深度学习开发全流程,简化各环节串联工作,大幅提升开发效率。
  2. 开源技术内核:集成飞桨领先的视觉算法和工具组件,提供简明易懂的Python API,完全开源开放,易于集成和二次开发。
  3. 产业深度兼容:沉淀质检、安防、遥感、零售、医疗等十多个行业实际应用场景的验证,有丰富的预置案例,并具备符合产业应用需要的诸多特点,如兼容Windows、Mac、Linux系统,支持GPU加速模型训练,支持本地开发以确保数据安全等。
  4. 完善的教程与服务:丰富的全流程开发文档,高效的技术服务支持,提供多种方式方便用户与技术团队直接交流。

PaddleX 快速使用方法

下面以MobileNetV3_ssld完成化妆品分类为例,介绍PaddleX训练模型方式。

该案例是创建一个识别化妆品的模型,输入时日常可见的各种化妆品图片,输出是化妆品的类别名称。如果读者是男性用户,该案例可以让大家在女朋友面前露一把脸。如果读者是女性用户,可以制作一个真懂化妆品的AI取悦自己。

MobileNetV3_ssld是通过SSLD(简单的半监督标签知识蒸馏)方式得到的新模型。相对比原有的MobileNetV3预训练模型,在参数量不变的情况下,MobileNetV3_ssld预训练模型在ImageNet数据集上的精度提升3%。

在下文中,我们将先后介绍API和可视化界面Demo两种使用PaddleX的方法。

PaddleX API的使用方法

1. 安装PaddleX

! pip install paddlex==1.3.11 -i https://mirror.baidu.com/pypi/simple

Looking in indexes: https://mirror.baidu.com/pypi/simple
Collecting paddlex==1.3.11
  Downloading https://mirror.baidu.com/pypi/packages/d6/a2/07435f4aa1e51fe22bdf06c
95d03bf1b78b7bc6625adbb51e35dc0804cc7/paddlex-1.3.11-py3-none-any.whl (516 kB) 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 517.0/
517.0 KB 15.8 MB/s eta 0:00:00 Requirement already satisfied: tqdm in /opt/conda/e
nvs/python35-paddle120-env/lib/python3.7/site-packages (from paddlex==1.3.11) (4.27.0)
Requirement already satisfied: psutil in /opt/conda/envs/python35-paddle120-env/lib
/python3.7/site-packages (from paddlex==1.3.11) (5.7.2)
Collecting paddleslim==1.1.1
  Downloading https://mirror.baidu.com/pypi/packages/d1/77/e257227bed9a70ff0d35a4a3c
4e70ac2d2362c803834c4c52018f7c4b762/paddleslim-1.1.1-py2.py3-none-any.whl (145 kB) 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 145.3/145.3
 KB 41.1 MB/s eta 0:00:00 Requirement already satisfied: colorama in /opt/conda/envs/p
ython35-paddle120-env/lib/python3.7/site-packages (from paddlex==1.3.11) (0.4.4)
Collecting paddlehub==2.1.0
  Downloading https://mirror.baidu.com/pypi/packages/7a/29/3bd0ca43c787181e9c22fe44b9
44b64d7fcb14ce66d3bf4602d9ad2ac76c/paddlehub-2.1.0-py3-none-any.whl (211 kB) ━━━━
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 211.4/211.4 KB 43
.8 MB/s eta 0:00:00 Collecting shapely>=1.7.0
  Downloading https://mirror.baidu.com/pypi/packages/d1/ec/3038263d69a0065d3ab6944ae839
f5f00896efd29b13ae62d73c00345b95/Shapely-1.8.2-cp37-cp37m-manylinux_2_12_x86_64.manylinu
x2010_x86_64.whl (2.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
━━━━━━━━━ 2.0/2.0 MB 10.9 MB/s eta 0:00:0000:0100:01
Requirement already satisfied: opencv-python in /opt/conda/envs/python35-paddle120-env/l
ib/python3.7/site-packages (from paddlex==1.3.11) (4.1.1.26)
Requirement already satisfied: flask-cors in /opt/conda/envs/python35-paddle120-env/lib/p
ython3.7/site-packages (from paddlex==1.3.11) (3.0.8)
Collecting pycocotools
  Downloading https://mirror.baidu.com/pypi/packages/75/5c/ac61ea715d7a89ecc31c090753bde2
8810238225ca8b71778dfe3e6a68bc/pycocotools-2.0.4.tar.gz (106 kB) ━━━━━━━━━━━━
━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.6/106.6 KB 12.4 MB/
s eta 0:00:00 Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: pyyaml in /opt/conda/envs/python35-paddle120-env/lib
/python3.7/site-packages (from paddlex==1.3.11) (5.1.2)
Requirement already satisfied: sklearn in /opt/conda/envs/python35-paddle120-env/lib/
python3.7/site-packages (from paddlex==1.3.11) (0.0)
Requirement already satisfied: visualdl>=2.0.0 in /opt/conda/envs/python35-paddle120-
env/lib/python3.7/site-packages (from paddlex==1.3.11) (2.2.3)
Collecting xlwt
  Downloading https://mirror.baidu.com/pypi/packages/44/48/def306413b25c3d01753603b1
a222a011b8621aed27cd7f89cbc27e6b0f4/xlwt-1.3.0-py2.py3-none-any.whl (99 kB) ━━━━
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.0/100.0 K
B 12.3 MB/s eta 0:00:00 Requirement already satisfied: paddlenlp>=2.0.0rc5 in /opt/
conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==2.1.0-
>paddlex==1.3.11) (2.1.1)
Collecting paddle2onnx>=0.5.1
  Downloading https://mirror.baidu.com/pypi/packages/cf/40/7ebb5e820e80b94dbd132164f6108
2df67f9588118580b93670543d6f7ad/paddle2onnx-0.9.5-cp37-cp37m-manylinux_2_17_x86_64.manyli
nux2014_x86_64.whl (2.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
━━━━━━━━━━ 2.7/2.7 MB 10.8 MB/s eta 0:00:0000:0100:01
Requirement already satisfied: easydict in /opt/conda/envs/python35-paddle120-env/lib/pyt
hon3.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (1.9)
Requirement already satisfied: pyzmq in /opt/conda/envs/python35-paddle120-env/lib/python3
.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (22.3.0)
Requirement already satisfied: gunicorn>=19.10.0 in /opt/conda/envs/python35-paddle120-env
/lib/python3.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (20.0.4)
Requirement already satisfied: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3
.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (1.19.5)
Requirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/py
thon3.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (2.2.3)
Requirement already satisfied: rarfile in /opt/conda/envs/python35-paddle120-env/lib/pyth
on3.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (3.1)
Requirement already satisfied: packaging in /opt/conda/envs/python35-paddle120-env/lib/py
thon3.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (21.3)
Requirement already satisfied: filelock in /opt/conda/envs/python35-paddle120-env/lib/pyt
hon3.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (3.0.12)
Requirement already satisfied: gitpython in /opt/conda/envs/python35-paddle120-env/lib/p
ython3.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (3.1.14)
Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/pytho
n3.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (8.2.0)
Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/pyt
hon3.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (4.1.0)
Requirement already satisfied: flask>=1.1.0 in /opt/conda/envs/python35-paddle120-env/lib/
python3.7/site-packages (from paddlehub==2.1.0->paddlex==1.3.11) (1.1.1)
Requirement already satisfied: pre-commit in /opt/conda/envs/python35-paddle120-env/lib/p
ython3.7/site-packages (from visualdl>=2.0.0->paddlex==1.3.11) (1.21.0)
Requirement already satisfied: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/
lib/python3.7/site-packages (from visualdl>=2.0.0->paddlex==1.3.11) (3.14.0)
Requirement already satisfied: Flask-Babel>=1.0.0 in /opt/conda/envs/python35-paddle120-en
v/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlex==1.3.11) (1.0.0)
Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/l
ib/python3.7/site-packages (from visualdl>=2.0.0->paddlex==1.3.11) (0.8.53)
Requirement already satisfied: six>=1.14.0 in /opt/conda/envs/python35-paddle120-env/lib/p
ython3.7/site-packages (from visualdl>=2.0.0->paddlex==1.3.11) (1.16.0)
Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/pyt
hon3.7/site-packages (from visualdl>=2.0.0->paddlex==1.3.11) (2.24.0)
Requirement already satisfied: shellcheck-py in /opt/conda/envs/python35-paddle120-env/li
b/python3.7/site-packages (from visualdl>=2.0.0->paddlex==1.3.11) (0.7.1.1)
Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/pyth
on3.7/site-packages (from visualdl>=2.0.0->paddlex==1.3.11) (1.1.5)
Requirement already satisfied: flake8>=3.7.9 in /opt/conda/envs/python35-paddle120-env/lib
/python3.7/site-packages (from visualdl>=2.0.0->paddlex==1.3.11) (4.0.1)
Requirement already satisfied: scikit-learn in /opt/conda/envs/python35-paddle120-env/lib
/python3.7/site-packages (from sklearn->paddlex==1.3.11) (0.24.2)
Requirement already satisfied: pycodestyle<2.9.0,>=2.8.0 in /opt/conda/envs/python35-pad
dle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl>=2.0.0->paddlex==1.3.11) (2.8.0)
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-
env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl>=2.0.0->paddlex==1.3.11) (0.6.1)
Requirement already satisfied: importlib-metadata<4.3 in /opt/conda/envs/python35-paddle120-en
v/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl>=2.0.0->paddlex==1.3.11) (4.2.0)
Requirement already satisfied: pyflakes<2.5.0,>=2.4.0 in /opt/conda/envs/python35-paddle120-
env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl>=2.0.0->paddlex==1.3.11) (2.4.0)
Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/p
ython3.7/site-packages (from flask>=1.1.0->paddlehub==2.1.0->paddlex==1.3.11) (0.16.0)
Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/l
ib/python3.7/site-packages (from flask>=1.1.0->paddlehub==2.1.0->paddlex==1.3.11) (1.1.0)
Requirement already satisfied: click>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/pytho
n3.7/site-packages (from flask>=1.1.0->paddlehub==2.1.0->paddlex==1.3.11) (7.0)
Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/envs/python35-paddle120-env/lib/p
ython3.7/site-packages (from flask>=1.1.0->paddlehub==2.1.0->paddlex==1.3.11) (3.0.0)
Requirement already satisfied: Babel>=2.3 in /opt/conda/envs/python35-paddle120-env/lib/pytho
n3.7/site-packages (from Flask-Babel>=1.0.0->visualdl>=2.0.0->paddlex==1.3.11) (2.8.0)
Requirement already satisfied: pytz in /opt/conda/envs/python35-paddle120-env/lib/python3.7/
site-packages (from Flask-Babel>=1.0.0->visualdl>=2.0.0->paddlex==1.3.11) (2019.3)
Requirement already satisfied: setuptools>=3.0 in /opt/conda/envs/python35-paddle120-env/lib/
python3.7/site-packages (from gunicorn>=19.10.0->paddlehub==2.1.0->paddlex==1.3.11) (56.2.0)
Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env
/lib/python3.7/site-packages (from matplotlib->paddlehub==2.1.0->paddlex==1.3.11) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/
lib/python3.7/site-packages (from matplotlib->paddlehub==2.1.0->paddlex==1.3.11) (1.1.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /opt/conda/envs/p
ython35-paddle120-env/lib/python3.7/site-packages (from matplotlib->paddlehub==2.1.0->paddlex==1.3.11) (3.0.8)
Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/pyt
hon3.7/site-packages (from matplotlib->paddlehub==2.1.0->paddlex==1.3.11) (0.10.0)
Collecting onnx<=1.9.0 Downloading https://mirror.baidu.com/pypi/packages/3f/9b/54c950d3256e27
f970a83cd0504efb183a24312702deed0179453316dbd0/onnx-1.9.0-cp37-cp37m-manylinux2010_x86_64.whl 
(12.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
2.2/12.2 MB 10.0 MB/s eta 0:00:0000:0100:01
Requirement already satisfied: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.
7/site-packages (from paddlenlp>=2.0.0rc5->paddlehub==2.1.0->paddlex==1.3.11) (1.2.2)
Requirement already satisfied: paddlefsl==1.0.0 in /opt/conda/envs/python35-paddle120-env/li
b/python3.7/site-packages (from paddlenlp>=2.0.0rc5->paddlehub==2.1.0->paddlex==1.3.11) (1.0.0)
Requirement already satisfied: multiprocess in /opt/conda/envs/python35-paddle120-env/lib/pytho
n3.7/site-packages (from paddlenlp>=2.0.0rc5->paddlehub==2.1.0->paddlex==1.3.11) (0.70.11.1)
Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.7/s
ite-packages (from paddlenlp>=2.0.0rc5->paddlehub==2.1.0->paddlex==1.3.11) (0.42.1)
Requirement already satisfied: h5py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/si
te-packages (from paddlenlp>=2.0.0rc5->paddlehub==2.1.0->paddlex==1.3.11) (2.9.0)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/pytho
n3.7/site-packages (from requests->visualdl>=2.0.0->paddlex==1.3.11) (2.8)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/li
b/python3.7/site-packages (from requests->visualdl>=2.0.0->paddlex==1.3.11) (2019.9.11)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/li
b/python3.7/site-packages (from requests->visualdl>=2.0.0->paddlex==1.3.11) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/py
thon35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl>=2.0.0->paddlex==1.3.11) (1.25.6)
Requirement already satisfied: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/py
thon3.7/site-packages (from bce-python-sdk->visualdl>=2.0.0->paddlex==1.3.11) (0.18.0)
Requirement already satisfied: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/
lib/python3.7/site-packages (from bce-python-sdk->visualdl>=2.0.0->paddlex==1.3.11) (3.9.9)
Requirement already satisfied: gitdb<5,>=4.0.1 in /opt/conda/envs/python35-paddle120-env/lib
/python3.7/site-packages (from gitpython->paddlehub==2.1.0->paddlex==1.3.11) (4.0.5)
Requirement already satisfied: cfgv>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/p
ython3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlex==1.3.11) (2.0.1)
Requirement already satisfied: toml in /opt/conda/envs/python35-paddle120-env/lib/python3
.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlex==1.3.11) (0.10.0)
Requirement already satisfied: virtualenv>=15.2 in /opt/conda/envs/python35-paddle120-e
nv/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlex==1.3.11) (16.7.9)
Requirement already satisfied: aspy.yaml in /opt/conda/envs/python35-paddle120-env/lib/py
thon3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlex==1.3.11) (1.3.0)
Requirement already satisfied: identify>=1.0.0 in /opt/conda/envs/python35-paddle120-env/l
ib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlex==1.3.11) (1.4.10)
Requirement already satisfied: nodeenv>=0.11.1 in /opt/conda/envs/python35-paddle120-env/li
/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlex==1.3.11) (1.3.4)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/envs/python35-paddle120-en
v/lib/python3.7/site-packages (from scikit-learn->sklearn->paddlex==1.3.11) (2.1.0)
Requirement already satisfied: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/py
thon3.7/site-packages (from scikit-learn->sklearn->paddlex==1.3.11) (0.14.1)
Requirement already satisfied: scipy>=0.19.1 in /opt/conda/envs/python35-paddle120-env/lib/py
thon3.7/site-packages (from scikit-learn->sklearn->paddlex==1.3.11) (1.6.3)
Requirement already satisfied: smmap<4,>=3.0.1 in /opt/conda/envs/python35-paddle120-env/lib/
python3.7/site-packages (from gitdb<5,>=4.0.1->gitpython->paddlehub==2.1.0->paddlex==1.3.11) (3.0.5)
Requirement already satisfied: typing-extensions>=3.6.4 in /opt/conda/envs/python35-paddle12
0-env/lib/python3.7/site-packages (from importlib-metadata<4.3->flake8>=3.7.9->visualdl>=2.0
.0->paddlex==1.3.11) (4.2.0)
Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/pyth
on3.7/site-packages (from importlib-metadata<4.3->flake8>=3.7.9->visualdl>=2.0.0->paddlex==1.3.11) (3.8.0)
Requirement already satisfied: MarkupSafe>=2.0.0rc2 in /opt/conda/envs/python35-paddle120-
env/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask>=1.1.0->paddlehub==2.1.0->paddlex==1.3.11) (2.0.1)
Requirement already satisfied: dill>=0.3.3 in /opt/conda/envs/python35-paddle120-env/lib/
python3.7/site-packages (from multiprocess->paddlenlp>=2.0.0rc5->paddlehub==2.1.0->paddlex==1.3.11) (0.3.3)
Building wheels for collected packages: pycocotools
  Building wheel for pycocotools (pyproject.toml) ... done
  Created wheel for pycocotools: filename=pycocotools-2.0.4-cp37-cp37m-linux_x86_64.whl 
size=273789 sha256=bafea6d86df5bf63e6dc2e6183a323192674c734375255b9c330cc05479f701b
  Stored in directory: /home/aistudio/.cache/pip/wheels/d0/74/13/98b11419a029
f3c25590419747f1ec26f5494beae1d457560b
Successfully built pycocotools
Installing collected packages: xlwt, shapely, paddleslim, onnx, pycocotools, paddle2onnx, paddlehub, paddlex
  Attempting uninstall: paddlehub
    Found existing installation: paddlehub 2.0.4
    Uninstalling paddlehub-2.0.4:
      Successfully uninstalled paddlehub-2.0.4
Successfully installed onnx-1.9.0 paddle2onnx-0.9.5 paddlehub-2.1.0 paddleslim-
1.1.1 paddlex-1.3.11 pycocotools-2.0.4 shapely-1.8.2 xlwt-1.3.0

3. 训练准备

3.1 配置训练环境,并导入PaddleX库

# jupyter中使用paddlex需要设置matplotlib import matplotlib
matplotlib.use('Agg') # 设置使用0号GPU卡(如无GPU,执行此代码后仍然会使用CPU训练模型) import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0' import paddlex as pdx

3.2 定义图像处理流程transforms

定义训练和验证过程中的图像处理流程,其中训练过程包括了部分数据增强操作(验证时不需要),如在本示例中,训练过程使用了RandomCrop和RandomHorizontalFlip两种数据增强方式,分别代表随机裁剪和随机翻转,并做了图像输入的标准化。验证数据集则做了调整到统一的尺寸。更多图像预处理流程transforms的使用可参见paddlex.cls.transforms。

from paddlex.cls import transforms
train_transforms = transforms.Compose([
    transforms.RandomCrop(crop_size=224),
    transforms.RandomHorizontalFlip(),
    transforms.Normalize()
])
eval_transforms = transforms.Compose([
    transforms.ResizeByShort(short_size=256),
    transforms.CenterCrop(crop_size=224),
    transforms.Normalize()
])

3.3 定义数据集Dataset

使用PaddleX内置的数据集读取器读取训练和验证数据集,并应用上面配置的图像处理流程。本示例采用ImageNet数据集格式,因此这里采用pdx.datasets.ImageNet来加载数据集,该接口的介绍可参见文档paddlex.datasets.VOCDetection。

train_dataset = pdx.datasets.ImageNet(
    data_dir='makeup',
    file_list='makeup/train_list.txt',
    label_list='makeup/labels.txt',
    transforms=train_transforms,
    shuffle=True)
eval_dataset = pdx.datasets.ImageNet(
    data_dir='makeup',
    file_list='makeup/val_list.txt',
    label_list='makeup/labels.txt',
    transforms=eval_transforms)
2022-05-06 17:06:50 [INFO]	Starting to read file list from dataset...
2022-05-06 17:06:50 [INFO]	710 samples in file makeup/train_list.txt
2022-05-06 17:06:50 [INFO]	Starting to read file list from dataset...
2022-05-06 17:06:50 [INFO]	132 samples in file makeup/val_list.txt

3.4 开始训练模型

在定义好数据集后,即可选择分类模型(这里使用了MobileNetV3_large_ssld模型)开始训练。MobileNetV3_large是面向移动端应用场景的模型,而MobileNetV3_large_ssld是百度通过SSLD蒸馏策略所得的模型,具有更高的精度表现。
关于分类模型训练,更多参数介绍可参见文档paddlex.cls.MobileNetV3_large_ssld。在如下代码中,模型训练过程每间隔save_interval_epochs轮次,会保存一次模型在save_dir目录下,同时在保存的过程中也会在验证数据集上计算相关指标,模型训练过程中相关日志的含义可参见文档。

注:本数据集在P40 GPU上训练MobileNetV3_large_ssld,模型的训练过程预估为10分钟左右;如无GPU,则预估为30分钟左右。

num_classes = len(train_dataset.labels)
model = pdx.cls.MobileNetV3_large_ssld(num_classes=num_classes)
model.train(num_epochs=10,
            train_dataset=train_dataset,
            train_batch_size=32,
            eval_dataset=eval_dataset,
            lr_decay_epochs=[4, 6, 8],
            save_interval_epochs=1,
            learning_rate=0.025,
            save_dir='output/mobilenetv3_large_ssld')

/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers
/math_op_patch.py:341: UserWarning: /opt/conda/envs/python35-paddle120-env/lib/python3.7
/site-packages/paddlex/cv/nets/mobilenet_v3.py:231
The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from
 Paddle 2.0. If your code works well in the older versions but crashes in this version,
 try to use elementwise_mul(X, Y, axis=0) instead of A * B. This 
transitional warning will be dropped in the future.
  op_type, op_type, EXPRESSION_MAP[method_name]))

2022-05-06 17:06:52 [INFO]	Decompressing output/mobilenetv3_large_ssld/pr
etrain/MobileNetV3_large_x1_0_ssld_pretrained.tar...

W0506 17:06:52.906878   498 gpu_context.cc:244] Please NOTE: device: 0, GPU Compute 
Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
W0506 17:06:52.913384   498 gpu_context.cc:272] device: 0, cuDNN Version: 7.6.

4. 模型预测

result = model.predict('makeup/mascara/27.jpeg', topk=1)
print("Predict Result:", result)

当然,PaddleX的功能不止这么简单,这是一个极简功能的展示案例。实际上,PaddleX可以和PaddleSeg、PaddleClas、PaddleDetection等开发套件一样,实现非常丰富的模型和训练配置。如果读者对这些功能感兴趣,欢迎继续查阅PaddleX的文档,或者可以通过PaddleX官方提供的图形化界面Demo了解。该Demo完整展示了基于PaddleX的API可以完成的功能,而且PaddleX的界面读者可以根据自己的需要重新设计。

PaddleX客户端使用方法

PaddleX 图形化开发界面

为了帮助开发者更好的了解飞桨的开发步骤以及所涉及的模块组件,进一步提升项目开发效率,飞桨为开发者提供了基于PaddleX实现的图形化开发界面产品,用户可以基于该界面示例进行改造,开发符合自己习惯的操作界面。开发者可以根据实际业务需求,直接调用或改造PaddleX后端技术内核来开发项目,或使用图形化开发界面快速体验飞桨模型开发全流程。

1. 下载PaddleX客户端。

您需要前往官网填写基本信息后下载试用PaddleX客户端。

2. 准备数据

在开始模型训练前,需要根据不同的任务类型,将数据标注为相应的格式。目前PaddleX支持图像分类、目标检测、语义分割、实例分割四种任务类型。不同类型任务的数据处理方式可查看数据集格式说明。

3. 导入数据集

(1)数据标注完成后,您需要根据不同的任务,将数据和标注文件,按照客户端提示更名并保存到正确的文件中。

(2)在客户端新建数据集,选择与数据集匹配的任务类型,并选择数据集对应的路径,将数据集导入。


如果想用自己的数据集,可以上传自己的数据集,这里我们以化妆品分类为例,上传化妆品的数据集。


上传成功后,会出现如下界面,如果需要重新划分训练集、验证集、测试集时,可以选择重新划分:

  • 训练集:用来训练模型
  • 验证集:中间小测验,用于进行模型评估,找到最优模型
  • 测试集:最终测试模型在现实场景的泛化误差,避免过拟合

4.创建项目

(1)在完成数据导入后,可点击“新建项目”创建一个项目。

(2)可根据实际任务需求选择项目的任务类型,需要注意项目所采用的数据集也带有任务类型属性,两者需要进行匹配。


5.项目开发

(1)选择数据:项目创建完成后,需要选择已载入客户端并校验后的数据集,并点击“下一步”,进入参数配置页面。



(2)配置参数:主要分为模型参数、训练参数、优化策略三部分。可根据实际需求选择模型结构及对应的训练参数、优化策略,使得任务效果最佳。



另外,可以在客户端中选择不同的数据增强方式:



参数配置完成后,点击“启动训练”,模型开始训练并进行效果评估。

(3)训练可视化:在训练过程中,可通过VisualDL查看模型训练过程时的参数变化、日志详情,及当前最优的训练集和验证集训练指标。模型在训练过程中通过点击“终止训练”随时终止训练过程。



PaddleX集成了飞桨可视化分析工具VisualDL,可以很方便地查看训练过程的指标数据。



模型训练结束后,点击”下一步“,从客户端中,也可以看到训练的完成进度和验证集精度。

(4)模型发布:当模型效果满意后,可根据实际的生产环境需求,将模型发布为需要的版本。



注意:这个带可视化界面的AI研发软件仅仅是基于PaddleX API做出来的一个Demo。受此启发,欢迎读者使用PaddleX API研发一款适合自己所在企业或行业使用的AI研发工具,图形化界面的软件功能可以根据场景的需要来灵活定制。

往届优秀学员作品展示

PaddleX-API扩展预览transform的效果

  1. 项目背景

我们在做深度学习计算机视觉项目的时候,经常会用到图像增广数据增强的操作,可是操作后的结果是什么样子呢?此项目带你揭开Transform的神秘面纱。

  1. 项目内容

使用tkinter的操作界面,可视化显示图像增强之后的图像形式。

  1. 实现方案

使用PaddleX的API扩展,预览Transform的效果。

  1. 实现结果


  1. 项目点评

不仅应用PaddleX GUI实现了应用自有数据对模型进行训练,增加了对卸妆水/卸妆油品类的识别,又自主基于PaddleX API完成了对数据增强Transform的可视化呈现工程。

  1. 项目链接 https://aistudio.baidu.com/aistudio/projectdetail/545565

总结

在本节中,简单的介绍了PaddleX的训练模型的使用方式,更多的使用请参考官方文档和PaddleX的Github。

相似文档
  • 人工智能在中国的发展和落地概况: 根据艾瑞的分析报告,人工智能在未来十年迎来落地应用的黄金期,会全面赋能实体经济,行业的经济规模年增长率达40%+。在过去中国经济高速发展的四十年,人们形成了统一的认知:对于个人发展,选择大于能力。一个人选择跳上一辆高速行驶的火车,比个人奔跑快要重要。人工智能在各行业落地相关的产业就是未来十年的高速列车,所以恭喜学习本教程的诸位读者。在可预见的未来,大家会成为各行业应用人工智能技术的弄潮儿。
  • Hi 大家好, 我是百度AI Studio小助手. 大家在学习《百度架构师手把手带你零基础入门深度学习》课程的过程中将会经常使用到AI Studio中的Notebook项目,所以今天给大家介绍一些Notebook项目基本操作.
  • Python数据结构、 Python面向对象、 Python JSON、 Python 异常处理、 常见Linux命令。 Python数据结构: 数字、字符串、列表、元组、字典。
  • 为什么图片能被计算机读取?为什么我们可以用CNN对成千上万中图片进行分类,这背后的原理是什么?在了解原理之前,先给大家补点数学知识。因为无论是深度学习还是机器学习,背后都是有一些数学原理和公式推导的,所以掌握必备的数学知识必不可少,下面会给大家简单科普下常用的数学知识有哪些~
  • 环境准备: 处理器:SW6A。 操作系统:普华, iSoft Linux 5。 Python 版本 2.7.15+/3.5.1+/3.6/3.7/3.8 (64 bit)。 pip 或 pip3 版本 9.0.1+ (64 bit)。 申威机器为 SW 架构,目前生态支持的软件比较有限,本文以比较 trick 的方式在申威机器上源码编译 Paddle,未来会随着申威软件的完善不断更新。
官方微信
联系客服
400-826-7010
7x24小时客服热线
分享
  • QQ好友
  • QQ空间
  • 微信
  • 微博
返回顶部