README.md

---
license: mit
pipeline_tag: image-to-image
---

# CodeFormer

This version of CoderFormer has been converted to run on the Axera NPU using **w8a16** quantization.

This model has been optimized with the following LoRA: 

Compatible with Pulsar2 version: 5.0-patch1

## Convert tools links:

For those who are interested in model conversion, you can try to export axmodel through 

- [The repo of AXera Platform](https://github.com/AXERA-TECH/CodeFormer.axera), which you can get the detail of guide

- [Pulsar2 Link, How to Convert ONNX to axmodel](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html) 


## Support Platform

- AX650
  - [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
  - [M.2 Accelerator card](https://axcl-docs.readthedocs.io/zh-cn/latest/doc_guide_hardware.html)
  
|Chips|model|cost|
|--|--|--|
|AX650|coderformer|408 ms|

## How to use

Download all files from this repository to the device

```

root@ax650:~/coderformer# tree
.
|-- model
|   `-- coderformer.axmodel
|   `-- yolov5l-face.axmodel
|   `-- realesrgan-x2.axmodel
|-- python
|   `-- run_axmodel.py
|   `-- run_whole_image.py
|   `-- requirements.txt



```

### Inference

Input Data:

```
|-- images
|   -- face
|      `-- 00_00.png
|   -- image
|      `-- 02.png

```
#### Inference with M.2 Accelerator card
```
$cd python
$python3 gradio_demo.py
[INFO] Available providers:  ['AXCLRTExecutionProvider']
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 5.0-patch1 6d9cc640
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 5.0-patch1 681a0b38
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 4.2-dirty 5e72cf06-dirty

==================================================
🌐 人脸修复工具 Web UI 已启动！
🔗 本地访问: http://127.0.0.1:7860
🔗 局域网访问: http://10.126.33.124:7860
==================================================
```
Then open the link in the terminal to access the web UI.

![gradio face](./assert/gradio_face.JPG)
![gradio demo](./assert/gradio_demo.JPG)

#### Inference with AX650 Host, such as M4N-Dock(爱芯派Pro)

运行人脸修复demo（需输入人脸图片）
```
root@ax650 ~/codeformer #python3 run_axmodel.py --inputs_path ./images/face --model_path ./coderformer.axmodel

[INFO] Available providers:  ['AxEngineExecutionProvider']
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.12.0s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.0-patch1 681a0b38
SR image save to `00_00.png`
```

Output:
![Example Image](images/result_0.png)

运行整图修复demo（输入正常图片即可）
```
root@ax650 ~/codeformer #python3 run_whole_image.py --input_path ./images/image --detect_model ./yolov5l-face.axmodel --restore_model codeformer.axmodel --bg_model realesrgan-x2.axmodel


[INFO] Available providers:  ['AxEngineExecutionProvider']
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.12.0s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.0-patch1 6d9cc640
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.0-patch1 681a0b38
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 4.2-dirty 5e72cf06-dirty
[1/1] Processing: 02.png
        detect 4 faces

SR image save to `02.png`
```

Output:
![Example Image](images/result_1.png)