fixed the readmes
Browse files
README.md
CHANGED
|
@@ -12,9 +12,11 @@ If you'd like to run these models in your own ExecuTorch runtime, refer to the
|
|
| 12 |
[official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
|
| 13 |
|
| 14 |
Each language ships as **one fused `.pte`** (CRAFT *detect* + CRNN *recognize* in a single
|
| 15 |
-
file) per backend,
|
| 16 |
-
is a pure tensor→tensor function; all
|
| 17 |
-
crop, CTC decode) is the client's job
|
|
|
|
|
|
|
| 18 |
|
| 19 |
## Languages
|
| 20 |
|
|
@@ -27,35 +29,57 @@ crop, CTC decode) is the client's job and is driven by `config.json`.
|
|
| 27 |
|
| 28 |
All languages share the same CRAFT detector and CRNN architecture — they differ **only** in
|
| 29 |
the recognizer charset. The detector half of each fused PTE is identical across languages.
|
|
|
|
| 30 |
|
| 31 |
-
##
|
| 32 |
|
| 33 |
-
|
|
| 34 |
|---|---|---|
|
| 35 |
-
| `
|
| 36 |
-
| `
|
| 37 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
-
|
| 43 |
-
config.json # shared base config (schema v1)
|
| 44 |
-
charsets/easyocr_<lang>.charset.txt # per-language CTC charset (JSON array)
|
| 45 |
-
<lang>/<backend>/easyocr_<lang>_<backend>_bucketed.pte
|
| 46 |
-
```
|
| 47 |
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
|
|
|
|
|
|
| 51 |
|
| 52 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
|
| 54 |
-
|
| 55 |
|
| 56 |
-
- **
|
| 57 |
-
|
| 58 |
-
|
|
|
|
| 59 |
|
| 60 |
## Compatibility
|
| 61 |
|
|
@@ -65,4 +89,3 @@ the compatibility note in the
|
|
| 65 |
[ExecuTorch GitHub repository](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md).
|
| 66 |
If you work with React Native ExecuTorch, the library constants guarantee compatibility with the
|
| 67 |
runtime used behind the scenes.
|
| 68 |
-
|
|
|
|
| 12 |
[official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
|
| 13 |
|
| 14 |
Each language ships as **one fused `.pte`** (CRAFT *detect* + CRNN *recognize* in a single
|
| 15 |
+
file) per backend, with a single **dynamic** `detect` method and one fixed-width `recognize`
|
| 16 |
+
method (no per-size method buckets). The `.pte` is a pure tensor→tensor function; all
|
| 17 |
+
pre/post-processing (resize, normalize, box extraction, crop, CTC decode) is the client's job
|
| 18 |
+
and is driven by `config.json`. EasyOCR is the *fallback* pipeline —
|
| 19 |
+
[PP-OCRv6](https://huggingface.co/software-mansion/react-native-executorch-pp-ocrv6) is primary.
|
| 20 |
|
| 21 |
## Languages
|
| 22 |
|
|
|
|
| 29 |
|
| 30 |
All languages share the same CRAFT detector and CRNN architecture — they differ **only** in
|
| 31 |
the recognizer charset. The detector half of each fused PTE is identical across languages.
|
| 32 |
+
Charset index `i` maps to logit `i + 1` (logit `0` is the CTC blank).
|
| 33 |
|
| 34 |
+
## Methods & I/O contract
|
| 35 |
|
| 36 |
+
| method | input | output |
|
| 37 |
|---|---|---|
|
| 38 |
+
| `detect` (CRAFT) | `[1,3,H,W]` f32 RGB, **ImageNet-normalized by the client**: `(x/255 − mean)/std`, `mean=[0.485,0.456,0.406]`, `std=[0.229,0.224,0.225]` | score `[1,H/2,W/2,2]` (region + affinity, NHWC) and feature `[1,32,H/2,W/2]` |
|
| 39 |
+
| `recognize` (CRNN) | `[1,3,64,512]` f32 RGB, client-normalized `(x/255 − 0.5)/0.5` (RGB→gray conv is baked) | `[1,127,V]` probs (softmax baked) |
|
| 40 |
+
|
| 41 |
+
**Nothing is baked for input normalization** — the client normalizes before calling, with
|
| 42 |
+
*different* norms per method (ImageNet for detect, `0.5/0.5` for recognize).
|
| 43 |
+
|
| 44 |
+
## Shape discovery (companion methods)
|
| 45 |
+
|
| 46 |
+
Every method carries exactly **one** no-arg discovery companion:
|
| 47 |
|
| 48 |
+
- `get_dynamic_dims_<method>` — dynamic method. One `int32 [rank, 3]` tensor per tensor
|
| 49 |
+
input; rows are `[min, max, step]` (static dims `[n, n, 1]`).
|
| 50 |
+
- `get_enum_shapes_<method>` — enumerated/fixed method. One `int32 [N, rank]` tensor per
|
| 51 |
+
tensor input; each row is a complete legal shape (cross-dimension coupling is exact —
|
| 52 |
+
`1280×320` listed does **not** imply `320×1280`). Snap inputs to the nearest row.
|
| 53 |
|
| 54 |
+
| backend | `detect` | `recognize` |
|
| 55 |
+
|---|---|---|
|
| 56 |
+
| `xnnpack` | `get_dynamic_dims_detect` → H, W ∈ `[320, 1280]` step 32 | `get_enum_shapes_recognize` → `[[1,3,64,512]]` |
|
| 57 |
+
| `vulkan` | `get_dynamic_dims_detect` → H ∈ `[800, 1280]`, W ∈ `[320, 1280]` step 32 | `get_enum_shapes_recognize` → `[[1,3,64,512]]` |
|
| 58 |
+
| `coreml` | `get_enum_shapes_detect` → `800²`, `1280²`, `1280×320` | `get_enum_shapes_recognize` → `[[1,3,64,512]]` |
|
| 59 |
+
|
| 60 |
+
`detect` runs once per image; `recognize` runs once per text line (the client snaps every
|
| 61 |
+
crop to width 512 — the CRNN's BiLSTM only delegates at a fixed time dimension).
|
| 62 |
|
| 63 |
+
## Backends
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
|
| 65 |
+
| backend | target | detect | recognize | warm latency (detect @800² / recognize) |
|
| 66 |
+
|---|---|---|---|---|
|
| 67 |
+
| `xnnpack` | CPU | int8, dynamic (see note) | int8 @512 | ~810 ms / ~24 ms (Galaxy S24) |
|
| 68 |
+
| `coreml` | Apple ANE | weight-only int8, enumerated | weight-only int8 @512 | ~83 ms / ~27 ms (Apple M-series ANE) |
|
| 69 |
+
| `vulkan` | Android GPU | fp16, dynamic (resize) | int8 @512 on **XNNPACK** (mixed-delegate) | ~750 ms / ~24 ms (Galaxy S24, Xclipse 940) |
|
| 70 |
|
| 71 |
+
> **XNNPACK detect accuracy note:** the int8 detector is calibrated for sizes **≤ 800 px**
|
| 72 |
+
> (its accurate operating band). Larger inputs up to 1280 are accepted but **best-effort** —
|
| 73 |
+
> static-activation int8 is not stable at ≥ 960 px (this was equally true, though unmeasured,
|
| 74 |
+
> of the previous per-bucket builds). Prefer resizing pages to ≤ 800 on CPU; the Vulkan and
|
| 75 |
+
> CoreML detectors are accurate over their full advertised ranges.
|
| 76 |
|
| 77 |
+
## CoreML notes (iOS)
|
| 78 |
|
| 79 |
+
- The CoreML `.pte` is a **multifunction** Core ML model (`detect` + `recognize` share one
|
| 80 |
+
precompiled `.mlmodelc`). Requires **iOS 18+** and an ExecuTorch runtime ≥ 1.3.
|
| 81 |
+
- First-ever load on a device triggers a one-time per-shape ANE specialization (OS-cached
|
| 82 |
+
afterwards) — warm each model once after install.
|
| 83 |
|
| 84 |
## Compatibility
|
| 85 |
|
|
|
|
| 89 |
[ExecuTorch GitHub repository](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md).
|
| 90 |
If you work with React Native ExecuTorch, the library constants guarantee compatibility with the
|
| 91 |
runtime used behind the scenes.
|
|
|