benito47 commited on
Commit
7f05070
·
1 Parent(s): 7d77e36

fixed the readmes

Browse files
Files changed (1) hide show
  1. README.md +46 -23
README.md CHANGED
@@ -12,9 +12,11 @@ If you'd like to run these models in your own ExecuTorch runtime, refer to the
12
  [official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
13
 
14
  Each language ships as **one fused `.pte`** (CRAFT *detect* + CRNN *recognize* in a single
15
- file) per backend, using **static bucketed methods** no dynamic-shape footguns. The `.pte`
16
- is a pure tensor→tensor function; all pre/post-processing (resize, normalize, box extraction,
17
- crop, CTC decode) is the client's job and is driven by `config.json`.
 
 
18
 
19
  ## Languages
20
 
@@ -27,35 +29,57 @@ crop, CTC decode) is the client's job and is driven by `config.json`.
27
 
28
  All languages share the same CRAFT detector and CRNN architecture — they differ **only** in
29
  the recognizer charset. The detector half of each fused PTE is identical across languages.
 
30
 
31
- ## Backends
32
 
33
- | backend | target | precision |
34
  |---|---|---|
35
- | `xnnpack` | CPU (Android/iOS) | int8 (CRAFT + CRNN) |
36
- | `coreml` | Apple ANE | weight-only int8 |
37
- | `vulkan` | Android GPU | CRAFT int8 (GPU) + CRNN int8 (XNNPACK, mixed-delegate) |
 
 
 
 
 
 
38
 
 
 
 
 
 
39
 
40
- ## Files
 
 
 
 
 
 
 
41
 
42
- ```
43
- config.json # shared base config (schema v1)
44
- charsets/easyocr_<lang>.charset.txt # per-language CTC charset (JSON array)
45
- <lang>/<backend>/easyocr_<lang>_<backend>_bucketed.pte
46
- ```
47
 
48
- - `config.json` is **shared** across all languages; it carries
49
- `charsetUrlPattern: "easyocr_<lang>.charset.txt"` so the client resolves the charset by
50
- language. Charset index `i` maps to logit `i + 1` (logit `0` is the CTC blank).
 
 
51
 
52
- ## Buckets
 
 
 
 
53
 
54
- Static per-size methods (`is_bucketed()` reports `[detect sides ; recognize widths]`):
55
 
56
- - **detect** (square sides): `800, 1280``detect_800`, `detect_1280` (+ a `1280×320` portrait method)
57
- - **recognize** (widths, height 64): `64, 128, 256, 512` → `recognize_64 … recognize_512`
58
- `detect` runs once per image; `recognize` runs once per text line.
 
59
 
60
  ## Compatibility
61
 
@@ -65,4 +89,3 @@ the compatibility note in the
65
  [ExecuTorch GitHub repository](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md).
66
  If you work with React Native ExecuTorch, the library constants guarantee compatibility with the
67
  runtime used behind the scenes.
68
-
 
12
  [official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
13
 
14
  Each language ships as **one fused `.pte`** (CRAFT *detect* + CRNN *recognize* in a single
15
+ file) per backend, with a single **dynamic** `detect` method and one fixed-width `recognize`
16
+ method (no per-size method buckets). The `.pte` is a pure tensor→tensor function; all
17
+ pre/post-processing (resize, normalize, box extraction, crop, CTC decode) is the client's job
18
+ and is driven by `config.json`. EasyOCR is the *fallback* pipeline —
19
+ [PP-OCRv6](https://huggingface.co/software-mansion/react-native-executorch-pp-ocrv6) is primary.
20
 
21
  ## Languages
22
 
 
29
 
30
  All languages share the same CRAFT detector and CRNN architecture — they differ **only** in
31
  the recognizer charset. The detector half of each fused PTE is identical across languages.
32
+ Charset index `i` maps to logit `i + 1` (logit `0` is the CTC blank).
33
 
34
+ ## Methods & I/O contract
35
 
36
+ | method | input | output |
37
  |---|---|---|
38
+ | `detect` (CRAFT) | `[1,3,H,W]` f32 RGB, **ImageNet-normalized by the client**: `(x/255 − mean)/std`, `mean=[0.485,0.456,0.406]`, `std=[0.229,0.224,0.225]` | score `[1,H/2,W/2,2]` (region + affinity, NHWC) and feature `[1,32,H/2,W/2]` |
39
+ | `recognize` (CRNN) | `[1,3,64,512]` f32 RGB, client-normalized `(x/255 − 0.5)/0.5` (RGB→gray conv is baked) | `[1,127,V]` probs (softmax baked) |
40
+
41
+ **Nothing is baked for input normalization** — the client normalizes before calling, with
42
+ *different* norms per method (ImageNet for detect, `0.5/0.5` for recognize).
43
+
44
+ ## Shape discovery (companion methods)
45
+
46
+ Every method carries exactly **one** no-arg discovery companion:
47
 
48
+ - `get_dynamic_dims_<method>` — dynamic method. One `int32 [rank, 3]` tensor per tensor
49
+ input; rows are `[min, max, step]` (static dims `[n, n, 1]`).
50
+ - `get_enum_shapes_<method>` — enumerated/fixed method. One `int32 [N, rank]` tensor per
51
+ tensor input; each row is a complete legal shape (cross-dimension coupling is exact —
52
+ `1280×320` listed does **not** imply `320×1280`). Snap inputs to the nearest row.
53
 
54
+ | backend | `detect` | `recognize` |
55
+ |---|---|---|
56
+ | `xnnpack` | `get_dynamic_dims_detect` → H, W ∈ `[320, 1280]` step 32 | `get_enum_shapes_recognize` → `[[1,3,64,512]]` |
57
+ | `vulkan` | `get_dynamic_dims_detect` → H ∈ `[800, 1280]`, W ∈ `[320, 1280]` step 32 | `get_enum_shapes_recognize` → `[[1,3,64,512]]` |
58
+ | `coreml` | `get_enum_shapes_detect` → `800²`, `1280²`, `1280×320` | `get_enum_shapes_recognize` → `[[1,3,64,512]]` |
59
+
60
+ `detect` runs once per image; `recognize` runs once per text line (the client snaps every
61
+ crop to width 512 — the CRNN's BiLSTM only delegates at a fixed time dimension).
62
 
63
+ ## Backends
 
 
 
 
64
 
65
+ | backend | target | detect | recognize | warm latency (detect @800² / recognize) |
66
+ |---|---|---|---|---|
67
+ | `xnnpack` | CPU | int8, dynamic (see note) | int8 @512 | ~810 ms / ~24 ms (Galaxy S24) |
68
+ | `coreml` | Apple ANE | weight-only int8, enumerated | weight-only int8 @512 | ~83 ms / ~27 ms (Apple M-series ANE) |
69
+ | `vulkan` | Android GPU | fp16, dynamic (resize) | int8 @512 on **XNNPACK** (mixed-delegate) | ~750 ms / ~24 ms (Galaxy S24, Xclipse 940) |
70
 
71
+ > **XNNPACK detect accuracy note:** the int8 detector is calibrated for sizes **≤ 800 px**
72
+ > (its accurate operating band). Larger inputs up to 1280 are accepted but **best-effort** —
73
+ > static-activation int8 is not stable at ≥ 960 px (this was equally true, though unmeasured,
74
+ > of the previous per-bucket builds). Prefer resizing pages to ≤ 800 on CPU; the Vulkan and
75
+ > CoreML detectors are accurate over their full advertised ranges.
76
 
77
+ ## CoreML notes (iOS)
78
 
79
+ - The CoreML `.pte` is a **multifunction** Core ML model (`detect` + `recognize` share one
80
+ precompiled `.mlmodelc`). Requires **iOS 18+** and an ExecuTorch runtime 1.3.
81
+ - First-ever load on a device triggers a one-time per-shape ANE specialization (OS-cached
82
+ afterwards) — warm each model once after install.
83
 
84
  ## Compatibility
85
 
 
89
  [ExecuTorch GitHub repository](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md).
90
  If you work with React Native ExecuTorch, the library constants guarantee compatibility with the
91
  runtime used behind the scenes.