Laya Vision

# Laya Vision

Image inputs for [Laya](https://github.com/NandhaKishorM/laya): typed, calibrated decisions (`choice`, `score`, `noul`) over an **image plus optional text**, in one forward pass with no text generation.

Laya Vision swaps Laya's ModernBERT text encoder for [SmolVLM-256M-Instruct](https://huggingface.co/HuggingFaceTB/SmolVLM-256M-Instruct), which already understands images. It keeps Laya's `predict(state, questions)` API, output schema, proper-scoring-rule training and temperature calibration.

- **Model:** [thaitea/laya-vision-smolvlm-256m](https://huggingface.co/thaitea/laya-vision-smolvlm-256m)
- **Try it in the browser:** [thaitea/laya-vision-demo](https://huggingface.co/spaces/thaitea/laya-vision-demo), a Hugging Face Space on free CPU at about 3 s per image. Its source is in `space/`.
- **Status:** experimental research fork. It is not affiliated with Convai Innovations, the authors of Laya.

## Results

This is the fine-tuned checkpoint `all3-3ep/best`: 3 passes over 72k training examples, about 33 minutes on one A100. Scores are on the full validation splits.

| Dataset | Type | Chance | Accuracy | ECE (raw → calibrated) |
|---|---|---|---|---|
| A-OKVQA | 4-way `choice` | 25% | 61.8% | 0.295 → 0.123 |
| ScienceQA (image subset) | 2–5-way `choice` | ~36% | 86.6% | 0.090 → 0.034 |
| VQAv2 yes/no (re-split of official val) | `noul` | 50% | 73.4% | 0.102 → 0.041 |
| **All** | | | **75.2%** | 0.124 → **0.034** |

- **Latency:** about 71 ms for one image question on an NVIDIA L4 (bf16). The image is encoded once and reused for every question in the call.
- **Option-order sensitivity:** across 4 rotations of the A-OKVQA option order, accuracy varies by 0.7 points.
- **`score` questions are not trained yet.** There was no ordinal image data, so treat `sco