site + contract
Documentation, evaluation JSON, 4 MB request guard, typed validation, authentication forwarding, and a 25 second upstream timeout.
●Architecture // one image, one shared state
One visual prefix.
Many typed decisions.
A deployment pattern faithful to the verified public model: Vercel owns the contract and control plane; a separate CUDA service owns inference.
01 / INPUTimageHTTPS URL or base64
02 / CONTEXTcaller metadataoptional OCR + layout
03 / ENCODELaya Visionshared visual prefix
04 / FAN OUTtyped questionsbatch over one image
05 / RETURNprobabilitiesyes/no · choice · score
OCR and layout metadata are optional caller context, not claimed internal model stages. This describes the public API and deployment boundary—not undisclosed model internals.
●Deployment split
Control plane here.
CUDA elsewhere.
The frontend stays fast and globally available without pretending a serverless function is a GPU runtime.
validated proxy
LAYA_VISION_BACKEND_URL and an optional bearer token connect the two planes. Without them, the gateway returns a truthful 503.
model service
A containerized Python service loads the pinned checkpoint, groups questions by image, and returns the documented typed result schema.
●Why grouping matters
18 questions cost 1.37× one.
On the verified RTX 3090 run, a single question took 112.9 ms median. Eighteen questions sharing an image took 154.7 ms median, delivering 112.9 questions per second.
| Path | Questions | Median latency | Effective throughput |
|---|---|---|---|
| Single call | 1 | 112.9 ms | — |
| Shared-prefix group | 18 | 154.7 ms | 112.9 q/s |