BENCHMARKING SMALL LANGUAGE MODELS FOR OFFLINE CHILD-DIRECTED STORYTELLING ON EDGE DEVICES

Abstract

Cloud-dependent storytelling assistants are unusable in low-connectivity settings and transmit children's speech to third-party servers. Deploying a generative language model entirely on-device removes both problems, but it is not established which models are actually viable within the compute envelope of a single-board computer, nor whether the widely assumed monotone trade-off between generation speed and narrative quality holds at sub-billion parameter scale. This paper reports a controlled benchmark of six openly available causal language models Qwen2.5-0.5B, GPT-2 Medium, GPT-2 Large, DistilGPT-2, GPT-Neo-125M and DialoGPT-small for offline, child-directed story generation. Each model was evaluated under three fixed decoding regimes with an identical prompt and a common 100-token budget, yielding paired latency, output-length and lexical-diversity measurements, and each output was scored by three raters, blinded to model identity, on a five-criterion rubric. Reliability was high (ICC(2,k) = 0.966; Krippendorff's α = 0.903). Research report three findings. First, the speed–quality trade-off does not hold in this regime: the rank correlation between latency and human quality is not significant (ρ = 0.714, p = 0.111), and Qwen2.5-0.5B simultaneously attains the highest quality (Q = 4.87/5) and a latency 9.1× lower than GPT-2 Large (Q = 2.87), which is 3.1× larger. Training recency and instruction alignment, not parameter count or latency, dominate narrative quality at this scale. Second, Distinct-n the standard automatic diversity metric is strongly confounded with output length in this regime (ρ = −0.765, p < 0.001); degenerate 4- to 9-token outputs achieve a perfect Distinct-1 of 1.00, and research specify a length-gated reporting protocol that removes the artefact. Third, research document a negative hardware result: the Coral Edge TPU cannot host autoregressive transformer decoding, for architectural reasons research make explicit, and single-board CPU inference is presently the only workable substrate for this class of system. The research combine these into a Quality–Efficiency Index and a quality-gated selection rule, and report a Raspberry Pi 5 deployment envelope for a full speech-to-speech pipeline.

Index Terms: Edge artificial intelligence, small language models, offline inference, child–computer interaction, natural language generation, human evaluation, Raspberry Pi, story generation, evaluation metrics.