ASIC-Inspired Small Language Model (<1B Parameters)
Topics
Most language model research focuses on scaling — more parameters, more compute, better performance. This project took the opposite direction: what if you designed a language model the way chip designers design ASICs — fixed-function, inference-optimised, minimum viable capability?
Motivation
ASICs (Application-Specific Integrated Circuits) are efficient because they do one thing extremely well, with no generalisation overhead. The hypothesis: a language model designed with a specific narrow task in mind — and aggressively optimised for inference at that task — could be practical on-device without a GPU.
Architecture Exploration
The research explored: - Reduced-head attention configurations trading quality for speed - Aggressive quantisation (INT8, INT4) from the training stage rather than post-training - Structured pruning to remove redundant attention heads and MLP neurons - Fixed vocabulary and tokenisation tuned for the target domain rather than general text
Findings
Aggressive quantisation and pruning combined can reduce model size by 60–80% with manageable quality loss for narrow tasks. The key insight: most of the quality loss in small models comes from trying to generalise — constraining the task definition early allows much deeper optimisation.
Status
Prototype and research notes. Not published. Ongoing exploration.
Tools
Python, PyTorch, HuggingFace, custom quantisation scripts.