The Question Behind the Paper
Our team at SuiAudit Lab has published a research paper, Can Small Models Beat Frontier LLMs? Introducing SuiAudit Pure1: Specialized Auditing via Extreme Low-Resource Fine-Tuning, on TechRxiv (DOI: 10.36227/techrxiv.177204934.48017152/v1). I was a key contributor alongside Chia-Lin Chen and Fang-Yu Lien; the resulting product, SuiAudit, is live at suiaudit.wal.app.
The question in the title is the honest version of what we were testing. If a general-purpose frontier model already writes and reads code well, what is left for a 7B-parameter model specialized on one language? Our answer is that the gap frontier models leave on Sui Move is not a gap in capability. It is a gap in what they were trained to expect.
Solidity Bias Is a Data Problem, Not a Scale Problem
General-purpose LLMs auditing Sui Move are plagued by what we call “Solidity Bias”: they mistakenly apply Ethereum vulnerabilities to Sui’s object-centric architecture.
My reading is that this is a wrong prior rather than a shortfall of capability, and that framing decides the whole approach: the fix is data that contradicts the prior.
So the dataset is the center of the work. It blends positive vulnerability samples, categorized A-F, with adversarial “hard negative” examples: safe code patterns engineered to trip up broader LLMs. The hard negatives are the part that matters. Training only on real vulnerabilities teaches a model to find something; training against code that looks vulnerable under Solidity assumptions but is safe under Sui’s teaches it when to say nothing.
The Pipeline, and How the Numbers Were Obtained
Pure1 is a 7B-parameter model built through a streamlined fine-tuning pipeline: Supervised Fine-Tuning (SFT) for domain mastery, followed by Direct Preference Optimization (DPO) enhanced with Chain-of-Thought reasoning.
Evaluation is on our own Sui-specific benchmark, where Pure1 reached 31.4% accuracy, ahead of Claude Opus 4.5 (26.6%) and GPT-5.2 (17.2%) — a margin of +4.7% over the strongest generalist model.
Pure1 is built on Mistral-7B-Instruct-v0.2 and quantized to 4-bit NF4, which is what lets it fit on a single NVIDIA T4 (16GB) or RTX 3060 (12GB). It runs fully offline, preserving developer privacy while delivering real-time insights via VS Code and Chrome extensions. We have open-sourced the core infrastructure on GitHub and shared the weights on Kaggle.
Where It Has Been Used
Pure1 is deployed in production.
The work also took 2nd Place at the 2025 Sui Hackathon Mandarin and 3rd at the Sui Taipei Dev Hackathon.
What I Take Away
The result I care about is narrow and, I think, generalizable: on a task where the dominant failure is a transferred prior rather than missing capability, curating the negatives beat scaling the model. That is a cheap lever, and it is available to anyone working in a language the frontier labs have little data for.
If you’re building on Sui or interested in AI-driven security, the full paper and the live tool are both linked above.