Engineering • HarmBlock

The Engineering Behind HarmBlock’s On-Device Safeguarding AI

Building private, real-time protection for children’s digital worlds

On-device visual AI    <40MB model footprint    Real-time classification

Children’s digital lives do not happen in one place. Harmful sexual content can appear in a browser, a messaging app, a camera preview, a downloaded file, a social media feed, a screenshot, a video or a livestream. It is not confined to a single platform, app or predictable pathway. It can appear anywhere a child can view, capture, receive, create, save or share visual content.

That creates a seismic – and fundamentally different - safeguarding challenge. 

Traditional online safety systems often rely on content being uploaded, reported or processed remotely. That may support platform moderation, but on-device safeguarding demands something different: protection that can act before, during or immediately after exposure, without first sending a child's private images or videos elsewhere for analysis. 

A child may move through thousands of images, videos, messages and screen interactions every day. Most are harmless. Some are ambiguous. A small number may be seriously harmful. Effective on-device safeguarding must therefore operate across millions of fleeting moments, often in noisy, distorted or cluttered environments where harmful content is visible only briefly.

That created the central engineering question: could a visual AI model make reliable safeguarding decisions directly on a child’s device, quickly enough for real-time intervention, while preserving privacy, dignity and everyday usability?

Answering it meant solving several problems at once. The model had to recognise harmful sexual content, including CSAM-relevant material, across screens, cameras and stored media. It had to work within strict limits on memory, processing power, latency, battery use and thermal performance. It had to remain reliable despite motion blur, compression, screen glare, moiré effects and other real-world distortions. It had to minimise unnecessary disruption without missing genuinely harmful content. 

And it had to do all of this without turning a child’s device into a surveillance system or compromising their privacy and dignity. 

That principle shaped every engineering decision we made. Effective child safeguarding should not mean constant watching, shaming, profiling or unnecessary restriction. It should protect children’s dignity as well as their online safety: supporting safe exploration, learning and independence, intervening when needed, and otherwise staying quietly out of the way of normal life. 

HarmBlock’s development therefore became a process of classification under constraint: preserving safeguarding performance while reducing latency, model size, computational load and benign disruption. 

Those engineering breakthroughs also became the foundation of SafeToNet's family of on-device safeguarding patents. Safeguarding came first; protecting the underlying innovation ensures those advances can continue to evolve, reach more devices, and create a lasting legacy of privacy-preserving protection that cannot easily be undermined. 

Architecture

Designing safeguarding intelligence for the device

Many high-capacity computer-vision systems are developed for cloud environments, where compute and memory are comparatively abundant. HarmBlock was designed around the opposite assumption: it had to run on ordinary consumer devices, including older and lower-end phones.

That constraint shaped the architecture from the beginning. Every parameter, layer and memory operation adds cost through latency, power consumption and thermal load. But compress too aggressively and the model can lose the nuance needed to distinguish harmful sexual content from visually similar benign material. 

That distinction is critical. The safeguarding task could not be solved by assessing levels of skin exposure alone. HarmBlock was trained to interpret combinations of visual features and context, rather than treating exposed skin as a proxy for harm. 

HarmBlock was therefore engineered around a lightweight visual backbone with a Mixture-of-Experts architecture, allowing specialist internal pathways to respond to different visual patterns and ambiguity zones without applying the same computational burden to every input. Internally, the model maintains a Top-K multi-label representation spanning more than 650 subcategory labels, enabling richer reasoning across visual features and contextual clues before emitting a focused safeguarding classification directly on the device.  

That capability was developed through a structured, multi-stage training programme using more than 25 million appropriately sourced images, including large-scale visual pre-training, supervised fine-tuning and curated refinement data. This training programme used appropriate, lawfully sourced and licensed datasets, with structured labelling and controlled handling throughout development.

The harder engineering challenge was preserving that representational capacity while making the model genuinely deployable. SafeToNet’s AI engineers worked through successive optimisation cycles combining model compression, architectural refinement, hardware-aware optimisation, runtime tuning and mobile acceleration pathways. Across those iterations, the model footprint was reduced from approximately 80MB to under 40MB, improving inference efficiency and deployability while maintaining strong benchmark performance. 

On an internally curated mobile-safety benchmark, the current mobile-ready baseline achieved:
97.78%
Overall accuracy
95.45%
F1 score
92.78%
True positive rate
0.54%
False positive rate
~92ms
Average model-only inference in lower-end CPU-based conditions
~185ms
Wider end-to-end pathway in the same conditions
~1ms
Individual model decisions in highly optimised, accelerator-delegated environments
10ms-class
End-to-end inference envelope demonstrated in other optimised configurations
Inference performance varies by hardware and runtime.

These figures represent different execution conditions, not a single universal speed. Model-only latency is not the same as the complete capture-to-decision pathway. The achievement is the combination: a high-performing safeguarding model compressed into a mobile-ready footprint and adapted across markedly different hardware environments without losing the core safeguarding task. 

Robustness

Making the model work outside the laboratory

A strong benchmark result does not guarantee that a model will behave reliably on a child’s device. 

Real inputs may be blurred, cropped, compressed, partially visible, surrounded by interface elements, or captured from another screen. Camera-based environments add further distortion through glare, flicker, reflections, rolling-shutter artefacts and the moiré effect created when one digital display is recorded by another imaging sensor.

Throughout early development, internal testing exposed the gap between clean evaluation images and lived device conditions. Some failure cases were predictable: low resolution, unusual lighting, partial occlusion or visually ambiguous benign content. Others were more surprising - and occasionally, funny. Pigs, hairless animals, skin-like textures and partial objects sometimes activated visual features associated with explicit content. 

Those cases were not dismissed as curiosities. They revealed how the model was interpreting colour, texture, shape and composition, and each unexpected result became a diagnostic signal rather than an isolated mistake. 

Our ML and evaluation teams logged and grouped failure patterns, traced them through the model and inference pipeline, and fed the findings into structured update cycles. Model-related failures drove targeted data enrichment, relabelling, class rebalancing, focused augmentation, supervised fine-tuning and broader retraining. Runtime failures were passed into a separate pipeline-optimisation loop. 

That feedback loop became central to HarmBlock’s development. Licensed and appropriate training data was complemented by internally curated mobile evaluation sets that reproduced the surfaces and distortions the model is expected to encounter. These controlled test sets guided augmentation, error analysis and comparative evaluation across successive model versions. 

Training and refinement incorporated transformations such as compression, motion blur, partial occlusion, variable lighting, interface overlays, grid layouts, screen recapture and controlled moiré-like distortion. The same refinement process also included adversarial and transformation-based testing to examine whether cropping, overlays, recolouring, compression, occlusion or screen recapture could systematically weaken the classification signal. 

The aim was not to teach the model to memorise perfect examples. It was to help it recognise stable visual features when the image is noisy, incomplete or presented in an unfamiliar form.

The model improved not by avoiding failure, but by turning each failure into the next engineering decision - and feeding that decision into the next measured development cycle.
Pipeline

Engineering an efficient capture-to-decision pipeline

On-device safeguarding is not only a model problem. It is also an inference-pipeline problem. 

The system had to select and prepare visual inputs, run inference efficiently and interpret outputs that fluctuated as cameras moved, lighting changed or users scrolled. That required careful coordination across preprocessing, inference and post-processing.

HarmBlock’s preprocessing pipeline applied frame selection, resizing, normalisation and surface-specific preparation so the model received a consistent input without losing relevant visual information. 

Post-processing addressed a different challenge: a single noisy frame should not become a disruptive user experience. Our engineers therefore incorporated confidence calibration, repeated-frame confirmation, temporal smoothing, moving averages and short-window decision stabilisation. 

Similarity-aware frame handling also reduced unnecessary computation. Where consecutive frames had not changed materially, the pipeline avoided repeating equivalent inference work; where meaningful visual change occurred, analysis was prioritised. This preserved battery life and processing headroom while maintaining responsiveness. 

The model’s role was to produce a reliable local classification signal. The surrounding device or application layer then determined the appropriate safeguarding response according to the deployment surface, policy and risk context. Keeping those responsibilities separate allowed the model stream to remain compact and efficient while supporting different product behaviours. 

The Principle Was Simple

Analyse only as often as necessary, stabilise noisy inputs, and respond quickly when the risk signal is meaningful.

Evaluation

Measuring performance, fairness, and trust

HarmBlock was not evaluated through a single headline score.

Scientific evaluation measured model discrimination using precision, recall, F1, true positive rate and false positive rate. 

Deployment evaluation verified that the exported model preserved expected behaviour across preprocessing, post-processing, SDK integration, runtime execution, latency and stability. 

User-acceptance and UX-directed evaluation examined how the integrated safeguard behaved in realistic journeys: how often false positives occurred, how severe they were, whether they were reproducible, whether decisions were delayed, and whether errors clustered around particular surfaces or conditions. 

This matters because a high benchmark score or narrow test methodology does not automatically prove that a safeguard will behave sensibly in everyday life. 

Fairness was evaluated through the same disciplined process. Dataset construction and evaluation used stratified sampling across relevant attributes such as age, gender and skin tone, with skin-tone assessment informed by the Fitzpatrick scale. Performance was reviewed through demographic and contextual slices to identify uneven error rates. 

During training, techniques including class balancing and focal-loss re-weighting helped reduce domination by easy or overrepresented examples and increased learning pressure on harder, underrepresented and ambiguity-prone cases.

The objective was not to claim perfect performance or the elimination of bias. It was to identify uneven behaviour, understand its source and reduce it through deliberate data design, targeted enrichment and continuing evaluation. This required close collaboration between our AI engineers, evaluation teams and safeguarding experts, combining quantitative performance analysis with informed review of how errors might affect children in practice.
Privacy & Security

Privacy and security through engineering

Privacy is not a promise added after the technology was built. It is one of the reasons HarmBlock was engineered to run on device in the first place. 

Visual analysis happens locally, and the model returns a narrowly scoped safeguarding classification. Images and videos do not need to be sent to the cloud for inference. HarmBlock does not need to identify the child, construct a behavioural profile or create a record of their habits. 

Deployment does not require images, embeddings, raw scores or intermediate representations to leave the device or persist after inference. HarmBlock also does not depend on distributing image-hash databases to the handset. Its purpose is deliberately narrow: analyse relevant visual content locally, return a safeguarding signal, and discard what is no longer needed. 

Running the model on device also creates a distinct security challenge: the model itself must be protected. HarmBlock’s deployment architecture therefore incorporates controls designed to reduce unauthorised extraction, modification and misuse, while limiting unnecessary exposure of confidence scores and intermediate outputs that could help an attacker reverse-engineer the decision boundary or optimise evasion. 

The model is not an image repository. Its weights encode learned statistical relationships used for classification, rather than a directly retrievable library of source images. Risks such as model extraction, inversion, tampering and adversarial misuse are addressed through threat modelling, controlled outputs, deployment safeguards and ongoing security testing. 

This combination matters. HarmBlock was not designed to make privacy dependent on trust in a remote service, or on promises about what might happen to uploaded content later. The architecture reduces the need for that transfer in the first place. 

Put simply: a child’s image does not need to become a cloud event for the device to make a safety decision.
Governance

Expert-governed CSAM evaluation

CSAM requires a separate level of governance because it depicts the abuse of real children, is illegal to possess or access outside tightly controlled and authorised circumstances, and cannot be handled as ordinary machine-learning data. 

This created a fundamental development challenge: how do you train and evaluate a model intended to recognise illegal content when the engineering team cannot lawfully access, inspect or move that material through ordinary development environments? 

SafeToNet therefore worked with the Internet Watch Foundation through controlled, restricted-access evaluation processes using dedicated infrastructure, air-gapped workflows, structured output logging and expert review by authorised professionals. 

Our engineering team adapted its development process to those safeguards: model software was introduced through controlled routes, results were reviewed without moving abusive and illegal imagery, and technical findings were fed back into development without requiring developer access to restricted material. 

This collaboration also provided the independent, expert-led evidence needed to test whether HarmBlock recognised CSAM in practice, without SafeToNet having to possess or reproduce the underlying material as proof. 

Within a controlled IWF evaluation, threshold analysis showed that the full sample was captured within the broader harmful-content category at the most protective operating point assessed. Approximately 80% was subsequently identified through secondary processing as distinct CSAM classifications. In the corresponding neutral evaluation data, that operating approach did not produce an observed increase in false positives. 

These findings also highlight an important distinction. HarmBlock is a visual AI classifier rather than a hash-matching system. Instead of relying on previously identified image fingerprints, it analyses visual characteristics to recognise harmful sexual content, enabling it to identify visual indicators within previously unseen, newly created or modified CSAM that would not be detectable through hash databases alone. As a discriminative vision model, its purpose is to classify existing content - not to generate it - reflecting SafeToNet's commitment to ensuring our safeguarding technology cannot itself create abusive imagery. 

The significance lies not only in the result, but in the engineering method behind it: demonstrating robust CSAM detection through independent, expert-governed evaluation of illegal material, without moving abusive content into ordinary engineering workflows, exposing developers to restricted imagery, or compromising the legal and ethical safeguards on which the development process depended.
Summary

Engineered for child-centred safeguarding

HarmBlock did not mature through one breakthrough. It matured through repeated testing, unexpected failure cases and successive engineering decisions that made the model smaller, faster and more reliable without losing sight of the safeguarding task. That process is continuing: each new model cycle is used to pursue greater accuracy, speed, efficiency and resilience across a wider range of devices and operating conditions. 

It is not simply an explicit-content classifier placed on a phone. It is a compact, embedded visual safeguarding capability engineered through careful trade-offs between model capacity, accuracy, latency, memory, robustness, fairness, battery use and privacy.

The technical achievement is the combination: high-performing visual classification compressed into a mobile-ready footprint, stabilised for noisy real-world inputs, accelerated across different hardware pathways, evaluated beyond benchmark accuracy, protected against extraction and misuse, and designed to make private safeguarding decisions directly on the device.

The ambition is to extend that capability as widely as practical – across newer and older devices and, over time, into other connected environments such as wearable and smart televisions. 

That is HarmBlock: on-device AI safeguarding built for the real digital world children live in.

Talk to us about HarmBlock

Learn how on-device safeguarding can work within your platform, device or service.

Get in touch Explore HarmBlock

July 15th, 2026 • By The SafeToNet Team