Faster Communication Between Device and Cloud AI Models

Small AI models on devices currently communicate with much larger cloud models through text, adding hundreds of milliseconds per exchange and losing information as they convert internal numerical representations into words. The goal is a lightweight, trainable translator that sends those representations directly as a compact numerical message, retaining at least 90% of text-based task performance with much less delay and bandwidth. The models encode information differently, so the translator must bridge those differences within the device’s tight computing and memory limits. It must also adapt across tasks and model families without a separate, costly translator for every pair. Compressing messages too far can abruptly erase details the cloud model needs, especially spatial and visual information used in robotics.

Idea 01 of 02Borrowed from Cochlear implant (hearing prosthesis) engineering

Borrowing receiver-specific tuning and complementary sound coding from cochlear implants, we pair different compressors to send a device model’s internal state to an unchanged cloud model through a tiny connection without discarding the details it needs.

How it works

A small model on a device must send its internal state to a larger cloud model through as few as eight compact data tokens. Reasoning signals can be sparse; factual context can require much more detail, so one compression method can miss useful information. We borrow from cochlear implants, where engineers tune sound coding to the listener and preserve different aspects of sound with different methods. First, we probe the unchanged cloud model to measure which signals it can distinguish and use. We then choose among compressors that spread features across channels, select the strongest features, or preserve more detail in receiver-sensitive groups of features. A second, different compressor encodes what the first leaves out. We send the main encoding first and the correction when bandwidth permits. This targets the blind spots of a single compressor and preserves a usable base message when the connection shrinks.

Where have we seen this before?

Cochlear implant signal processingMED-EL FineHearingFineHearing encodes both the overall loudness pattern and fine timing details of sound. It varies electrical pulse intensity to convey the loudness pattern and pulse timing to convey details that envelope-only coding leaves out.Shared idea Complementary coding methods preserve different parts of the same signal.Envelope and Fine Structure: the Sounds of FineHearing (opens in a new tab)
Neural audio compressionGoogle SoundStreamEach quantizer stage encodes the error left by earlier stages. Training with some stages dropped lets one model support bitrates from 3 to 18 kbps, so the sender can omit refinement stages when bandwidth is limited.Shared idea Successive compression of the remaining error, with optional refinement layers.SoundStream: An End-to-End Neural Audio Codec (opens in a new tab)
Language-model interpretabilityROME causal tracingResearchers disrupted and restored internal activations to locate computations that affect factual predictions. They identified middle-layer feed-forward modules involved in factual recall and used those findings to guide targeted edits; this proposal borrows the probing step while leaving the receiver’s weights unchanged.Shared idea Probe a model’s internal responses before choosing where to intervene.Locating and Editing Factual Associations in GPT (opens in a new tab)

Idea 02 of 02Borrowed from Computational neuroscience

Borrowing the brain’s compress-then-expand relay to the cerebellum, we shape edge-to-cloud messages around the receiving AI’s internal processing to target a tenfold cut in transmitted data.

How it works

A small AI on a device can send its internal signals to a larger cloud AI, but bandwidth limits how much it can share. Matching those signals to the cloud model’s usual inputs does not ensure they remain useful after processing. We borrow a principle from models of the brain’s pontine relay: choose what to compress according to how the receiver expands it. First, we probe the cloud AI’s early attention layers, which mix information across a sequence. We then calculate which combinations of the device’s signals those layers can best use. Those combinations become the starting point for a small encoder, with more bits assigned to the highest-ranked channels. A cloud-side adapter feeds the compressed message into those same layers. We refine the encoder while keeping both AIs fixed, targeting roughly tenfold smaller messages with an encoder capped at 5% of the device model’s parameter count.

Where have we seen this before?

Computational neuroscienceMuscinelli, Wagner and Litwin-Kumar’s model of the pontine relayThe researchers modeled the narrow relay between the cerebral cortex and the cerebellum’s much larger granule-cell layer. Their theory shows how structured compression followed by expansion supports flexible computation, providing the rationale for designing compression around downstream processing.Shared idea Structured compression before a receiver expands the signal for computation.Optimal routing to cerebellum-like structures (opens in a new tab)
Language-model computation reuseCross-model attention-cache transfer, including Qwen3 14B to 32BResearchers used ridge regression to translate one model’s stored attention data for another model, so the receiver could reuse earlier processing of the prompt. Across four of six tested model pairs, the mapping retained 73–98% of the receiver’s accuracy when it processed the prompt itself.Shared idea A small, fitted mathematical mapping between fixed models’ internal representations.Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse (opens in a new tab)
Document image analysisPCA-initialized neural networks for page-layout analysisResearchers turned principal component analysis, which finds major patterns of variation in data, into encoder and decoder layers. On document-layout analysis, this initialization outperformed the random-weight initialization methods they tested.Shared idea Starting an encoder with calculated signal directions rather than random weights.PCA-Initialized Deep Neural Networks Applied To Document Image Analysis (opens in a new tab)

Faster Communication Between Device and Cloud AI Models

Two Compressors, One Tiny AI Message

How it works

Compress for the AI That Receives the Message

How it works