D-MoE: Distributed Mixture of Experts for Edge LLMs

The label D-MoE is used for more than one idea in the literature. It can refer to a dynamic or deep mixture of experts, while distributed MoE work places routing and expert computation across devices. The distinction should be stated before making a systems claim.

This note uses D-MoE as a practical shorthand for a distributed mixture-of-experts deployment.

1. What is distributed

Suppose the router is hosted at node $r$ and expert $E_i$ at node $v_i$. A serial execution approximation includes network transfer:

$$T(h_t) = T_{r} + \sum_{i\in g_t}\left(T_{\text{link}}(r,v_i)+T_{E_i}\right) + T_{\text{merge}}.$$If the formula does not fit, focus it and use the left and right arrows, or scroll horizontally.

Sparse activation reduces expert computation per token, but a poor placement can turn communication into the dominant cost.

2. Edge constraints

A distributed deployment must account for:

• heterogeneous accelerators,
• link latency and intermittent availability,
• expert memory residency,
• privacy of token representations,
• failures during a routed request,
• load imbalance caused by correlated queries.

Routing should include availability and cost, not only model affinity.

3. Distributed does not automatically mean decentralized

An architecture can distribute experts across edge nodes while keeping one central router and one authority. That may be the right engineering choice. It is different from a peer-to-peer or decentralized system where coordination and membership are also distributed.

Making this distinction explicit avoids claiming a privacy or resilience property that the architecture does not provide.

4. Reliability and security controls

Use authenticated routing messages, bounded payloads, per-expert health signals and a fallback path when an expert is unavailable. Keep a record of which experts handled each request and how often a route was changed.

An adversary who can influence routing may create denial of service, extract information from a specialized expert or cause systematic quality drift. These threats belong in the evaluation plan from the start.

5. A measured deployment path

Begin with a centralized control plane and distributed expert workers. Measure latency, utilization and failure recovery. Then test whether decentralizing routing improves the target requirement enough to justify its coordination cost.

D-MoE is a systems problem before it is a slogan. The placement of the router, experts and evidence determines its real properties.

The critical path matters more than adding expert times

When experts execute in parallel, a layer's time depends on the slowest expert required for the combination. An approximation for that case is:

$$T_{layer}\approx T_{router}+\max_{i\in g_t} (T_{send,i}+T_{queue,i}+T_{expert,i}+T_{return,i})+T_{merge}.$$If the formula does not fit, focus it and use the left and right arrows, or scroll horizontally.

Adding times fits serial execution but overestimates ideal parallel execution. A maximum does not fully describe a shared network either: transfers may compete for the same link. State the assumption behind each estimate before comparing results.

WDMoE studies wireless expert distribution with base-station coordination and hardware evaluation. It is a specific reference for edge collaboration; distributing experts does not automatically decentralize the control plane.

A reproducible placement test

As a proposal, compare experts resident on one server, distributed over a local network and distributed under bandwidth constraints. Keep the checkpoint, numerical precision, queries and concurrency limit fixed. Separate weight preloading, cold starts and warm-cache operation.

Record bytes per token, queue time and deadline misses alongside tokens per second. If an expert stops responding, substitution is valid only if the model and engine support it and its effect has been evaluated. Arbitrarily choosing another expert changes the model's computation.

What crosses the network

Activations are not readable text, but should not be assumed anonymous. The threat model should include the remote expert operator, routing traces and error logs. Define retention limits and avoid mixing user caches.

The MoE foundations guide explains routing; golden-set evaluation helps check that infrastructure improvements do not conceal answer regressions.