cargo install typed-pictures-cli</code></pre>Using the CLI, you can generate an SVG from a hexpr</a>--our textual syntax for open hypergraphs. For example, this expression describes a one-bit half-adder whose outputs are the sum and carry bits: typed-pictures circuit/circuit.hex \ --padding 0.25 \ --no-wire-labels \ --generators circuit/circuit-generators.json \ '([x y . x y x y] {xor and})'</code></pre> This produces the image below: </div> Everything needed to reproduce it is included with this post: the circuit definitions</a>, the generator manifest</a>, and custom SVGs for gates such as XOR</a> and AND</a>. I generated the gate drawings with Codex; typed-pictures</code> handles their layout and wiring. The generation script</a> rebuilds every diagram in the post. Drawing Open Hypergraphs</h1> Open hypergraphs</a> provide a datastructure for "circuit-like syntax". If you can think of your syntax as "boxes with ports connected by wires", open hypergraphs are for you! Electronic circuits are a natural example. Here is a NOR</code> gate, generated from its open hypergraph representation: </div> In the underlying hypergraph, wires are nodes and boxes are hyperedges. This reverses the usual graph-drawing convention of treating boxes as nodes, but lets each box retain an ordered list of input and output ports. Now, to generate these diagrams, we need some software. Previously, I used dot</a>, but it isn't designed for hypergraphs and requires a lot of hackery. I wanted something that: Natively supports boxes with multiple ordered inputs and outputs</li> Can be called as a library, rather than as an external tool</li> Supports custom box drawings (like the gates in our diagram above)</li> </ol> The third point is especially useful: you can draw the individual boxes however you like--even ask Codex to generate them--then let typed-pictures</code> handle the layout and wiring. So how does it work? The NOR</code> diagram can be drawn using layered graph drawing</a>: reachability induces a partial order on the operations, and incomparable operations can be grouped into the same layer. Here, the first layer is {xor, and, 1}</code>, followed by {xor, id}</code>, and finally {xor}</code>. This direct layering step does not apply when the input hypergraph contains a cycle, as in this latch circuit: </div> So how do we deal with cycles? It turns out that we can replace any cyclic open hypergraph with an equivalent acyclic one using the Frobenius decomposition</a>. Intuition</h1> Here's a sketch before going into detail. Say we have a NOT gate with its output connected to its input: </div> This is represented as a hypergraph with: One edge (labeled not</code>)</li> One node (the single wire)</li> </ul> Reachability no longer defines a partial order: the NOT gate can reach itself. Instead, we first replace each node (wire) with a pair of edges (boxes): A left spider, representing all its occurrences as an input</li> A right spider, representing all its occurrences as an output</li> </ol> Here's what that looks like for our "cyclic NOT" diagram: </div> Now there are no directed cycles: A node (a wire) can only reach another node via an edge (a box), but we've replaced all the cyclic paths with ones that "terminate" at a cup</code>. This may seem arbitrary, but it has a nice theoretical interpretation: we're computing a Frobenius decomposition. Fusing the explicit spiders recovers the original hypergraph, and such a decomposition always exists. Let's see how this works! Open Hypergraphs</h1> Let's first talk datastructures. An open hypergraph is a hypergraph with a designated global interface (that's the "open" part). The hypergraph has nodes and edges, where each edge connects an ordered list of source nodes to an ordered list of target nodes. Intuitively, edges are operations (boxes), while nodes are wires. Recall the cyclic NOT diagram from before: </div> The corresponding open hypergraph contains the following data, shown here as Python-like pseudocode: nodes = [ (), # Node 0: the circuit's single wire ] edges = [ { "label": "not", "sources": [0], # The NOT gate reads from node 0... "targets": [0], # ...and writes back to it, forming the cycle }, ] # The circuit has no dangling input or output wires. global_sources = [] global_targets = []</code></pre> Nodes can appear any number of times in operation sources and targets, and in the global source and target. Mathematically, this kind of open hypergraph corresponds to a morphism of the free symmetric monoidal category presented by \(\Sigma + \mathbf{Frob}\). For more precision, see the Addendum below. For now, the important part is \(+ \mathbf{Frob}\): it is as if we had added four special operations to our hypergraph: </div> I say "as if", because these operations (and some associated equations) are not represented as explicit edges, but instead are implicit in the connectivity of the hypergraph. That's the key trick: instead of keeping the Frobenius operations implicit, we make them explicit operations. Making every wire's Frobenius structure explicit in this way gives us a monogamous acyclic open hypergraph. This restriction was introduced in Rewriting modulo symmetric monoidal structure</a>, and open hypergraphs of this kind represent morphisms in theories without implicit Frobenius operations. For example, a combinational circuit is acyclic, but it is monogamous only when operations such as fan-out and discard are made explicit. Monogamous acyclic open hypergraphs represent diagrams that are: Acyclic: there are no directed cycles.</li> Monogamous: every node appears... exactly once as a source (or global target), and</li> exactly once as a target (or global source).</li> </ul> </li> </ol> The monogamy condition more intuitively says that we can think of the wires in our diagrams as always connecting exactly two "ports". The NOR</code> diagram is acyclic but not monogamous: each input wire fans out to both xor</code> and and</code>. The cyclic NOT diagram is neither. Mathematically, a monogamous acyclic open hypergraph corresponds to a morphism of the free symmetric monoidal category presented by \(\Sigma\)--notice the lack of \(\mathbf{Frob}\) here. Now, the cool part is this: given an unrestricted hypergraph, a full Frobenius decomposition gives us an equivalent monogamous acyclic one. Frobenius Decompositions</h1> We've already seen the intuition for how this works above; now we'll be a little more precise. For a very precise treatment, see Data-Parallel Algorithms for String Diagrams</a>. First, we extend our set of operations with an explicit Spider</code> operation. This is really a family of operations--one for each pair of input and output arities--where each spider stands for a connected diagram built from the four Frobenius generators above. Then, for each node (wire) \(x\) in the original hypergraph, we introduce two spiders: A left spider \(s_x^-\), whose inputs correspond to occurrences of \(x\) on the global source boundary, and whose outputs correspond to occurrences of \(x\) as an operation input.</li> A right spider \(s_x^+\), whose inputs correspond to occurrences of \(x\) as an operation output, and whose outputs correspond to occurrences of \(x\) on the global target boundary.</li> </ol> There is one extra port on each spider, used to connect \(s_x^-\) directly to \(s_x^+\). More precisely, if \(x\) occurs \(p\) times on the global source boundary, \(m\) times as an operation input, \(n\) times as an operation output, and \(q\) times on the global target boundary, then: \[ s_x^- : p \longrightarrow 1 + m \qquad\text{and}\qquad s_x^+ : 1 + n \longrightarrow q. \] Every other occurrence is moved onto its own fresh wire, carrying the same label as \(x\). This makes the result monogamous: each fresh wire connects exactly one spider port to one operation port or global boundary. It also makes the result acyclic. All original operations lie between the left and right spiders, so every directed path moves from left spiders, through at most one original operation, to right spiders. Finally, the construction is reversible modulo the Frobenius equations. Fusing each explicit spider and identifying its incident wires recovers the original node \(x\), and doing this for every node recovers the original open hypergraph. Now let's look at a small acyclic example: the composition of f</code> followed by g</code>. </div> Here's its Frobenius decomposition, computed as described above: </div> Unfortunately, this is incredibly ugly. Notice that by decomposing in this way, we always get a three-layer diagram: source spiders, operations, and target spiders. Even this small acyclic example becomes difficult to read. What can we do about this? Cycle Breaking</h1> To fix this, we replace wires with explicit left and right spiders only where necessary. Which wires need to be replaced? Precisely those which cause cycles! Finding this set is the process of cycle breaking, closely related to the cycle-removal stage of layered graph drawing algorithms such as dot</a>. Finding a minimum set of wires that intersects every directed cycle is NP-hard, so we use a fast greedy heuristic. The one used in typed-pictures</a> works as follows: Turn the hypergraph into a directed bipartite graph with one vertex for each wire and one for each operation. An operation input becomes an arc wire → operation</code>, and an operation output becomes an arc operation → wire</code>.</li> Repeatedly remove every source and sink--that is, every vertex with no incoming or no outgoing arcs. Such a vertex cannot be part of a directed cycle, so removing it does not require us to spiderize anything. We call this step peeling the acyclic fringe.</li> If vertices remain after peeling, the residual graph contains a cycle. Choose the remaining wire with the largest indegree × outdegree</code> score, record it as a cycle breaker, and remove it. This score favours wires whose removal is likely to break several cycles at once.</li> Peel again, and repeat until no vertices remain.</li> </ol> The selected wires are not necessarily the smallest possible set, but they are guaranteed to intersect every directed cycle: anything not selected was eventually removed as part of an acyclic fringe. Spiderizing exactly these wires therefore makes the diagram acyclic while leaving the rest of its wiring unchanged. For comparison, here is the latch after a full Frobenius decomposition, with every wire replaced by explicit spiders: </div> For this latch, cycle breaking needs only one wire. Spiderizing that wire yields an acyclic diagram, shown here: </div> Both diagrams represent the same open hypergraph modulo the Frobenius equations, but cycle-broken diagrams are typically easier to read. Summing up</h1> To sum up: Frobenius decompositions let us lift an algorithm for rendering acyclic open hypergraphs into one that works on any open hypergraph. In practice, we only add spiders where we need them to break cycles. That means simple diagrams stay simple, while cyclic ones still get an equivalent acyclic representation using the usual Frobenius structure. Addendum: \(\Sigma\)-terms, Open Hypergraphs, and Monogamous Acyclicity</h1> For the categorically minded, here is the categorical picture. Let \(\Sigma = (\Sigma_0, \Sigma_1)\) be a monoidal signature consisting of generating objects and typed operations. Write \(\mathrm{Free}_\Sigma\) for its category of terms, built using composition and tensor product and taken modulo the symmetric monoidal laws. If the signature also carries equations \(\Sigma_2\), quotient every category below by the congruence they generate. Let \(\mathrm{Hyp}_\Sigma\) be the category of \(\Sigma\)-labelled open hypergraphs, and let \(\mathrm{MAHyp}_\Sigma\) be its monogamous acyclic subcategory. The standard correspondences, together with the optional factorisation on the right, fit into this commuting diagram: </div> In symbols, \(\mathrm{Free}_\Sigma \cong \mathrm{MAHyp}_\Sigma\), and \(\mathrm{Free}_{\Sigma+\mathbf{Frob}} \cong \mathrm{Hyp}_\Sigma\). On the right, the Frobenius equations identify each connected region of Frobenius wiring with a single hypergraph node. The right-hand equivalence may be factored through \(\mathrm{MAHyp}_{\Sigma+\mathbf{Frob}}/{\equiv_{\mathbf{Frob}}}\): monogamous acyclic hypergraphs with explicit Frobenius generators, taken modulo the Frobenius equations. The specific full decomposition in this post is guaranteed to be monogamous and acyclic; an arbitrary way of making spiders explicit need not have this property. Expanding its spiders gives a \((\Sigma+\mathbf{Frob})\)-term, while fusing them recovers the original open hypergraph. Such a representative is not unique, but all valid choices agree modulo the Frobenius equations. The cycle-breaking optimisation is deliberately weaker: it makes only the spiders needed to remove directed cycles explicit. The result is guaranteed to be acyclic, but it need not be monogamous because unselected wires may still branch or merge implicitly.

thunderbolt-ibverbs: We have InfiniBand at home

hellas.ai — Thu, 28 May 2026 00:00:00 +0000

I spent the past few weeks building a linux kernel module that makes ordinary USB4/Thunderbolt ports on AMD mini PCs pretend to be InfiniBand devices. The goal is simple: let existing AI runtimes like vLLM/RCCL split inference or training across multiple boxes at home, without buying enterprise networking gear.

TL;DR. We built experimental RDMA-over-USB4 for 128GB Strix Halo mini PCs. It lets two consumer boxes talk fast enough to run tensor-parallel inference and FSDP workloads across both machines: ~95 Gb/s bidirectional raw RDMA, ~7 µs one-way latency, a MiniMax-M2.7 TP=2 inference run that does not fit on one box, and a Gemma 3 27B LoRA FSDP step falling from 1359 s over Ethernet to 126 s over 4-HCA USB4 RDMA.

~48 Gb/s per direction (~95 Gb/s bidi total) sustained ib_write_bw</code>, 4-HCA aggregate at 1 MiB / 8 QPs with IOMMU off — vs ~2.3 Gb/s</code> over the onboard 2.5 GbE and ~9 Gb/s</code> for soft-RoCE on top of thunderbolt-net</code> at the per-rail level.</li>
~7 µs one-way ib_write_lat</code> at 64 B, single QP — vs ~28 µs</code> over RXE/2.5 GbE and ~65 µs</code> over RXE/TBnet.</li> </ul> DISCLAIMER: this is research code, most of it AI-generated, and it loads experimental kernel modules on machines I was willing to crash repeatedly. I made an effort to understand enough of it to keep it on-track, but there are almost certainly false assumptions and sharp edges throughout. No warranty, no support promise, not production software.

Inductive Types in Lean

hellas.ai — Thu, 02 Apr 2026 00:00:00 +0000

Inductive types in Lean allow for conservative extensions of the core theory (the calculus of inductive constructions) by adding new freely generated types. Here "conservative" is a term of art: it means that our additions do not "increase the power" of the underlying theory by adding new axioms; instead, they extend the language in a way designed to preserve consistency.

But how does this work internally? What exactly is added to the logical theory when we write inductive ...</code>? This post answers that question using natural numbers as the running example. Suppose we declare a Nat</code> type as follows:

namespace</span> MyNat</span></span>
</span>
  inductive</span> Nat</span> : </span>Type</span></span>
  | zero : Nat</span></span>
  | succ : Nat → Nat</span></span>
</span>
end</span> MyNat</span></span></code></pre>
When we do this, Lean adds constructors, or "introduction rules":</p>
Nat.zero : Nat</span></span>
Nat.succ : Nat → Nat</span></span></code></pre>
These let us construct</em> (or 'introduce') Nat</code> values.
For example, the value for 1</code> is introduced as Nat.succ Nat.zero : Nat</code>.</p>
But we also need to deconstruct</em> or ('eliminate') a Nat</code> value.
Lean adds a "recursor" Nat.rec</code> for this, whose type can be printed using
#check Nat.rec</code>:</p>
MyNat.Nat.rec.{u}</span></span>
    {motive : Nat → Sort u}</span></span>
    (zero : motive Nat.zero)</span></span>
    (succ : (a : Nat) → motive a → motive a.succ)</span></span>
    (t : Nat) : motive t</span></span></code></pre>
Spelling this out, we have:</p>

A dependent function motive</code> assigning to each Nat</code> value n</code> some type denoted motive n</code>.</li>
A base case - a value</em> zero</code> of type motive Nat.zero</code></li>
An inductive case - a function</em> succ</code> mapping a value</em> of type motive a</code> to a value motive (Nat.succ a)</code></li>
</ol>
With these three arguments applied, we're left with a function of type
(t : Nat) → motive t</code>.
This is a dependent function type</em> mapping each nat value to a result whose
type depends on the value</em>.</p>
A motivating proof-shaped use is ∀ n, n = n</code>.
In Lean, proving this means constructing a term of that type.
Nat.rec</code> is exactly the induction/recursion principle that lets us build such
terms by giving:</p>

a value at zero</code>, and</li>
a way to extend a value at a</code> to one at succ a</code>.</li>
</ol>
Together, the constructors (introduction rules) and recursor (elimination rule)
form the core logical/computational content of an inductive declaration.
However, Lean also automatically derives a couple useful utilities.</p>
Helpers: casesOn</code> and noConfusion</code></h2>
In addition to rec</code>, lean adds a special case: Nat.casesOn</code>, which lets us do
a 'shallow match' of cases, without recursing.</p>
-- casesOn</span></span>
#check</span> Nat.casesOn</span></span>
-- MyNat.Nat.casesOn.{u} {motive : Nat → </span>Sort</span> u} (t : Nat) (zero : motive Nat.zero) (succ : (a : Nat) → motive a.succ) :</span></span>
  motive t</span></span></code></pre>
As a quick aside, note that if you try to add a case to Nat with the same name
as one of these automatically added functions, you will get an error! E.g., adding</p>
inductive</span> Nat</span> : </span>Type</span></span>
| zero : Nat</span></span>
| casesOn : Nat -- will cause an error</span></span>
| succ : Nat → Nat</span></span></code></pre>
... will cause an error like this:</p>
error: (kernel) constant has already been declared 'MyNat.Nat.casesOn'</span></span></code></pre>
A more important helper is Nat.noConfusion</code>.
This is a theorem that says if t = t'</code>, then:</p>

Matching constructors imply equal arguments</li>
Different constructors are impossible</li>
</ol>
In table form:</p>
t</code></th> t'</code></th> ???</th></tr></thead>

zero</code></td> zero</code></td> Trivial - no args to compare</td></tr>
zero</code></td> succ a</code></td> Impossible - different constructors</td></tr>
succ a</code></td> zero</code></td> Impossible - different constructors</td></tr>
succ a</code></td> succ a₁</code></td> a = a₁</code></td></tr>
</tbody></table>
Note that noConfusion</code> is purely a helper: we could</em> write it by hand, but it's both tedious
and</em> mechanically derivable, so Lean gives it to us automatically.
How does this work? Let's examine Nat.noConfusion</code> by #check</code>ing it first.</p>
-- #check Nat.noConfusion</span></span>
MyNat.Nat.noConfusion.{u} {P : Sort u} {t t' : Nat} (eq : t = t') : Nat.noConfusionType P t t'</span></span></code></pre>
This isn't particularly helpful until we examine the definition</em> of noConfusionType</code>.
Informally, it unpacks as the following case analysis:</p>
t</code></th> t'</code></th> Nat.noConfusionType P t t'</code></th></tr></thead>

zero</code></td> zero</code></td> P → P</code></td></tr>
zero</code></td> succ a</code></td> P</code></td></tr>
succ a</code></td> zero</code></td> P</code></td></tr>
succ a</code></td> succ a₁</code></td> (a = a₁ → P) → P</code></td></tr>
</tbody></table>
More precisely, when we #print Nat.noConfusionType</code>, we get this nested case
analysis which first unpacks t</code>, then t'</code> within each branch.
I've indented the #print</code> for easier reading:</p>
@[reducible] protected def</span> MyNat.Nat.noConfusionType.</span>{u} : </span>Sort</span> u → Nat → Nat → </span>Sort</span> u :=</span></span>
    fun</span> P t t' =></span></span>
        Nat.casesOn t</span></span>
            (Nat.casesOn t'</span></span>
                (P → P)</span></span>
                fun</span> a => P</span></span>
            )</span></span>
            (</span>fun</span> a => Nat.casesOn t'</span></span>
                        P</span></span>
                        (</span>fun</span> a_1 => (a = a_1 → P) → P)</span></span>
            )</span></span></code></pre>
Lean will also add an injectivity helper Nat.succ.inj</code> as a useful special
case proof that Succ(a) = Succ(a') ⇒ a = a'</code> - and this generalises to cases
of other inductive types across each constructor.</p>


Will AI do to Software Engineering what Offshoring did to Manufacturing?
hellas.ai — Fri, 31 Oct 2025 00:00:00 +0000
AI is replacing work traditionally given to junior software engineers^{1</a></sup>.
The thesis2</a></sup> 3</a></sup> is that repetitive, boilerplate work can now be automated,
freeing senior engineers to apply the tacit</em> skills of programming which AI is not yet able to automate.</p>}
But this tacit knowledge is developed through years of practice:
the design and management of large codebases, managing complexity, knowing when
to incur tech debt, when to pay it off, and so on.</p>
So if there are no longer incentives to train junior software engineers,
what will happen to the industry?
We can make a prediction by observing how the same pattern unfolded in another
industry: manufacturing.</p>
US manufacturing employment peaked in 19794</a></sup>, after which offshoring began to
erode the labor force.
Consequently, there was less demand for trainees, resulting in a
"labor population pyramid" skewed toward older (senior) workers.
Now, as those seniors retire, they take their tacit knowledge with them, and it
becomes harder and harder to train new juniors5</a></sup>.
Ultimately, offshoring caused a vicious cycle which led to a skills gap around
30 years later6</a></sup>.</p>
Note that the opposite of this effect happens as well, to quote Pisano and
Shih</a>:</p>

Once an industrial commons has taken root in a region, a powerful virtuous
cycle feeds its growth. Experts flock there because that’s where the jobs and
knowledge networks are. Firms do the same to tap the talent pool, stay
abreast of advances, and be near suppliers and potential partners.</p>
</blockquote>
Now let's translate this to software engineering to make a prediction.
We have the same initial conditions, except instead of offshoring replacing
juniors, it's AI.
If the same pattern unfolds, then in ~30 years time we'd expect to see much of
the tacit knowledge of programming disappear from the workforce, and a similar
skills gap.</p>
Will this actually happen? This time there are some differences:</p>

AI may progress enough to replace seniors too (exacerbating the problem?)</li>
Software skills become outdated faster (we'll be training more juniors anyway)</li>
Software has a low barrier to entry for learning, and AI can help you learn</li>
Junior work is not offshored (if the AI is US-based), so is the "industrial commons" really eroded?</li>
The "industrial commons" of open source software is distributed and
accessible from anywhere</li>
</ol>
I'd bet against the erosion of software engineering purely based on (3) and
(5): it's easy enough to train oneself without significant monetary investment,
and the "industrial commons" of software exists at least partly in open source
projects where new developers can learn from seniors by contributing their
labor for free.</p>
However, I would</em> predict that new software engineers will have to front more
of the cost of learning that could previously be done on the job, and that this
will mean fewer people will select software engineering as a career for purely
economic reasons over "love of the game".</p>
Time will tell!</p>
^1</sup>Demand for junior developers softens as AI takes over</a></p>
</div>
^2</sup>https://x.com/yacineMTB/status/1984161544570335676</a></p>
</div>
^3</sup>Impact of AI on the 2025 Software Engineering Job Market</a></p>
</div>
^4</sup>Forty years of falling manufacturing employment</a></p>
</div>
^5</sup>The Manufacturing Skills Gap </a></p>
</div>
^6</sup>The skills gap in U.S. manufacturing</a></p>
</div>


Three Solutions to Nondeterminism in AI
hellas.ai — Mon, 29 Sep 2025 00:00:00 +0000
I want to convince you that verification is necessary</em> for you to
safely offload cognitive work onto AI while still trusting the results</em>.
Then I want to show you three ways to do this:</p>

Deal with floating-point nonassociativity</li>
Remove floating-point calculations from models entirely</li>
Verify tasks directly</li>
</ol>
I'll frame the discussion in terms of LLMs, but it applies to a much
broader class of models.</p>
(Side note: You may have seen a
recent blog post by Horace He</a>
at Thinking Machines
which discusses how non-determinism in tensor compute "turns on-policy RL into off-policy RL".
It's a great blog post which you should read, but consequences for RL are not
what I want to talk about here.)</p>
The Problem: Closed Source is not Verifiable</h2>
Assume you have no access to model weights, code, or inputs (e.g., the system prompt).
This is the state of affairs when you use a closed model provider like OpenAI or Anthropic.
In this case, the model provider is able to undetectably</em> behave in ways which
are not aligned with your interests, e.g.:</p>

Routing to lower-quality models at peak usage times</li>
Quantizing or degrading model quality to save money</li>
Injecting hidden messages into your context</li>
Censoring tokens, words, or full requests and responses outright</li>
Inserting overt or covert advertisements into responses</li>
</ul>
This is really just the AI flavour of
enshittification</a>,
where closed-source
software vendors extract value from you by degrading your experience.</p>
... but even open code and open weights are not enough</h2>
Now suppose you have an open model where code, weights, and inputs are all known to you,
and that you trust it to behave roughly in-line with your interests.
Assume that you don't have the hardware to run this model, and instead you ask
a third party compute provider to run it for you.
Ideally, you could in principle verify your model's outputs as genuine and detect any
naughty behaviour.</p>
However, nondeterminism means this is still</em> not possible in practice.
Simply put, nondeterminism means model outputs are not uniquely determined by inputs,
so you cannot distinguish between:</p>

Honest behaviour (A genuine model output which is different to the one you calculate)</li>
Dishonest behaviour (A faked model output which has been manipulated on purpose)</li>
</ol>
Model nondeterminism fundamentally arises from non-associativity of floating point arithmetic:
the "Original Sin" as Horace He puts it</a>.
A simple example is the sum below</p>
>>> 0.1 + (0.2 + 0.3)</span></span>
... 0.6</span></span>
>>> (0.1 + 0.2) + 0.3</span></span>
... 0.6000000000000001</span></span></code></pre>
Even small differences like this can cause large output changes when a model
contains nonlinearities:</p>
import</span> numpy</span> as</span> np</span></span>
</span>
def</span> foo</span>(w, x, y):</span></span>
    return</span> (</span>sum</span>(w)</span> ></span> x)</span> *</span> y</span></span>
</span>
# returns either 0 or 3.14e10, depending on chosen reduction bracketing.</span></span>
foo(np.array([</span>0.1</span>,</span> 0.2</span>,</span> 0.3</span>]),</span> 0.6</span>,</span> 3.14e10</span>)</span></span></code></pre>
This example is a little contrived, but the problem does</em> occur in real
models. For example, in
Nondeterminism and instability in neural network optimization</a>
the authors claim:</p>

(..) that even one- bit changes in initial parameters result in models
converging to vastly different values.</p>
</blockquote>
So how do we solve this?</p>
Solution 1: Say what you mean</h1>
The essence of the problem is that a model is not a single function,
but instead a set</em> of functions considered equal
up to associativity of floating point arithmetic</em>.
For example, the two "bracketings" x + (y + z)</code> and (x + y) + z</code> represent
two different functions (because addition is not associative).
But when we write sum([x, y, z])</code>, it is ambiguous which function is meant!</p>
In fact, this ambiguity is useful</em>: if we avoid specifying, our compiler can
choose the function which will perform best on some specified hardware--
typically in the form of specialised instructions.</p>
The first solution to nondeterminism is then to simply not throw away the
information</em> about which bracketing was chosen by the compiler.
Concretely:</p>

Choose a model (m), representing a set of functions equal up to FP nonassociativity</li>
Compiler chooses a (deterministic) function (f \in m) optimized for chosen hardware</li>
The user runs (y = f(x)) on the untrusted compute provider</li>
The result (y) is now deterministic and verifiable given inputs (x).</li>
</ol>
Incidentally, supporting this flow is a major design goal of catgrad</a>.</p>
Note that there are still a couple practical challenges:</p>

Intrinsics like exp</code> can vary across devices (even the same device can have
multiple versions; see CUDA's exp</code> vs
__expf</code></a>.</li>
Interleaving compute with other requests for efficiency requires batch
invariance</em>; read more about this latter point in the Thinking Machines
post</a>.</li>
</ul>
Solution 2: Avoid floating-point</h1>
The second solution is simple to state but harder to do:
don't use floating point arithmetic in your model</strong>.
If we want to verify AI tensor compute in general, this means floats have to be
eliminated at both inference and</em> train time.</p>
There are several approaches in this direction
but to my knowledge, none achieves the gold standard of completely
eliminating floating point at inference and</em> train time (including in the
optimizer) while also providing convergence guarantees.</p>
Some approaches which get close are below:</p>

BitNet</a> replaces floating-point weights in linear layers with 1-bit weights, but but does not fully eliminate the use of floating point weights at inference time.</li>
1 bit LLMs</a> binarize all</em> weights, but still require floats at train time</li>
RDA</a> defines a backprop procedure for training boolean circuits entirely without floats, but provides no convergence guarantees.</li>
BOLD</a> propose a method for directly training models with boolean weights, provides a convergence proof, but requires floating point values at train time.</li>
</ul>
... and here's how they fare at eliminating floats where ✅= no floats, ❓= floats in some layers, and ❌= floats required:</p>
Paper</th> Inference</th> Training</th> Optimizer</th> Convergence</th></tr></thead>

BitNet</a></td> ❓</td> ❌</td> ❌</td> ✅ (empirical)</td></tr>
1-bit LLMs</a></td> ✅</td> ❌</td> ❌</td> ✅ (empirical)</td></tr>
RDA</a></td> ✅</td> ✅</td> ✅</td> ❌</td></tr>
BOLD</a></td> ✅</td> ❌</td> ❌</td> ✅</td></tr>
</tbody></table>
My (and others</a>) opinion is that this
research direction is still underexplored: not only should we seek to replace
the underlying arithmetic upon which models are built, but also reconsider the
architectures and optimization procedures used.</p>
My own contribution in this direction is a paper to appear at OPT 2025 at NeurIPS
(blog post and arxiv link to follow)
in which we show that convergence guarantees can be obtained even when parameter updates are fully discrete.
We give an example based on multinomial sampling which still requires full precision gradients,
but in principle our method allows for fully discrete inference, training and
optimization with convergence guarantees as long as some standard assumptions are
satisfied.</p>
Paper</th> Inference</th> Training</th> Optimizer</th> Convergence</th></tr></thead>

Multinomial</td> ✅ None</td> ❌ Yes</td> ✅ Yes</td> ✅ Yes</td></tr>
</tbody></table>
At Hellas</a> we continue to investigate this direction,
in particular towards finding an optimizer which satisfies the "gold standard".</p>
Solution Level 3: Verify a task</em>, not compute</h1>
Another solution is to embrace model nondeterminism, and instead verify tasks</em> instead.
This does not work for every task; only those which are mechanically verifiable</em>.
Nevertheless, there are several interesting/useful examples:</p>

Mathematical: e.g., prove a theorem</li>
Algorithmic/search: e.g., find a negative-weight cycle in this graph of exchange rates</li>
Software engineering: e.g. write a function which passes these unit tests</li>
Cryptographic: e.g., obtain a signature for message (m) from the owner of public key (k)</li>
</ul>
This last point hints at a more "agentic" ecosystem: imagine for example where
(k) is the public key identifier of an online shop, and the message (m) is a
proof that a user purchased an item.
This would allow an agent to show that the task has been completed "up to
trust of real-world delivery".</p>
The advantage of direct task verification is that it also unlocks "intelligence markets":
one no longer has to care about which model was run (or even if it was a model at all!)
Instead, one can directly verify that the model achieved the goal you wanted to
achieve.
This means that instead of having compute providers</em> compete on cost to run a particular model,
we can instead of intelligence providers</em> which compete on solutions to tasks</em>:
a true, efficient market for cognitive work.</p>
Conclusion</h1>
To sum up, we talked about three ways to offload cognitive work while still trusting the results:</p>

Make models deterministic again</li>
Throw away your floating point</li>
Verify tasks directly</li>
</ol>
We're continuing to work on all three of these, so if you find this
interesting, I'd love to hear from you on the Hellas
discord</a>!</p>


Visualising LLMs with Open Hypergraphs and Catgrad
hellas.ai — Tue, 10 Jun 2025 00:00:00 +0000
Let's visualise some LLM architectures!
In this blog post, I'll show you how to generate these diagrams
using
open hypergraphs</a>
and
catgrad</a>.
We'll produce three examples, including the attention diagram above and a
huge SVG of every single op in the Qwen architecture.</p>
Open Hypergraphs</h2>
Let's start with
a simple example -- a residual connection around a linear layer.
In catgrad, layers and architectures are represented as Open Hypergraphs</em>:
a datastructure for representing "circuit-like" syntax.
Here's the code defining our layer:</p>
pub fn</span> residual</span>(x</span>:</span> Var</span><</span>NdArrayType</span>,</span> Operation</span>>)</span> -></span> Var</span><</span>NdArrayType</span>,</span> Operation</span>> {</span></span>
    linear_layer</span>(</span>"linear"</span>, x</span>.</span>clone</span>())</span> +</span> x</span></span>
}</span></span></code></pre>
Under the hood, this constructs an open hypergraph
which we can visualise with the
open-hypergraphs-dot</a> library:</p>
</p>
Let's break this down:</p>

Nodes are depicted as black circles ● labeled with an array shape</em> (e.g., [8, 8]</code>).</li>
Hyperedges are depicted as boxes with multiple inputs and outputs. They correspond to operations</em> like MatrixMultiply</code>.</li>
Some nodes are designated as inputs and outputs: these are depicted as dashed, open-ended lines.</li>
</ol>
Point (3) is why these are open</strong> hypergraphs, and not just hypergraphs.</p>
Importantly, copying is explicit in the hypergraph structure</strong>.
See the multiple outgoing edges of the top-right node ● which encode the reuse of
the x</code> variable inside the residual</code> function.
Aside from being a useful way to visualise variable sharing, representing
copying explicitly is important to how our
ahead-of-time autodiff algorithm</a> works,
enabling decentralised training</em> on the Hellas Network.1</a></sup></p>
Attention Please</h2>
Now let's do a complete, self-contained example: an Attention layer. Here's the code:</p>
use</span> catgrad</span>::</span>core</span>::</span>nn</span>::</span>layers</span>::*</span>;</span></span>
use</span> catgrad</span>::</span>core</span>::</span>{</span>Dtype</span>,</span> NdArrayType</span>,</span> Operation</span>,</span> Shape</span>};</span></span>
use</span> open_hypergraphs</span>::</span>lax</span>::</span>{</span>OpenHypergraph</span>,</span> functor</span>::*</span>, var,</span> var</span>::</span>Var</span>};</span></span>
</span>
use</span> std</span>::</span>cell</span>::</span>RefCell</span>;</span></span>
use</span> std</span>::</span>rc</span>::</span>Rc</span>;</span></span>
</span>
// 1. Create an OpenHypergraph for Gemma's attention layer,</span></span>
// 2. Turn explicit copy operations into *nodes* in the hypergraph</span></span>
// 3. Save as an SVG.</span></span>
pub fn</span> main</span>()</span> -></span> std</span>::</span>io</span>::</span>Result</span><()> {</span></span>
    let</span> arrow</span> =</span> attention_arrow</span>();</span></span>
    let</span> arrow</span> =</span> var</span>::</span>forget</span>::</span>Forget</span>.</span>map_arrow</span>(</span>&</span>arrow);</span></span>
    save_svg</span>(</span>&</span>arrow,</span> "images/attention.svg"</span>)</span></span>
}</span></span>
</span>
pub fn</span> attention</span>(</span></span>
    builder</span>: &</span>Rc</span><</span>RefCell</span><</span>OpenHypergraph</span><</span>NdArrayType</span>,</span> Operation</span>>>>,</span></span>
    dim</span>:</span> usize</span>,</span></span>
    name</span>: &</span>str</span>,</span></span>
    x</span>:</span> Var</span><</span>NdArrayType</span>,</span> Operation</span>>,</span></span>
)</span> -></span> Var</span><</span>NdArrayType</span>,</span> Operation</span>> {</span></span>
    let</span> num_heads</span> =</span> 4</span>;</span></span>
    let</span> head_dim</span> =</span> dim</span> /</span> num_heads;</span></span>
    let</span> b</span> =</span> x</span>.</span>label</span>.</span>shape</span>.</span>0</span>[</span>0</span>];</span></span>
    let</span> s</span> =</span> x</span>.</span>label</span>.</span>shape</span>.</span>0</span>[</span>1</span>];</span></span>
</span>
    let</span> k</span> =</span> linear</span>(builder, dim, dim,</span> &</span>format!</span>(</span>"{name}.key"</span>), x</span>.</span>clone</span>());</span></span>
    let</span> q</span> =</span> linear</span>(builder, dim, dim,</span> &</span>format!</span>(</span>"{name}.query"</span>), x</span>.</span>clone</span>());</span></span>
    let</span> v</span> =</span> linear</span>(builder, dim, dim,</span> &</span>format!</span>(</span>"{name}.value"</span>), x);</span></span>
</span>
    let</span> q</span> =</span> reshape</span>(builder,</span> Shape</span>(</span>vec!</span>[b, s, num_heads, head_dim]), q);</span></span>
    let</span> k</span> =</span> reshape</span>(builder,</span> Shape</span>(</span>vec!</span>[b, s, num_heads, head_dim]), k);</span></span>
    let</span> v</span> =</span> reshape</span>(builder,</span> Shape</span>(</span>vec!</span>[b, s, num_heads, head_dim]), v);</span></span>
</span>
    let</span> q</span> =</span> transpose</span>(builder,</span> 1</span>,</span> 2</span>, q);</span></span>
    let</span> k</span> =</span> transpose</span>(builder,</span> 1</span>,</span> 2</span>, k);</span></span>
    let</span> v</span> =</span> transpose</span>(builder,</span> 1</span>,</span> 2</span>, v);</span></span>
</span>
    let</span> tk</span> =</span> transpose</span>(builder,</span> 2</span>,</span> 3</span>, k);</span></span>
    let</span> attn</span> =</span> mat_mul</span>(builder, q, tk);</span></span>
    let</span> denom</span> =</span> constant</span>(builder, attn</span>.</span>label</span>.</span>clone</span>(),</span> f32</span>::</span>sqrt</span>(head_dim</span> as</span> f32</span>));</span></span>
    let</span> attn</span> =</span> attn</span> /</span> denom;</span></span>
    let</span> attn</span> =</span> softmax</span>(builder, attn);</span></span>
    let</span> attn</span> =</span> mat_mul</span>(builder, attn, v);</span></span>
    let</span> x</span> =</span> transpose</span>(builder,</span> 1</span>,</span> 2</span>, attn);</span></span>
    let</span> x</span> =</span> reshape</span>(builder,</span> Shape</span>(</span>vec!</span>[b, s, dim]), x);</span></span>
    linear</span>(builder, dim, dim,</span> &</span>format!</span>(</span>"{name}.proj"</span>), x)</span></span>
}</span></span>
</span>
// Build the open hypergraph by creating a Var and calling the attention function</span></span>
fn</span> attention_arrow</span>()</span> -></span> OpenHypergraph</span><</span>NdArrayType</span>,</span> Operation</span>> {</span></span>
    let</span> dim</span> =</span> 8</span>;</span></span>
    let</span> name</span> =</span> "attention"</span>;</span></span>
    var</span>::</span>build</span>(</span>|</span>state</span>|</span> {</span></span>
        let</span> x</span> =</span> Var</span>::</span>new</span>(</span></span>
            state</span>.</span>clone</span>(),</span></span>
            NdArrayType</span>::</span>new</span>(</span>Shape</span>(</span>vec!</span>[</span>1</span>,</span> 1</span>,</span> 8</span>]),</span> Dtype</span>::</span>F32</span>),</span></span>
        );</span></span>
        let</span> y</span> =</span> attention</span>(</span>&</span>state, dim, name, x</span>.</span>clone</span>());</span></span>
        (</span>vec!</span>[x],</span> vec!</span>[y])</span></span>
    })</span></span>
    .</span>unwrap</span>()</span></span>
}</span></span>
</span>
use</span> graphviz_rust</span>::</span>cmd</span>::</span>{</span>CommandArg</span>,</span> Format</span>};</span></span>
</span>
// Render an OpenHypergraph to an SVG using `open-hypergraphs-dot`</span></span>
fn</span> save_svg</span>(arrow</span>: &</span>OpenHypergraph</span><</span>NdArrayType</span>,</span> Operation</span>>, filename</span>: &</span>str</span>)</span> -></span> std</span>::</span>io</span>::</span>Result</span><()> {</span></span>
    let</span> dot_graph</span> =</span> open_hypergraphs_dot</span>::</span>generate_dot</span>(arrow);</span></span>
    let</span> png_bytes</span> =</span> graphviz_rust</span>::</span>exec</span>(</span></span>
        dot_graph,</span></span>
        &mut</span> graphviz_rust</span>::</span>printer</span>::</span>PrinterContext</span>::</span>default</span>(),</span></span>
        vec!</span>[</span>CommandArg</span>::</span>Format</span>(</span>Format</span>::</span>Svg</span>)],</span></span>
    )</span>?</span>;</span></span>
    std</span>::</span>fs</span>::</span>write</span>(filename, png_bytes)</span>?</span>;</span></span>
</span>
    Ok</span>(())</span></span>
}</span></span></code></pre>
This produces the following diagram:</p>
</p>
See here</a> for a github repo with both examples.</p>
Bonus: Qwen3-0.6B</h2>
To conclude, you can download the whole
Qwen3-0.6B</a> architecture as a diagram
here</a>.
I haven't included it on this page because it's 7MB!</p>
The qwen code is more complex, spread across multiple functions in catgrad</a>,
so to reproduce this diagram, see the code on this branch</a>,
and run:</p>
cargo run --release --example llm -- -m Qwen/Qwen3-0.6B -p "Catgrad is " -s 10 --model-svg qwen.svg</span></span></code></pre>
Finally, I want to emphasize that open hypergraphs are a general</em>
datastructure for syntax, not just for neural networks and catgrad.
Any kind of "circuit-like" term is a natural fit: from actual circuits to the
kind of boxes-and-wires visual languages used in game engines.</p>
If you have an idea for how you could use open hypergraphs and you want some help with the library,
hop in our discord</a> and let us know!</p>
^1</sup>For further reading about open hypergraphs for (differentiable) syntax,
you should know that the diagrams produced here are called
string diagrams</a>,
a formal graphical syntax for
Symmetric Monoidal Categories</a> (SMCs).
"Open Hypergraphs" (aka cospans of hypergraphs) formally</em> correspond to arrows
of SMCs, and our autodiff algorithm</a>
is built on this correspondence this to implement Reverse
Derivatives</a> for AoT autodiff.</p>
</div>


Announcing Hellas Gate
hellas.ai — Mon, 19 May 2025 00:00:00 +0000
Today we're launching Hellas Gate</a>.
Right now, it's an LLM gateway similar to
OpenRouter</a>,
LiteLLM</a>, and others, allowing you
to access models from many different providers through a central API.</p>
But this is table stakes.</p>
We're aiming for something a little different: empowering individual devs, not
enterprises.
Here's a quick taste of our roadmap, and how we're planning to do that.</p>
Local Compute</h1>
First up, we're making your local LLM accessible from anywhere.
Think "tailscale for your home GPU".</p>
You will be able to:</p>

Access your own vLLM/Ollama API from anywhere</li>
Pool shared compute resources with friends</li>
Inspect, debug, and modify the prompts and chats made by your local tools</li>
</ul>
... and more to come.</p>
Virtual Models and  Smart Routing</h1>
Next up: virtual models and smart routing.
Our aim here is to save you money, and level up your tools so they use the best
models for any given task.</p>
Writing some SQL? Use a cost-effective SQL fine-tune.</p>
Paying Anthropic $10 for RAG with claude-cli?
Use your local Qwen instance instead.</p>
Here's how it works.</p>
Model Aliases</h2>

Create a model alias like myusername/coding-rag</code>.</li>
Configure routing this alias in the Hellas dashboard. For example, have it always use QwQ-32B</code> on your local GPU.</li>
Configure your local tools to use the model myusername/coding-rag</code> for RAG</li>
</ul>
Want to change things later?
It's managed in one central place: edit your alias in the dashboard.
No fiddling with all your different tool configs!</p>
Smart Routing</h2>
Model aliases aren't just for picking one model.
You can set rules to route to different models dynamically based on criteria.
For example, let's say we want to use DeepSeek, but only when during off-peak times for lower cost.</p>
Achieve this, by configuring the myusername/coding-rag</code> alias with price limits</em>.</p>
Other filters and options in smart routing:</p>

Geography (for data privacy)</li>
Providers (e.g., "OpenAI models only")</li>
Attributes (e.g., "Best model for coding")</li>
</ul>
... and more to come.</p>
Conclusion</h1>
We're constantly improving Gate.
If you have questions, feedback, or just want to hang out, come talk to us on
discord</a>.</p>


Hello, World!
hellas.ai — Thu, 01 May 2025 00:00:00 +0000
Hello, World!</p>
Welcome to the blog for Hellas: a decentralised network for AI.</p>
We're building Hellas to guarantee an open-source future where the power of
artificial intelligence concentrates in the hands of individuals, and not</em> in
a few big companies.</p>
Keep an eye on the blog for research, product updates, and community announcements,
and make sure to join our discord</a>!</p>

References</h3>
Nvidia article on floating point</a></p>
Correctly Rounded Evaluation of a Function: Why, How, and at What Cost?</a></p>
Elementary Functions: Algorithms and Implementation</a>, a book by Jean-Michel Muller</p>

Visualising LLMs with Open Hypergraphs and Catgrad

Announcing Hellas Gate

Conclusion</h1>
We're constantly improving Gate. If you have questions, feedback, or just want to hang out, come talk to us on discord</a>.</p>

Hello, World!

Hellas Blog

Layered Cyclic Diagram Layout

Deterministic transcendental functions

thunderbolt-ibverbs: We have InfiniBand at home

Inductive Types in Lean

Will AI do to Software Engineering what Offshoring did to Manufacturing?

Three Solutions to Nondeterminism in AI

Hellas Blog

Layered Cyclic Diagram Layout

Intuition</h1> Here's a sketch before going into detail. Say we have a NOT gate with its output connected to its input:</p> </div> This is represented as a hypergraph with:</p>

Deterministic transcendental functions

References</h3> Nvidia article on floating point</a></p> Correctly Rounded Evaluation of a Function: Why, How, and at What Cost?</a></p> Elementary Functions: Algorithms and Implementation</a>, a book by Jean-Michel Muller</p>

thunderbolt-ibverbs: We have InfiniBand at home

Inductive Types in Lean

Will AI do to Software Engineering what Offshoring did to Manufacturing?

Three Solutions to Nondeterminism in AI

Visualising LLMs with Open Hypergraphs and Catgrad

Announcing Hellas Gate

Local Compute</h1> First up, we're making your local LLM accessible from anywhere. Think "tailscale for your home GPU".</p> You will be able to:</p> Access your own vLLM/Ollama API from anywhere</li> Pool shared compute resources with friends</li>

Model Aliases</h2> Create a model alias like myusername/coding-rag</code>.</li> Configure routing this alias in the Hellas dashboard. For example, have it always use QwQ-32B</code> on your local GPU.</li>

Conclusion</h1> We're constantly improving Gate. If you have questions, feedback, or just want to hang out, come talk to us on discord</a>.</p>

Hello, World!

References</h3>
Nvidia article on floating point</a></p>
Correctly Rounded Evaluation of a Function: Why, How, and at What Cost?</a></p>
Elementary Functions: Algorithms and Implementation</a>, a book by Jean-Michel Muller</p>

Conclusion</h1>
We're constantly improving Gate. If you have questions, feedback, or just want to hang out, come talk to us on discord</a>.</p>