Who Holds the Chalk

Co-authored by Claude · Version 1.0.0, changed in rev 133

A teacher stands at a slate blackboard, reaching up to write with chalk, while five students at their desks watch; one raises a hand. — A classroom blackboard: one surface the whole room can see, and one hand at a time holding the chalk.

One store shared by many agents, and the old idea of a blackboard

A blackboard is a surface anyone in the room can write on and everyone in the room can read. That is why it has lasted. It is also why it fails as a record: it does not say whose hand wrote a line, and a cloth takes everything off it without a trace.

Software agents working on one problem want the first half of that and cannot afford the second. This essay is about a DKE store used as their shared surface: many agents, one store, and a key of its own for each agent. Everything it says about DKE is something DKE's published documents say, and it links them.

A board for the whole room

The idea is older than the word. Early in the 11th century Alberuni described the schools of India: "They use black tablets for the children in the schools, and write upon them along the long side, not the broadside, writing with a white material from the left to the right." Large boards for a whole class are harder to date, but Europe was using them to teach music and composition as far back as the 16th century.

English has had the word since the middle of the 18th century, and the Oxford English Dictionary gives a citation from 1739 for writing with chalk on a black board. The first attested use of chalk on a blackboard in the United States was on 21 September 1801, in a course of mathematics lectures given by George Baron. The first boards were thin sheets of smooth black or dark grey slate.

What made the board useful is that everyone in the room sees the same thing at the same moment, and nobody has to send a copy to anyone. What it never offered is a memory. Whoever holds the chalk can write anything, and whoever holds the cloth can take it away.

Specialists around one board

Computing borrowed the picture. In a blackboard system, a group of specialist programs, called knowledge sources, work on one problem through a shared surface, the blackboard. The problem is written there first. Each specialist watches the board, and when something appears that its expertise can build on, it adds a partial solution of its own, which may let another specialist add one in turn. The work goes on until the problem is solved. The model was designed for complex, ill-defined problems, where the solution is the sum of its parts.

One of the well-known early systems was Hearsay-II, which recognised speech. Erman, Hayes-Roth, Lesser and Reddy described it in ACM Computing Surveys in 1980, and H. Penny Nii wrote a Stanford report titled Blackboard Systems in 1986.

The classic account has three parts: the specialists, the board, and a control shell that organises the flow of the work. The shell is there because, as the account puts it, the specialists "need a moderator to prevent them from trampling each other in a mad dash to grab the chalk".

A room of people answers two questions without thinking about them: who wrote this line, and who may wipe it off. A surface shared by programs has to answer both in writing.

What a store holds

A store is where DKE keeps what you tell it and what follows from it. It holds facts, and every fact written to it names the source it cites. It holds standing rules and the conclusions they draw, each conclusion in force exactly while its condition holds. And it holds history: a value you revise keeps the value it replaced, and you can ask what has changed since a moment you marked.

Used by many agents, a store is the board in that picture. It is one surface every agent reads, where what one agent writes is there for the next to build on, and where a standing rule's conclusions stay in force for all of them exactly as long as the facts beneath them hold.

Keys are the chalk

An agent reaches a store over MCP by presenting a store key, a string that begins dke_s_, sent as Authorization: Bearer dke_s_…. A key addresses exactly one store, and that store is fixed for the life of the key: the agent does not choose a store per call. A store can have many keys, and an agent that works in two stores holds two keys.

A key is issued at one of five access levels: read; read-add, which records new facts and cannot change or delete anything already recorded; read-write; read-write-delete; or none, which keeps the key but lets it do nothing except call spec and report until it is given a level again. An additional key starts at read, which the documents call "the right level for anything exploring your data rather than building it", and a client that only reports what it sees, such as a sensor or an import job, needs no more than read-add.

A read-add key that tries to change a value already recorded is refused with scope_forbidden at the statement that tries it, and the call stops there, keeping what it wrote before. A key's level can be changed later from the account dashboard. Its store cannot.

That answers the room's second question. On a classroom board, anyone with a cloth can wipe. On a store, every tool that forgets takes a key that may delete, which is read-write-delete, and so does bringing removed values back, or giving up the means to bring them back.

Every line is signed

The first question has an answer too. Every claim written through a key is stamped with that key, and written_by reports it. The stamp comes from the credential the call was authenticated with, never from anything the program or its caller states, so it "cannot be asserted, only observed".

That is a different thing from a fact's source. The source is what the writer cited, a claim about the data that DKE records and cannot check, while written_by is who performed the write. In the Specification's words, "a claim may cite a trusted registry and have been written by anyone holding a key to the store." A conclusion that a standing rule draws has no writer at all, because nothing wrote it.

Both can be counted. agreement counts the sources cited for a value, and writers counts the distinct keys that wrote it. As the Python Reference puts it, "three sources cited by one key is corroboration one party assembled, while one source written by three keys is three parties independently recording the same thing."

All of this rests on one habit, which the documents ask for plainly: give every client its own key. "Two clients sharing one key produce a history in which their work cannot be told apart, and nothing can separate it afterwards." What a client says about itself when it connects is recorded against its key as well, but it is self-declared: it labels a connection and never authenticates one.

One call at a time

A store runs one call at a time. A call that arrives while another is running waits, and runs once the store is free, so two agents sharing a store are never both running a call on it at once.

That is the part of the moderator's job a store does, and only that part. Nothing on the wire decides which agent ought to act next, or ranks one agent's contribution above another's. That stays with whatever runs the agents.

A long call can be ended with stop, which reaches the call running when it is asked. A key that may not write is refused it, "because a stop can cut a call's writes short."

Two hands, two answers

On a classroom board, the second person to write an answer usually rubs out the first. A store does not. Two claims disagree when they stand at one location with different values, and "who recorded them does not enter into it". Disagreement is a state the store holds, not an error: both claims remain, reads keep working, and DKE never chooses between them. You say which source you go by, and that judgement leaves the disagreement standing for anyone who looks. A standing rule that reads a disputed value draws from each value that satisfies its condition, so the disagreement travels into the conclusion instead of being settled on the way.

The same holds when removed values come back. A restore never overwrites what was written after the mark. If a value's place has been written again under the same attribution, the old value is skipped, and kept, rather than put over the newer one. If it was written again under a different attribution, the old value returns beside the new, and the place holds both.

The cloth

Removal is deliberate, and so is its undoing. forget removes a fact and the conclusions drawn from it, except a conclusion that some other standing rule still supports, which stays. It needs a read-write-delete key.

Undoing a removal needs a mark set before it. checkpoint marks where the store is now, under a name you choose. It records a position, not a copy, and changes nothing. restore brings back the values removed since a mark, at the store's current position: it is not a rewind, and nothing done after the mark is undone. Removed values are kept only while a mark is standing, so a removal made with no checkpoint in place cannot be brought back, and nothing at the moment of removal says whether one was. Dropping a mark lets go of what it was keeping, and it needs a key that may delete, as restore does, since what it lets go of is what a restore could have brought back.

That restore acts inside one store. Restoring a whole store from a snapshot is a different act, done from the account dashboard rather than by an agent, and the documents warn that the two share only the word.

A boundary, not a hole

The document this essay grew from offered one more picture: a store as a black hole. It is worth testing, because it fits in one place and breaks in another.

An event horizon, in astrophysics, is a boundary in spacetime beyond which no signal can ever reach a given observer. Wolfgang Rindler coined the term in the 1950s. The idea behind it goes back to 1784, when John Michell proposed that gravity near a massive, compact object can be strong enough that even light cannot escape.

The picture fits the boundary. A store is one well-defined region with one way in, a key, and every write that crosses into it is recorded under the key that made it.

It breaks inside. Nothing comes back out from behind an event horizon, while everything in a store reads back out to a key allowed to read it. The one thing a store never gives back is a program's source text, because it keeps only the compiled form, and you keep your own source. Nor does a store lose anything to gravity. On the wire, a stored fact goes only when a key that may delete forgets it, a rule's conclusions are withdrawn when what they rest on goes, and a forgotten value comes back only if a mark was standing when it went.

So the black hole is the right picture for the edge and the wrong one for what is inside, which is the right way round for a shared record: a hard boundary, and nothing lost within it unless someone holding the right key chose to lose it.