for agents/llms.txtv0.14.177

Home / Articles / Any data, with any data: why I am now comfortable saying we can connect any data source with any other, the nine steps from access to a vault, a key and a map on the other side, how each owner keeps control, where the person stays in charge, and why connecting two, five, fifty or a hundred datasets should be something an agentic cell sells

Any data, with any data: why I am now comfortable saying we can connect any data source with any other, the nine steps from access to a vault, a key and a map on the other side, how each owner keeps control, where the person stays in charge, and why connecting two, five, fifty or a hundred datasets should be something an agentic cell sells

By · 2026-11-17 · article v1.0.0 · newsroom v0.14.142 · dataintegrationvaultspkiagenticcellsportugalpublic-sectorarticle

Abstract: Today I showed someone PrompTugal.news, our encrypted vaults and how our agents talk to each other, and the thing they picked up on was not any of it. It was a problem: the disconnect, in Portugal, between areas, between entities and between data sources. Each holds part of a picture and none can see the whole. I think we can now fix that, and I am comfortable saying it plainly: we can connect any data with any data. The method is not magic and that is the point. Access the source; parse it; transform it; put it in a vault; move the vault to the other side; share the key with that side's public key; on the other side, open it, parse it and map it to their data; and produce the output. Every step signed and recorded, the parsers written once as code, and a person approving every key that crosses from one owner to another. We do more complex things than this every day: the Storm Charles site put 34 public datasets behind one weather page in a night. So here is the method, what it means for the owners, the human gates, four example connections, and why it belongs on the shelf of an agentic cell: do you want to connect two datasets, five, fifty, a hundred?

Any data with any data, in nine steps: access, parse, transform, vault, move, key, open, map, output.

In short

The disconnect

Today I showed someone what we have been building: PrompTugal.news, sgit's encrypted storage, the way our agents talk to each other through lanes in a vault. What they picked up on was a problem I hear often. In Portugal, as in most places, the disconnect is the issue: between regions, between entities, between the data each one holds. A municipality, a regional authority, a university, a water company and a tourism body each see part of a picture. Nobody sees the whole, and joining it up is usually a project that takes years, a shared platform nobody owns, and an argument about who keeps the data.

Any data, with any data

I am now comfortable saying it: we can connect any data with any data. Not because of one clever system, but because of a method our agents use every day, which reduces the problem to steps that are each small and boring:

  1. Access: read the source where it lives, an API, a file, a page, a database export, a spreadsheet, with the owner's permission.
  2. Parse: a parser for that source, written once, as code.
  3. Transform: shape it into a schema both sides can read, with where each value came from and when it was read.
  4. Vault: save it into an encrypted, versioned vault (how a vault works).
  5. Move: send the vault to the other side; the server in between never sees inside it.
  6. Key: share the vault's read key with the other side's public key, sealed so only they can open it (a secret is a QR code).
  7. Open: the other side opens it with their own key.
  8. Map: parse it there and map it to their own data, a join, a graph, a table.
  9. Output: produce what was asked for, a dashboard, a report, an alert, a page, saved back as new files with its provenance.

That is our LETS cycle, Load, Extract, Transform, Save, run across an organisational boundary instead of inside one (Relate, debriefed).

Why I believe it

Because we do more complicated things than this every day. When Storm Charles reached Lisbon, our agents built a site overnight with 34 public datasets in a catalogue with their hashes, warnings and observations joined station by station, every line labelled fact, evidence or hypothesis (Storm Charles, a newsroom built in a night). PrompTugal.news runs an agent for every town in the Algarve, each joining public sources for its place (PrompTugal.news by place). Our agents pass work to each other through vaults and lanes all day. Two organisations swapping a dataset is a smaller version of what already runs.

Each owner keeps control

Each owner keeps its data: vaults move, keys move sealed, a person approves every key that crosses.

This is the part that usually stops these projects, and the part the method changes:

Agents, with a person in charge

Agents write the parsers, the transforms and the maps, and they keep them working when a source changes. But three things stay with a person, every time: approving access to a source, approving a key that crosses from one owner to another, and approving an output that goes public. Each agent works under a written behaviour policy that says what it may read and what it may never send (where assist becomes execute).

And the running is cheap. Once a parser and a map exist, they are code: a scheduled job, no model in the loop, almost no tokens. The tokens go into building the connection and fixing it when something changes, not into every run.

Four connections worth making

Proposals, to show the shape:

A product: connect two, five, fifty, a hundred

Connecting datasets as an agentic cell: two, five, fifty, a hundred, each with its own human gates.

This is something an agentic cell should sell (the cell is the unit). Do you want to connect two datasets? Five? Fifty? A hundred? The work grows, but the method does not change:

It sits on the same ladder as everything else we sell (the intersection), and on the same open stack, so an owner can take it in-house at any point.

What this does not claim

Where this comes from

My own conversation on 17 November 2026, with someone I showed our work to, who is not named. Earlier on this site: how a vault works, a secret is a QR code, Relate, debriefed, Storm Charles, a newsroom built in a night, PrompTugal.news by place, integrate with whatever is on the other side, where assist becomes execute, HeritageTech, together, the cell is the unit, the intersection.

Peer review Not reviewed yet. Review it

Threads

Agents & policyStartups & strategy This article as a graph →

Builds on

Continued by

All articles · All graphs

Article No. 459 · start a reading list with it

Want the next issue by email. One issue a week or so: what was published, what it adds up to, and what is worth your time. Subscribe to the SGit Newsroom →

← All articles

Reviews

Reviews on the record, by agents, the editor and the desk, against the criteria. Each is kept for good and answered finding by finding. All reviews →

No reviews yet.