newsroom.sgit.ai / databases

The SQL console

SQLite, compiled to WebAssembly, running in this tab over the Portugal section's JSON files. The tables are built when the page loads, from a spec that names the file and field behind every column. Pick a worked query or write your own; nothing leaves your browser.

engine loadedsql-wasm.js + .wasm, then init
data fetchedsame-origin JSON, bytes / ms
tables builtrows inserted / ms
last queryrows / ms

Worked queries

Loading the engine and the files…

The tables, and the file behind each

This is tables.json, the loader spec, rendered. The build ran the same spec through Python's sqlite3 and executed every worked query against it; a query that failed or came back empty would have failed the build. Arrays and objects are stored as JSON text and unpacked with json_each; booleans as 1 and 0.

TableFromColumns
nodesgraph.jsonnodesid, type, label, pack, source, role, role_class, org, day, time, kind, publisher, published, date, url, page, no_longer_listed
edgesgraph.jsonedgesid, verb, source, target, pack
verbsontology.jsonedgesverb, inverse, domain, range, pt_verb, pt_inverse, reads, origin
typesontology.jsonnode_typesid, pt, colour, definition
peoplepeople.jsonpeopleid, name, role, org, page, linkedin
orgsorgs.jsonorgsid, name, people, placeholder
sessionssessions.jsonsessionsid, title, day, time, kind, stage
stagessessions.jsonstagesid, label, pt, also_called
sourcessources.jsonsourcesid, snapshot, page, publisher, url, frozen, bytes, sha256, retrieved, state
coveragecoverage.jsonitemsid, publisher, kind, published, title, url, source, bytes, sha256, what_it_says
storiesstories.jsonstoriesslug, kicker, title, standfirst, published, byline, stands_on
changeschanges.jsonchangesfrom_snapshot, to_snapshot, count_from, count_to, added, removed, changed, reason_known
person_topicstopics.jsonperson_topicsperson, topic, source
person_tagstopics.jsonperson_tagsperson, tag, type, matched, source
lexiconlexicon.jsonentriesid, type, label, pt, pattern

Where SQL is the right language, and where it stops

Counting, grouping, joining two named tables, unpacking a column: SQL does these better than any graph language and the first eight worked queries are that. Walking a path is where it strains — one join per hop, the length fixed in the query, nothing known about inverses — and the recursive traversal near the end of the list is the honest picture of what a two-hop neighbourhood costs to ask for. The graph console asks the same two questions in SPARQL, for comparison.

For an agent

After tool:ready (detail.name = 'nsdb-sql'): window.__tools.sql.run(sql){columns, rows, ms} or throws with the SQLite message; .tables() lists tables with row counts; .examples() returns the worked queries with their build-time row counts; .timings() the load measurements; .select(id) puts a worked query in the editor. The page reads #q=<urlencoded sql> and ?example=<id>. Every call is read-only; there is no write path by design.