pdigger scada · SM1 · design paper

Control-room screens. As data.What SM1 is A screen-reading model. A small computer on the screens’ video cables reads up to three operator screens, four times a second, and keeps a running account of them: parts and their states, values and labels, the direction of each chart, and the presses and typing it sees. Plant data from HMI screens in English or Russian, with no change to the control system.

Read from the video cables. Nothing installed on the plant’s computers.

Synthetic test session · confirmed examples only
  • 3screens of 1920 × 1080Three screens Up to three monitors of a control desk, each read from its own HDMI signal, the picture the operator sees.
  • 4readings a secondA fixed rate One reading every quarter of a second, on a fixed clock, however busy the screens are.
  • 48of 603 places a readingA fixed budget The three screens are divided into 603 places, from whole screens down to small blocks. Each reading looks at 48 of them: whole screens, the pointer, changes, and places stale or new.
  • 5minutes of setupSetup A live survey of the screens after the device is fitted. No labelling and no training on site.
  • 2languages, English and RussianLanguages Every line is read in English or Russian; a script head decides which recogniser reads each line.

Abstract

The screen is the one interface every control system has.

SM1 reads it the way an operator does: from the picture. A budgeted region tree spends a fixed number of looks a reading on the whole screens, the pointer, what changed and what has not been seen for a while; a frozen text model and a small trained network read those tiles, each with a memory of how its place looked last time; a world on the CPU keeps the account; and a scene API gives it back in the plant’s own words.Why this matters Plant data is often locked inside control systems that cannot be touched: old, unsupported, validated, or licensed per connection. The screen shows all of it to the operator already. Reading the screen turns what the operator sees into data without changing anything that runs the plant. One approach across vendors and ages of system, in English or Russian, with no change to the control system.

01 Only pixels in

On the cable. Not in the system.

The plant keeps its computers, software and monitors as they are.Only the picture A pass-through on each monitor cable copies the picture to SM1. It sees exactly what the operator sees and nothing else: no network tap, no USB tap, no access to the SCADA software. No change on the control side: the device only receives the picture.

Where SM1 sits Three plant computers drive their monitors as before. A pass-through on each video cable copies the picture to SM1, a small computer, which serves the scene API to the plant's own systems. pass-through Plant PC 1 Plant PC 2 Plant PC 3 operator screens, as before pixels only SM1small computer, reads the pictures Scene API Your systemshistorian · MES · reports · alerts The pass-through A capture device on each monitor cable passes the signal on to the monitor unchanged and copies it to SM1. The operator's screens look and behave as before. SM1, the small computer One edge computer for three screens, sized for a Jetson Orin Nano Super with 8 GB. It runs the network, the text recognisers, the world and the scene API. The scene API Plain HTTP and JSON with live streams: parts, names, events and alert rules, ready for a historian, a report or an alarm system.
Figure 1 Three screens, one small computer. The device reads; it never writes back.
  • No integration projectNo integration project No drivers, OPC servers, database access or vendor licences. Fit the pass-throughs, let SM1 survey the screens, then name what matters. Fit, survey, name.
  • Any vendor, any ageAny vendor, any age A grey high-performance HMI, a classic 3-D one, a dark DCS, a flat web page or a legacy screen in Russian: if the operator can read it in English or Russian, SM1 is built to read it. One way to get data from a mixed estate of control systems.
  • The operator’s own viewThe operator's own view SM1 records what the screens showed, the presses and typing it sees, and what changed at the same moment: the same view the people in the control room had. One running account of what the screens showed, for audits and incident reviews.

02 Principles

Eight ideas behind SM1.

Each one solves a problem every screen reader meets.

  1. 01

    Only pixels inOnly pixels in The video signal is the only input. What the operator can see, SM1 can learn to read; nothing it does can reach the plant’s systems. The control side stays untouched.

  2. 02

    Look where it changesA fixed budget of looks Most of a control screen is still most of the time. Each reading spends a fixed number of looks: on the whole screens, the pointer, the places that changed, and places not seen for a while. A fixed cost per reading, whatever the screens show.

  3. 03

    Reuse what is provenFrozen text models Text is found and read by published PP-OCRv5 models, used as they are. Only the small parts that make SM1 a screen reader are trained. Published text models, used unchanged, in both languages.

  4. 04

    Remember, never feed on itselfMemory that does not feed itself Each place carries its last look and how long ago it was taken. The memory holds the screens’ own pixels as the network saw them, never its own answers, so its input stays as in training, however long it runs. Designed so the network sees the same kind of input on day 30 as on day 1.

  5. 05

    Learn states, don’t list themStates are learned Green means running in one plant and stopped in another. SM1 does not hard-code meanings: it learns the set of looks each part shows and gives each a state name from its vocabulary, which you can rename. Works with each plant’s own conventions, without a configuration project.

  6. 06

    One account of the screensThe world A world on the CPU turns each reading into lasting facts: parts with stable identities, a text table, pages, colours and charts, with time in each state. One running account of what the screens showed, and when.

  7. 07

    Your words, onceName it once Name a page, a type of part or one part, and what a state means. Every event, past and future, then comes back under those names. Data arrives labelled the way the plant already talks.

  8. 08

    Judged against exact labelsExact labels, unseen plants Training screens are generated, so every text, part, state and input is known exactly. SM1 is judged on plant projects it never saw in training, and only confirmed outputs are shown. Decisions rest on evidence, stage by stage.

03 One reading

Every quarter second, eight steps.

Look where the screens changed, read what matters, update the account.One reading Blue steps run on the graphics processor, orange ones on the CPU. The network runs once a reading, on that reading’s tiles of all three screens together. Steady, predictable work per reading: the basis for running on a small device.

  1. 1

    CaptureCapture The newest frame of each screen, plus the two captured since the last reading. Around the pointer every frame counts, so a click shorter than a reading is still seen.

    3 screens

  2. 2

    Change mapsChange maps Where pixels moved: against the last reading, against the screen’s stable picture, and the most change in the short burst.

    what moved

  3. 3

    Region treeRegion tree Chooses exactly K tiles of 128 × 128 pixels where the screens vary most, in fixed lanes: pointer, change, refresh, stale, survey.

    48 tiles

  4. 4

    Tile engineTile engine The frozen PP-OCRv5 text detector, plus a small trained stem that adds what a text detector throws away: colour, and how the tile changed.

    frozen + stem

  5. 5

    Network and headsNetwork and heads Each tile becomes 16 tokens, each joined with its place’s last look. Four transformer blocks run over the tiles of all three screens; heads give parts, states, colours, text roles, pointer, inputs and pages.

    one pass

  6. 6

    DecodeDecode Turns the heads’ maps into things: parts with boxes and states, and text lines cut from one canvas per screen, so a line that crosses a tile edge comes out whole.

    maps to things

  7. 7

    Read textRead text Frozen PP-OCRv5 recognisers read the lines that changed, or have not been read for a while, within a fixed width each reading: one for Latin script, one for Cyrillic.

    changed lines

  8. 8

    WorldWorld Updates the parts and their states, the text table and the pages, then writes the reading’s packet for the scene API.

    the account

One network reads every tile

Small on purpose.

3.0 million trained parameters in all, beside a frozen text detector.Where the parameters are Every trained weight sits outside the frozen detector: the stem, the four transformer blocks and the heads. The transformer blocks are most of it. Small by design.

  • Parts
  • States
  • Colours
  • Text roles
  • Pointer
  • Inputs
  • Effects
  • Page

04 Where to look

Spend the pixels where screens vary.

A tree of places, one tile size, a fixed budget.The region tree Each screen is a tree of places, from the whole screen down to the 16 × 9 grid of 120-pixel blocks that tiles 1920 × 1080 exactly. Every place is seen through the same 128 × 128 tile: native pixels for a block, so text stays readable, and two times smaller for each level up. The number of looks per reading is set in advance, not by how busy the screens are.

The region tree of one screen A 16 by 9 screen with the grids of its levels: blocks, then larger places, up to the whole screen. Tiles chosen in one reading are marked by lane: the pointer and the block it heads for, places that changed, a place not looked at for a while, and a place being surveyed. pointer change stale survey illustration · one screen of three
Figure 2 Five levels of places on each screen; 603 on three. Block-level looks are filled, larger places outlined.
  • Pointerthe block under it and the one it heads forPointer lane Always served first: clicks, drags and typing happen here, and a press shorter than a reading still shows in the frames around the pointer.
  • Changeplaces that changed more than they usually doChange lane Change is judged against each place’s own usual behaviour, so a trend that always moves does not win every reading.
  • Refreshplaces the world asks to see againRefresh lane The lane lets the world ask to see a place again.
  • Staleplaces looked at longest agoStale lane Quiet places still get looked at regularly, so no place goes unvisited for long.
  • Surveyplaces never seen, then random onesSurvey lane During setup the whole tree is swept within seconds; afterwards, random places keep the picture fresh.

The budget is the rate control.One network, four budgets K is 16, 32, 48 or 64 tiles a reading, with one trained network for all of them. A global offset is solved each reading so that exactly K tiles are used, and a reading that runs long lowers K for the next. After a long reading, the next ones take fewer looks.

48 looks of 128 × 128 pixels a reading, across three screens.

The tiles chosen on a synthetic session

05 Memory

Most of the screen is not recomputed. Its last look stands in.

Memory that does not feed itself.Made for long running Training sees minutes; a device is meant to run for weeks. SM1’s memory holds pixel features, never the network’s own answers, so the single step that training shows is what runs on the device, however long it runs. Every store is a fixed pool with a cap and an eviction rule. Fixed memory by design.

How a tile is joined with its past The tokens of a tile read now are joined with the stored tokens of the same place from its last look, with that look's age. Tokens of the pointer's tiles from the last readings are added. Four transformer blocks run over all tiles of the three screens and feed the heads. The new pixel tokens are written back to the region bank. Tile, now16 tokens of width 192 Same place, last lookfrom the region bank, with its age Pointer, last readingsa ring of tokens + joined Transformer4 blocks over the tilesof all three screens Headsparts · states · colourstext roles · pointer · inputseffects · page pixel tokens written back · no answers fed back
Figure 3 The token count stays the same however much history there is.
  • Ages, not hidden statesAge-coded looks Each stored look carries how long ago it was taken. In training, ages are drawn from one reading up to minutes, and each episode rebuilds that earlier frame.
  • Drags and typing followedThe pointer ring A ring of the pointer’s tiles from the last readings adds a few tokens, so a drag or a typed word can be followed across readings.
  • Time per placeTime as a dimension Each place and each part keeps its own time axis, from seconds to hours and thirty days in all, at the resolution it needs.
  • Memory that does not growBounded memory The world keeps time in each state per hour and per day, ranges and a text change log, in a fixed budget of about 70 MiB. A fixed memory budget, by design.

06 The world

A world that remembers.

The network says what it sees now. The world keeps what is true over time.On the CPU, by rules The world runs on the CPU, beside the network. It keeps parts with stable identities through dialogs and absences, the states each has shown, the text on screen, and the pages, with rules that decide when a change counts. Lasting identities and histories: the basis for logs, reports and alerts.

Each part’s states over time
  • Registryparts and the states they have shownThe registry Each part keeps its identity while dialogs cover it or its page is away. The looks it takes are clustered into its own set of states, which keeps growing when it shows a new one; analog parts keep their range.
  • Text tableevery line, its role and latest readingThe text table Labels, values, units, buttons, titles, alarms, menus and inputs. Changed lines are read first, within a fixed width each reading.
  • Pagesscreens seen before, recognised againThe page bank Returning to a known page restores what was known about it, so its parts keep their identities.
  • Colour watchwhich light is on, and blinksColours The network names the colour each part shows from eleven names and learns which element carries it; the world reads its exact colour from the screen. A new colour counts after two sightings; a light that keeps alternating is reported once, as blinking between its two colours.
  • Chart readereach pen in wordsCharts, by rules The plot area from its gridlines, each pen by its colour, a 64-point curve per pen, the scale and time span from the axis labels. Each pen gets a sentence: rising, falling, steady or oscillating, now and over the window.
  • Kinds of partsthe same template, groupedKinds of parts A small embedding puts parts of one template close together whatever state they show, so the world can group them, and a group keeps its identity as parts come and go.

Plant A

Running

Stopped

Plant B

Stopped

Running

Figure 4 The same colour, two meanings. SM1 learns each part’s looks; the plant names them.States are learned, not listed No colour or shape is given a fixed meaning. The registry learns the set of states each part shows; one call to the scene API names what a state means for a class or a type of part.

07 What it reads

Parts, text, charts, pictures, inputs.

Five things a control room shows, each as data.What these films are Synthetic test sessions made by our screen farm: plant projects not seen in training. SM1’s own outputs are shown only where the session’s labels confirm them; the names are ours, given once as a user would.

Synthetic test sessions · confirmed examples only

Parts, in your words. Its vocabulary covers valves, pumps, motors, conveyors, gauges, lamps, buttons and alarm rows; parts keep their states, and alerts come in the plant’s own words.

08 Scene API

Name it once. Filter, stream, alert.

The account, in the plant’s own words, over plain HTTP.The scene API It runs beside the world on the device. One filter serves a query, a live stream and an alert rule. Records are numbered and kept in memory and in a file; a client that reconnects first gets the records it missed that are still in memory. Standard HTTP, JSON and server-sent events.

  1. 1 · Name

    A part, a state, a page.Names that stick A part’s name sticks to its place on its page: when the world drops it and finds it again there, the name moves to the new entry. Names are resolved when a record is read, so naming a part renames its past events too. Rename once; reports and history follow.

    POST /api/names
    {"part": "#463", "name": "Belt conveyor CV05"}
    
    POST /api/names
    {"state": "tripped", "class": "conveyor",
     "name": "Motor trip"}
  2. 2 · Alert

    A rule is a filter, optionally held.Held rules, cooldowns, webhooks A held rule waits and fires only if the part is still in the state it went to, so a pump that trips and restarts within the hold raises nothing. A cooldown applies per part. A fired alert is a record of its own and can be posted to a webhook. A held rule ignores a trip that clears itself within the hold.

    POST /api/alerts
    {"when": {"kind": "state",
              "part": "Belt conveyor CV05",
              "to": "Motor trip"},
     "name": "Conveyor trip"}
  3. 3 · Ask and stream

    The same filter, now or live.Queries and streams Filter by kind, part, type, class, screen, page, the state or colour a change went from or to, time and text. Streams are server-sent events with the same filters.

    GET /api/events?kind=state
        &part=Belt+conveyor+CV05
    
    [{"part": "Belt conveyor CV05",
      "from": "running", "to": "Motor trip", …}]
    
    GET /api/events/stream?kind=alert
The same event, with and without names

09 Workflows

From fitting to a filtered feed.

Two workflows: on site, and behind the model.

On site

  1. Fita pass-through on each monitor cableFit The device and its capture inputs go beside the control desk. The plant’s computers, software and monitors stay as they are.
  2. Surveyfive minutes on the live screensSurvey The same network runs in survey mode: it visits all 603 places within seconds, reads every line and builds the registry of parts. The rest of the five minutes learns which places animate on their own and which looks each part takes. Ready the same day: no labelling, no training on site.
  3. Runfour readings a second, steadilySteady state After setup the budget goes where things change, and the rate no longer depends on how busy the screens are.
  4. Namepages, parts and what states meanName A part with no name is shown by its caption, such as “Pressure Water pump”, until it is given one. Names are kept in a file and survive a restart.
  5. Usequery, stream, alert, reportUse The historian, MES, alarm tools or a shift report read the scene API. Each record keeps its time and screen, and its part and page where it has them.

Behind the model

  1. Screen farmsynthetic plants, styled after real screensThe screen farm A generator of HMI pages driven by plant simulators for brick, cement, foundry and concrete lines. Three headless browser pages are the three monitors; a scripted operator moves one pointer and types with human timing.
  2. Exact labelsfrom the page itselfLabels for free Every frame’s text, parts and states come from the page that drew it, and every input from the operator’s log. Training data without a labelling team.
  3. The cable, simulatedcolour detail, range, noiseThe capture path Training sees what a capture card delivers: halved colour detail, limited range and noise. The cursor is drawn in training too, so any cursor theme can be used.
  4. Many looksten style families, two languagesLooks and fonts Grey high-performance HMI, classic 3-D, dark DCS, flat web and a Russian legacy look, among others, in English and Russian. Four more families and three fonts are kept apart for testing.
  5. Train, then judgeon plants never seenHeld-out sessions Ten-minute test sessions of plant projects that training never saw. Each stage of the system is scored against the labels, so the next piece of work is chosen from evidence.

A pilot, step by step

Your screens, in the farm.

  1. 01Share screenshots of your screensScreenshots Screenshots of each page type show the looks, parts and languages a control room uses.
  2. 02We style the farm after themYour look in training The generated pages follow your screens’ style, so training sees parts that look like yours. A new kind of page can need new generator code.
  3. 03Check on sessions in your lookChecked first Scored on generated sessions in your style before anything goes to site.
  4. 04Fit, survey and name on siteOn site Pass-throughs on your monitor cables, the five-minute survey, then your names for pages and parts.
Join the pilot

10 How it is judged

Evidence first. Stage by stage.

How we decide what works, and what is next.

  • Against exact labelsEvery score compares SM1’s output with what the generated page really showed.
  • On unseen plantsTest sessions are plant projects that training never saw.
  • With ceilingsThe world, the text reader and the decoder are each replayed with perfect input, to find which stage limits the result.Ceilings If the world given perfect parts does well and the system does not, the parts are the gap, not the world. Each replay names the stage to work on next. Effort goes where it moves the result.
  • Confirmed onlyFilms and pages show SM1’s own outputs only where the labels confirm them.

Honest about pixels

An input that changes no pixel cannot be seen. Some inputs look the same.What the picture cannot tell Enter and a click on OK can leave the same picture. In both cases SM1 reports what changed, such as a dialog closed or a value entered, and the kind of input it saw: a press or typing, not which key or button. Built from the picture alone.

11 Specification

SM1 at a glance.

The design of SM1
InputUp to three screens of 1920 × 1080, from HDMI pass-through capture
RateFour readings a second, on a fixed clock, after a five-minute setup
BudgetK = 16, 32, 48 or 64 tiles of 128 × 128 pixels a reading, of 603 places; one network for all
TextPP-OCRv5 mobile detector and recognisers, frozen; English and Russian
Network16 tokens of width 192 per tile, each with its place’s last look; four transformer blocks; 3.0 million trained parameters
OutputsParts, states and colours; text with roles; pointer and inputs, beside what changed at the same moment; pages; charts in words
WorldRegistry, text table, page bank, colour watch, chart reader, kinds of parts; bounded memory
APIHTTP and JSON; names, filters, held alerts, cooldowns, webhooks; server-sent event streams that resume
DeviceSized for one Jetson Orin Nano Super, 8 GB
On siteNo software on the plant’s computers; no labelling or training on site

Pilot programme

Your screens, as data.

Tell us about your control room: how many screens, which systems, and what you would like to know from them. Contact us to join our pilot programme.