GRIME3 RC10 — Operational Measurement, History, and Playback
Plan approved October 18, 2026. RC10 is planned work; RC9 remains the current published release.
RC10 will let a hydrologist configure a deterministic processing station, process incoming images continuously, retain durable measurement history, recover after interruption, review stage in a hydrograph, export selected results, and replay images with overlays. Linux and native Windows acceptance, plus the established Conda and Docker paths, are required before delivery.
This public plan is derived from the detailed RC10 working plan at source commit 2c3d72a60cc91e6652ddfa33aa314f98b766fea0. The development roadmap defines the complete RC10–RC13 sequence toward v1.0.0.
Functional scope
1. Preserve the RC9 scientific and desktop baseline
Retain calibration, octagon detection/refinement, deterministic waterline and stage measurement, configuration compatibility checks, image-quality and linefind-quality reporting, overlays, single-image and folder/tree processing, browser/CLI access, catalog/Data Roots/assignments, portable configurations, native file browsing and its fallback, and launcher lifecycle behavior.
Any scientific-output changes require an explained reason and regression evidence. Existing historical guides, fixtures, user images/configurations, and other protected files remain protected.
2. Durable measurement database
The database is authoritative for production measurement history and hydrographs. PNG metadata is a portable provenance copy.
Records must preserve, as applicable:
- Site identity and source-image identity/location.
- Capture timestamp, its source, timezone interpretation, and processing timestamp.
- Water level and units.
- Measurement outcome, rejection/failure reason, and available quality scores.
- Configuration/calibration association and reproducible identity.
- Application/source/build and processing provenance.
- Paths or references to result JSON and associated images/overlays.
- Processing-attempt or revision association sufficient to explain controlled reprocessing.
Requirements:
- Automatic database initialization and a versioned schema.
- Durable commits and a defined recovery policy.
- Consistent records while ingestion writes and the browser reads.
- A documented backup/restore procedure that accounts for the database's actual storage mode.
- No dependence on the development checkout to read production history.
- Identity rules scoped appropriately to site, image, configuration, and processing revision.
- Deliberate reprocessing is distinguishable from an accidental duplicate and does not silently erase history.
- Output-write failures cannot be presented as a fully successful result with nonexistent artifacts.
Implementation preference: a local embedded database, likely SQLite, if inspection confirms it fits. The engine is a kickoff design choice, not a new requirement for an external database server.
3. Portable output metadata
Write important measurement/provenance information into generated PNG outputs:
- Image/site identity.
- Capture time and stage/units when available.
- Measurement status and quality information.
- Configuration/calibration identity.
- GRIME3 source/build identity and metadata-schema version.
Preserve original input files. Define how existing non-PNG overlays/results are handled without changing their meaning. Large diagnostics can remain in JSON/database records; the portable metadata contract must be documented.
4. Unattended ingestion and processing
- Select an input folder and, where configured, its folder tree.
- Resolve the applicable site configuration using existing assignment rules.
- Establish initial-backlog versus newly arriving image behavior explicitly.
- Wait for a completed write; handle temporarily unavailable or incompletely copied files.
- Use the existing scientific pipeline.
- Save results, overlays, PNG metadata, and database history.
- Continue past an individual rejected measurement or unreadable file with an understandable recorded outcome.
- Prevent accidental duplicate successful results during rescans or restarts.
- Recover interrupted/pending work according to a documented policy.
- Provide controlled retries and a visible terminal failure state rather than an endless silent retry.
- Provide start/stop and visible operational state through the supported user workflow, with CLI operation available.
- Prevent conflicting workers from processing the same station unintentionally.
- Prevent generated outputs from being repeatedly ingested when directory layouts overlap.
- Preserve pending work across an orderly shutdown.
A genuine scientific rejection is a completed measurement outcome, distinct from a transient file-read or processing failure.
5. Basic station setup and operation
A simple browser setup workflow should collect:
- Site/configuration and incoming-image folder.
- Whether subfolders are included.
- Output location.
- Database/history location.
- Whether unattended processing starts automatically.
Validate locations, permissions, configuration compatibility, and obviously conflicting directory choices. Show status, recent progress, counts, and actionable failures in user terms.
Provide a supported and documented automatic-start mechanism on Linux and Windows, reusing operating-system facilities where practical. Document how to stop or disable it. Exact mechanisms are selected during inspection; RC10 does not require a general-purpose service-management framework.
Normal setup should not require users to initialize database tables or edit Python code.
6. Hydrograph review
- Display deterministic stage against capture time for the selected site.
- Read production history from the authoritative database.
- Independently adjustable start and end range handles.
- Visible date/time values and precise date/time entry.
- Consistent timezone display and enforcement of start before end.
- Update the chart while endpoints change without restarting processing.
- Show rejected/failed outcomes where meaningful, with a documented visual distinction.
- Do not imply accepted stage values for failed measurements.
- Represent record gaps honestly; document line-connection behavior.
- Select a record and inspect its measurement details, configuration provenance, source image, and overlay where available.
- Handle missing/moved image files without losing measurement history.
- Refresh while unattended measurements arrive.
- Stay responsive on a realistically sized representative history.
RC10 displays raw deterministic history. SDF and Bayesian estimates enter in later candidates.
7. Export and bounded API
- Export selected history to CSV using the same site/time filters as the hydrograph.
- Include documented timestamps, units, status, quality, and source identity as applicable.
- Provide a stable, documented read contract for an external hydrograph utility.
- Define timestamp/timezone, ordering, selected-range boundaries, missing values, statuses, and schema/version behavior.
- Bound or paginate large requests as needed for responsive operation.
- Preserve existing access-control and local-server protections.
This is a measurement-history interface. It is not an expansion into a general hydrologic analysis platform or a new public hosting/authentication product.
8. Demonstration and playback
- Select a folder or tree of images.
- Establish deterministic ordering using the documented timestamp policy.
- Process images with the real GRIME3 pipeline, or replay compatible saved results.
- Show the image and selected measurement overlays.
- Update a synchronized hydrograph as images are presented.
- Pause/resume, stop, and loop.
- Apply time-range selection where practical.
- Show progress and errors clearly.
- Do not recompute scientific results merely because the display loops when compatible cached results exist.
- Keep demo session state and outputs separate from authoritative production history unless the user explicitly performs a production import.
Screen playback uses individual still images. Video export, animation encoding, FFmpeg, and OpenCV videoio are outside RC10.
Work order and resumable batches
| Batch | Work | Exit evidence |
|---|---|---|
| A | Baseline inspection, scope reconciliation and design decisions | Verified baseline and explicit requirement/test matrix |
| B | Database schema, result identity, durable writes, PNG metadata, result/overlay association | Consistent history and artifacts; documented reprocessing/recovery semantics |
| C | Unattended worker, backlog/new-image handling, readiness checks, duplicate prevention, recovery, logs | Fault/interruption tests and a working Linux ingestion workflow |
| D | Hydrograph, details/image linkage, range controls, CSV and read API | Browser behavior and agreement between chart, database, and exports |
| E | Demo/playback and basic station setup/automatic operation | Complete interactive and unattended Linux user journeys |
| F | Linux standalone/Conda/Docker packaging, documentation, extended operation | Linux acceptance report and accepted source checkpoint |
| G | Native Windows completion and packaged acceptance | Windows workflow evidence and cross-platform scientific comparison |
| H | RC10 release preparation, approved publication, website synchronization, cleanup | Coordinated downloads/docs, source provenance, committed/pushed changes |
An early, small native Windows check should follow the first working database/worker checkpoint. It checks imports, storage locations, background execution and a basic frozen launch, and should not become a full parallel Windows implementation effort.
Each batch must be rerunnable or resumable, record its checkpoint, and stop on an unexpected starting state. Combine independent checks, avoid redundant complete-suite runs, and rebuild only affected distributions. Full required checks still run at release gates.
Platform policy and packaging
Complete substantive Linux work first. Then complete Windows acceptance before RC10 publication. Keep one shared scientific implementation and one accepted source identity across distributions.
Deliver:
- Linux x86_64 standalone archive.
- Native Windows x64 PyInstaller distribution.
- Updated Conda package/instructions using the established supported path.
- Updated Docker/OCI distribution with persistent configuration, images, results, and database locations.
- Relevant source/release materials, checksums, build identity, and third-party notices.
Do not silently add claims for ARM64, RPi, macOS, new package registries, or operating-system versions not validated. Preserve any existing documented best-effort support without making it a new RC10 acceptance platform.
Preserve RC9's FFmpeg/videoio removal and dependency/license constraints. RC10 introduces no ML runtime. Training/inference packaging is RC12 work.
Acceptance criteria
The release is complete only when applicable checks pass in actual supported packages, not only the source environment.
Scientific and workflow regression
- Existing calibration, octagon, waterline, and stage fixtures retain accepted results.
- Browser, CLI, unattended and fresh-playback measurement paths agree.
- Rejected and invalid images have correct status and never fabricate stage.
- Site/configuration assignments and image compatibility remain enforced.
Storage and recovery
- Partially copied image, unreadable image, duplicate arrival/rescan, and interrupted work.
- Restart after an orderly stop and forced termination during representative processing stages.
- No unexplained duplicate accepted records or lost committed history.
- Missing output, failed output write, unwritable database/output, and missing source image.
- Deliberate reprocessing with changed configuration produces traceable history.
- PNG metadata and database provenance agree.
- Backup and restoration recover usable history.
Browser review and playback
- Independent endpoint adjustment and precise inputs; invalid range handling.
- Known timezone/timestamp cases and consistent chronological ordering.
- Chart, selected details, CSV, and API return matching records.
- Live refresh while ingestion is active.
- Pause/resume/stop/loop without duplicating station history.
- Responsive behavior on a representative larger history; test size is recorded.
Operational acceptance
- Fresh setup, backlog processing, new-image processing, and automatic startup.
- Stop, restart, and recovery demonstrated on Linux and native Windows.
- Proposed initial soak target: at least 24 hours on Linux and an extended native Windows run, with duration/load and outcomes recorded. Final duration/load criteria are fixed at kickoff.
- Throughput, failures, backlog, and resource behavior measured against the intended image-arrival workload; no invented universal throughput promise.
Distribution and release acceptance
- Fresh Linux standalone, Conda, Docker, and native Windows packaged use.
- Native dialogs/fallback and paths containing spaces.
- Persistent data survives application updates/restarts.
- Artifacts identify source, build, platform, architecture, checksums, and applicable licenses.
- No accidental return of excluded dependencies.
- Download instructions and final published links verified.
- Source and website changes committed/pushed to their respective verified remotes.
- Owned temporary work is cleaned up; protected data/toolchains are retained.
Documentation and user deliverables
- RC10 release notes and known limitations.
- Current installation/start/stop/upgrade instructions for supported distributions.
- Site setup and unattended-operation guide.
- Hydrograph, export, and playback instructions.
- Backup/recovery and reprocessing instructions.
- API/data/PNG metadata contracts at a practical level.
- Updated current user guide/help, preserving protected historical guides.
- Scientific/operational/platform acceptance report.
- Synchronized repository roadmap and GaugeCam public roadmap/download pages.
Scope boundaries
RC11 owns annotation UI and dataset preparation. RC12 owns SDF model development, scientific evaluation, execution modes, and optional ML deployments. RC13 owns Bayesian temporal analysis and final production integration.
RC10 does not include:
- Revival of the archived RC7 AI model path.
- New waterline/calibration algorithms without a demonstrated RC10 defect.
- Video encoding, animation export, FFmpeg, or OpenCV videoio.
- RPi camera capture, ARM64, additional target types, or whole-image discovery.
- General hydrologic modeling, cloud synchronization, or a broad plugin/API platform.
- Mandatory Debian/Ubuntu archive, WinGet/Store, PyPI, or Homebrew publication.
These boundaries do not delete future roadmap items.