Add realtime_trace_jsonl recipe for structured real-time optimization progress streaming

[MySweetEden]

Motivation

PySCIPOpt already has recipe(s) that store optimization progress in memory. However, in-memory traces are not suitable for real-time, external observation (e.g., another process tailing progress, dashboards, log collectors).

This recipe focuses on the missing piece: a stream-friendly, structured output that can be consumed outside the running Python process during solve.

Design Decisions

• JSONL format: Designed for streaming writes and partial reads; remains readable even if the run is interrupted or crashes
• Real-time external output is the primary value:
  • Records progress updates as one JSON object per line
  • Flushes on key events so downstream consumers can react immediately
• Schema compatibility with setTracefile() (PR Add setTracefile() method for structured optimization progress loggingAdd settracefile api #1158):
  • Uses the same JSONL field names (type, time, primalbound, dualbound, gap, nodes, nsol) for consistency across tracing approaches.
• In-memory + file output: Keeps model.data["trace"] for convenience/testing, but the recipe is centered on file streaming via path=...
• Robust termination signaling for external monitoring:
  • Always emits a final run_end record on normal termination, interruption, or exception
  • On exceptions, run_end includes structured error metadata (status, exception type, message)
  • Flushes run_end to make completion detection reliable

Events Recorded

• bestsol_found: when a new best solution is found
• dualbound_improved: when the dual bound improves
• run_end: when optimization terminates (also emitted on interrupt/exception)

Fields

type, time, primalbound, dualbound, gap, nodes, nsol (aligned with the JSONL trace schema introduced in PR #1158)
(run_end may additionally include: status, exception, message on failure)

[Copilot]

Pull request overview

Adds a new PySCIPOpt recipe to stream structured optimization progress in real time using JSONL, enabling external tailing/monitoring while the solver runs.

Changes:

• Introduces realtime_trace_jsonl recipe with optimizeTrace() / optimizeNogilTrace() to record selected SCIP events into model.data["trace"] and optionally a JSONL file.
• Records bestsol_found, dualbound_improved, and a final run_end event with flushing intended for real-time consumption.
• Adds tests covering in-memory tracing, file output, and interrupt handling; updates changelog.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

File	Description
src/pyscipopt/recipes/realtime_trace_jsonl.py	Implements the real-time JSONL tracing recipe and event handling.
tests/test_recipe_realtime_trace_jsonl.py	Adds tests for in-memory traces, JSONL file output, and interruption behavior.
CHANGELOG.md	Documents the addition of the new recipe.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

includeEventhdlr() registers an event handler plugin permanently (there is no corresponding remove/uninclude API). Calling optimizeTrace()/optimizeNogilTrace() multiple times on the same model will attempt to include another handler with the same name (realtime_trace_jsonl), which can raise a SCIP error and/or leave multiple live handlers capturing closed file handles and old _TraceRun instances. Refactor to include the handler at most once per model (e.g., stash/reuse it in model.data), and make the handler read its current sink (trace list / file handle) from mutable attributes rather than a closure over a per-run object.

[MySweetEden]

Not Addressed

1. includeEventhdlr() multiple invocation issue

The concern about permanent handler registration is valid:

• includeEventhdlr() registers handlers permanently with no removal API
• dropEvent() only unsubscribes from events, not the handler itself

Scope: Refactoring to a handler-reuse pattern would require architectural changes and will be addressed separately. The current implementation assumes single-run usage.

[Copilot]

For a recipe marketed as “real-time JSONL streaming”, not flushing dualbound_improved events can delay visibility for external consumers tailing the file. Consider flushing here as well (or making flushing policy configurable), especially since dualbound_improved is one of the primary progress signals you record.

[MySweetEden]

Not Addressed

2. dualbound_improved flush policy

dualbound_improved events are intentionally not flushed:

• Frequency asymmetry: dualbound_improved fires hundreds to thousands of times during optimization, while bestsol_found fires only a few dozen times at most; flushing on every dual bound update would accumulate significant I/O overhead
• OS buffering suffices: Events naturally flush within seconds via OS buffering, providing adequate real-time visibility
• Context: Optimizations typically run for minutes to hours, making second-scale buffering delays negligible

Discussion: I'm open to reconsidering the flush policy if there are use cases where immediate flushing of dualbound_improved events is valuable (e.g., sub-minute monitoring). Would making it configurable be useful, or is the current approach acceptable?

[MySweetEden]

I’ll address the comments over the weekend and push updates soon.

[MySweetEden]

Addressed

1. dropEvent() refcount underflow prevention

SCIP does not provide a dedicated "solve finished" event, so we cannot rely on an event-driven shutdown callback (like eventexit()) for cleanup. Therefore, cleanup is performed in __exit__, with guards to avoid invalid dropEvent() calls.

Implementation:

• Added self._caught_events (set) to track which events were successfully caught in eventinit()
• Modified __exit__ to drop only the events that were actually caught
• This prevents incorrect dropEvent() calls if eventinit() partially fails

2. Trace data initialization

Changed from setdefault() to explicit assignment:

self.model.data["trace"] = []

Rationale: Ensures each traced run starts with a fresh list, preventing data from previous runs from mixing and keeping in-memory behavior consistent with file output (which always truncates).

3. Event handler parameter naming

Changed the first parameter from s to hdlr to improve readability while avoiding collision with the outer _TraceRun.self.
Note: hdlr is a conventional abbreviation commonly used in PySCIPOPT/SCIP codebase.

4. Exception handling and flush comments

Added explanatory comments to exception handling and flush behavior for clarity.

---

Not Addressed

1. includeEventhdlr() multiple invocation issue

The concern about permanent handler registration is valid:

• includeEventhdlr() registers handlers permanently with no removal API
• dropEvent() only unsubscribes from events, not the handler itself

Scope: Refactoring to a handler-reuse pattern would require architectural changes and will be addressed separately. The current implementation assumes single-run usage.

2. dualbound_improved flush policy

dualbound_improved events are intentionally not flushed:

• Frequency asymmetry: dualbound_improved fires hundreds to thousands of times during optimization, while bestsol_found fires only a few dozen times at most; flushing on every dual bound update would accumulate significant I/O overhead
• OS buffering suffices: Events naturally flush within seconds via OS buffering, providing adequate real-time visibility
• Context: Optimizations typically run for minutes to hours, making second-scale buffering delays negligible

Discussion: I'm open to reconsidering the flush policy if there are use cases where immediate flushing of dualbound_improved events is valuable (e.g., sub-minute monitoring). Would making it configurable be useful, or is the current approach acceptable?

[MySweetEden]

I addressed the actionable review items and all checks are green. A couple of higher-level/trade-off points are intentionally left open for discussion. Could you take another look when you have time?

[Joao-Dionisio]

Hey @MySweetEden , yes I will have a look! I will try to lay low for a little bit, for my own sake, but this should get merged, don't worry :)