solprop.driver.executors.pool#

Run bounded spawned workers with explicit threads and memory admission control.

Functions

peak_rss()

Return this process's peak resident memory in bytes.

Classes

`ControlledWork`(args, *kwargs)	Point work a pool can stop and watch (SEP 41, S4 and S17).
`MemoryBudget`(limit, controller, per_worker, ...)	Record the admission calculation without treating it as a hard RSS limit.
`PlaneScalars`(n_steps, n_rejected, max_norm, ...)	The four scalars of a plane's summary, as a worker sends them to its controller.
`Pool`([workers, memory_limit, threads, pin, ...])	Execute independent points in memory-sized spawned processes.

class solprop.driver.executors.pool.ControlledWork(*args, **kwargs)[source]#

Bases: Protocol

Point work a pool can stop and watch (SEP 41, S4 and S17).

Optional, beside the probe and failure hooks: a pool runs any callable, and only work with these members is stopped through its worker’s pipe and has its run’s progress sent back. The scan’s own work implements it.

dispatched(point)[source]#

Record in the controller that a point was sent to a worker.

Parameters:

pointobject: The point description.

relay(point, call, args)[source]#

Make, in the controller, a reporter call a worker sent for a point.

Parameters:

pointobject: The point description the worker is running.
callstr: The reporter method: summary, started, plane, finished or interrupted.
argstuple: Its arguments, as _WorkerReporter sends them.

run_point(point, stop, reporter)[source]#

Run one point in a worker under a stop flag and reporter of the worker’s.

Parameters:

pointobject: The point description.
stopStopFlag: Set when the controller asks the worker to stop.
reporterProgressReporter or None: Sends the run’s calls to the controller; None when not watched.

Returns:

object: The point’s result.

property stop: StopFlag | None[source]#

Return the caller’s stop flag, read in the controlling process.

Returns:

StopFlag or None: The flag; None never stops.

property watched: bool[source]#

Return whether a worker sends its point’s progress to the controller.

Returns:

bool: True when the controller has a reporter for points.

class solprop.driver.executors.pool.MemoryBudget(limit, controller, per_worker, cores, threads)[source]#

Bases: object

Record the admission calculation without treating it as a hard RSS limit.

Attributes:

limitint: Effective memory allocation, bytes, positive.
controllerint: Controller resident memory and output allowance, bytes, non-negative.
per_workerint: Worker estimate, bytes, positive; backend RSS is counted once.
coresint: CPUs available to this controller, positive.
threadsint: Threads per worker, positive.

controller: int#

cores: int#

limit: int#

per_worker: int#

threads: int#

workers(requested=None)[source]#

Choose a count within both CPU and memory allocations.

Parameters:

requestedint or None, optional: Optional upper bound on workers, positive.

Returns:

int: Admitted workers after reserving 20 percent of memory.

Raises:

DriverError: If inputs are invalid or one worker cannot fit.

class solprop.driver.executors.pool.PlaneScalars(n_steps, n_rejected, max_norm, wall_time)[source]#

Bases: object

The four scalars of a plane’s summary, as a worker sends them to its controller.

Satisfies solprop.log.PlaneSummary; a frozen copy rather than the backend’s own summary so that what crosses the pipe is small and always pickles.

Attributes:

n_stepsint: Steps taken to reach the plane, accepted and rejected together.
n_rejectedint: Rejected steps among them.
max_normfloat: Largest scaled error norm over the accepted steps.
wall_timefloat: Seconds the plane took, s.

Examples

>>> from solprop.log import PlaneSummary
>>> isinstance(PlaneScalars(8, 0, 0.4, 0.1), PlaneSummary)
True

max_norm: float#

n_rejected: int#

n_steps: int#

classmethod of(summary)[source]#

Copy the scalars a reporter reads from a plane’s summary.

Parameters:

summarysolprop.log.PlaneSummary: The summary the run handed its reporter.

Returns:

PlaneScalars: Its picklable copy.

wall_time: float#

class solprop.driver.executors.pool.Pool(workers=None, memory_limit=None, threads=1, pin=True, fresh=False)[source]#

Bases: object

Execute independent points in memory-sized spawned processes.

Attributes:

workersint or None: Positive maximum workers; None uses the CPU and memory admission rule.
memory_limitint or None: Positive memory cap, bytes; None reads the allocation and physical memory.
threadsint: Threads per worker, positive and one by default.
pinbool: Pin workers within Linux CPU affinity; unavailable on macOS.
freshbool: Start a fresh interpreter for every point instead of persistent workers.
budgetMemoryBudget or None: Last measured admission calculation, absent before execution.

budget: MemoryBudget | None = None#

fresh: bool = False#

memory_limit: int | None = None#

pin: bool = True#

run(points, work)[source]#

Probe memory and run bounded work, yielding results as they finish.

Work that implements ControlledWork is stopped and watched through each worker’s pipe (SEP 41, S4 and S17). Once its caller’s flag is set, the pool sends every busy worker the stop message, which that worker’s run polls before every trial step, collects their results, dispatches nothing more and shuts the workers down; the abrupt terminate of a worker that does not exit stays the last resort. While a watched point runs, its worker sends the run’s reporter calls, which the pool hands to ControlledWork.relay().

Parameters:

pointsiterable: Independent point descriptions with global indices.
workcallable: Pickleable worker operation; an optional probe reports RSS and ledger.

Yields:

object: Results in completion order, each retaining its input identity.

Raises:

DriverError: If resource admission fails or a worker exits abnormally.

threads: int = 1#

validate(work)[source]#

Reject unpickleable builders before creating output or claiming work.

Parameters:

workcallable: Worker operation, including its builder and reducers.

Raises:

DriverError: If standard-library pickle cannot transfer the callable.

workers: int | None = None#

solprop.driver.executors.pool.peak_rss()[source]#

Return this process’s peak resident memory in bytes.

Returns:

int: Peak RSS, converting Linux KiB and retaining Darwin bytes; on Windows the peak working set.