dplutils.pipeline.stream.StreamingGraphExecutor.task_submittable

abstractmethod StreamingGraphExecutor.task_submittable(task: PipelineTask, rank: int) → bool

Preflight check if task can be submitted

Return True if current conditions enable the task to be submitted. The rank argument is an indicator of relative importance, and is incremented whenever the pending data for a given tasks meets the batching requirements as driver walks the task graph backward. Thus Rank=0 represents the task furthest along and so the highest priority for submission.