# Shutting Down GNU Radio 4.0 Flowgraphs Proposal for a shutdown contract --- ## What's this? ### Purpose of this Document - Agree on *what* we want to happen to a GR4 flowgraph - Identify implementation constraints - Identify design pitfalls to avoid ### Non-Purpose of this Document Not intended to: - Specify implementation *how* in code-level detail - talk about code, instead of principles - (of course excellent if we have a "you could do it with this and that C++ feature" in our heads) - Reproduce GR3 - (written by someone with GR3 experience, but dedicatedly *not* what happens in GR3) --- ## Structure - Conventions - Fundamental Examples & Problem statement (Overview) - Takeaways - Advanced Example - Takeaways II - Proposed Mechanism --- ## Conventions - The following will contain GRC screencaps of example flow graphs - Does not imply GR3, but it's an intuitive illustration - HM Lemmy, the Lemming King, signifies the first block to be done <br><img data-src="lemmy.png" style="height: var(--r-heading1-size);" /> --- # Examples ,,, ## Source-Finished Path Flowgraph I <img src="srcend.svg" width="100%" /> Intent: - all data still in-flight at point of Src being finished must be passed to the sink, - then want the flow graph to deinit and deallocate <!-- Scenario: --> <!-- - Source knows when it will be finished --> <!-- - Information that there's no more work that can be finished passes downstream --> <!-- - When the last piece of work consumed by sink, flow graph shuts down --> <!-- - Could be dealt with with tags --> ,,, ## Source-Finished Path Flowgraph II <img src="indeterminate-srcend.svg" width="100%" /> Difference (to Source-Finished Flowgraph I): - Source cannot know when it'll be finished - Being done happens between work calls ,,, ## Sink-Finished Path Flowgraph <img src="sinkend.svg" width="100%" /> - Sink "surprisingly, but not accidentally" done - In-flight data lost ,,, ## Sink-Finished Bifurcating Flowgraph <img src="bifurcated-sinkend.svg" width="100%" /> - First sink to be finished initiates shutdown - In-flight data needs to be processed to other sink - Raises questions of order of block finishing ,,, ## Sink-Finished Multi-Depth Flowgraph <img src="bifurcated-multi-sinkend.svg" width="100%" /> Intent: - Different life expectancy for data in flight - Copy doesn't need to run anymore, red data lost - yellow (Write ptr - Copy's read ptr) irrelevant - Green data should be handed down to Vector sink --- ## Takeaways - Becoming finished is an event that needs to pass - both upstream and downstream - through the graph structure (not necessarily through the same mechanisms as items) - Which block finishes first has influence on the order in which blocks are shut down - There's usually not a known-beforehand item Nr. at which a block is finished --- ## Transceiver example **TODO FG Image** - Receiver state machine transitioning based on received data; simplest example, 4 states: - `WAIT_FOR_PREAMBLE`, `DROP_PREAMBLE`, `PASSTHROUGH`, `FINISHED` - Emitter state machine: - accepts data from async port, inserts filler samples when timeout - `WAIT_FOR_DATA` (produce nothing), `INSERT_FILLER` (produce filler), `INSERT` (produce payload) - Receiver → Transmitter async info on finishing up Tricky! - Source feeding data into the transmitter block can be done - Let the transmitter finish its current filler - Sink accepting data from the receiver block can be done - Allow the receiver to async inform the transmitter block to shut down --- ## Takeaways II - Blocks that wait for async messages to produce data while blocking need to be interruptible by done - Blocks that wait for IO need to have a signaller to subscribe to to interrupt their waiting - Blocks without fixed in/output relationship map badly to "upstream's done, so this block must be done" - same for downstream being done --- ## Proposed Mechanism ### Doneness - Done-ness is a hierarchy: - A flow graph is done when all blocks are done - A block is done when all its ports are done and it's finished its current processing - A stream input port is done - when the block itself declares it done, or - when the scheduler declares it done after the upstream block's corresponding port's done, and the items on the buffer gone - An async input port is done - when the block itself declares it done, or - when *all* the upstream blocks declare it done - An output port is done - when the block itself declares it done, or - when all the downstream blocks declare it done --- ## Proposed Mechanism ### Message Propagation - Upon being informed by a block about a done port, the scheduler informs: - Downstream direction (for output port) - Scheduler notes the remaining number of consumable in-flight items - sets downstream (input) port done after they've been consumed - Upstream direction (for input port) - No non-done other input ports on the same output: - set connected output port to done - Bubble "finish ASAP" information up the graph - don't inform block unless specifically requested, change port state - Further active input ports on the same output: - Bubble "finish ASAP" information up the graph - don't inform block unless specifically requested, change port state - Upon "finish ASAP" reaching a source - let source finish producing, mark output done --- ## Proposed Mechanism ### Handling state changes - Fixed-rate blocks: - done input port → automatic done output port - no need for custom in-block logic - Variable-rate blocks: - handle information about input being done themselves - might still produce with all done input ports - Scheduler keeps track of port, block states - Remove from scheduling algorithm as block becomes done - Deinits blocks as soon as all ports done - When all blocks done, deallocate all blocks, flowgraph - Deallocation order unimportant --- ## Proposed Contract ### Block Contract 1. When a block notices it cannot produce any more items on an output, it *must* notify the scheduler about the affected output ports 1. When a block notices it cannot consume any more items on an input, it *must* notify the scheduler about the affected input ports 1. A block *must* not produce any more items on a done output port 1. A block *must* not consume any more items from a done input port 1. A block that blocks waiting for an external event during its `work()` *must* register a callback that either cancels the blocking, or indicate that all ports should be considered done, the `work` still being blocked nonwithstanding 1. A block which might produce items without consuming every `work()` call *shall* register callbacks that handle port state changes 1. In these callbacks, the block *shall* modify its port states accordingly, or defer the state change to the appropriate time during `work` 1. A block *can* register a function to get notified about done state changes on adjacent ports, and about "finish ASAP" state changes --- ## Proposed Contract ### Scheduler Contract 1. *shall* immediately upon observing the last input port on a buffer becoming done set the feeding output port(s) to "finish ASAP" 2. *must* upon observing a output port becoming done 1. Determine the connected downstream block(s), and for each of these 2. if this is the last output port connected to the input, wait for their readers to catch up as far as multiplicity/alignment allows to the write pointer's final position 3. set the port to done, or call the registered handler 3. *shall* upon seeing a "finish ASAP" state on a block output, 1. Check for the presence of a registered handler for that, 2. Call the handler if present or propagate this state to the block's upstream neighbors' output ports 4. …*shall* remove blocks from its internal scheduling data structures as their block states indicate non-schedulability 5. …*shall* deinitialize blocks whose every port has been marked done and who are not currently executing work 6. …*shall* deallocate all blocks as soon as all blocks have been deinitialized