Function takeOutgoingRequests

Drains every outstanding request this library needs the product to send to its homeserver, or feed to another device -- design doc section 3bis. An addition to the frozen surface, not a change to it: OlmMachine has an outbound side (device/one-time key uploads, key queries, key claims, and the to-device requests that actually carry a shared session key), and discarding what this returns is the mistake section 3bis is named for -- a machine that encrypts to nobody and never learns that any of it happened.

The returned order is significant, and you must preserve it. Send the requests in the order this returns them -- not "start them in that order and let them race": each one has to reach your homeserver before the next is sent, because the server relays them to the other device in the order it receives them.

That is a real constraint with exactly one source, and it is worth naming so it is not optimised away. A verification flow's last two messages are a confirmation and the acknowledgement that closes the flow, and the far side silently discards an acknowledgement that arrives before the confirmation it acknowledges. It then waits for one that has already been sent. The failure is asymmetric -- your side completes and records the other device as verified, the other side records nothing -- and neither side is told. Both messages can land in the same batch, from two different queues inside the library, so a product that pumps on a timer rather than after every call is the one that meets this.

Resolving them with markRequestSent is a different matter and is not ordered at all. It is a lookup by id, so mark them in whatever order the responses come back, and do not wait for request n to be marked before sending request n+1.

Requests from different batches were never orderable against each other -- a batch is a snapshot -- and nothing here changes that. What this function guarantees is that within one batch, the order it returns is the order the requests were produced in, across both of the places inside the library they come from.

What that guarantee is worth, stated so it is not read as more. Two requests this library produced in an order that matters come out in it, which is the whole point and is what the verification pair needs. Two requests with no ordering requirement between them may come out in either order, run to run, and nothing here promises otherwise -- the last paragraph of this comment is a measured example of exactly that. So: preserve the order you are given, and do not read meaning into the relative position of two requests that have none.

Up to and including 0.1.0-rc.2 this comment said the opposite: that the array was an unordered set and a product must not infer sequencing from position. That was true of every request the library could then produce, and it stopped being true when device verification arrived. The sentence is recorded here rather than deleted because a consumer who read the old one and built on it has to be able to find out that it changed.

markRequestSent is not the only thing that ends a request's life. A later call to this function ends some of them too. Four of the kinds handed out here -- keys_upload, keys_query, keys_claim and signing_keys_upload -- are evicted the moment a subsequent call hands out a fresh request of the same kind, whether or not the older one was ever marked sent. markRequestSent then rejects that older id with unknown_request.

That is designed, not a defect, and it is worth knowing why, because unknown_request for an id a product is legitimately holding otherwise reads as a library bug. The first three describe a standing need ("these keys want uploading", "these users want querying") rather than one message. matrix-sdk-crypto re-derives that need from current state on every call, mints a new and uncorrelated id for it, and forgets the id it handed out last. So once a fresh one exists, the older id names nothing the machine is still waiting to hear about, and the fresh request in that same batch carries what the older one was for.

signing_keys_upload is in that group for a different reason and on a narrower trigger, and it is the one that will actually catch a product out. Nothing upstream forgets its id; this library re-derives it, and only when bootstrapCrossSigning is called again. A second bootstrap publishes the identical three keys, so keeping both entries would hand a caller two ids for one publication and two rounds of user-interactive authentication to finish it. An ordinary second drain does not touch it, because no fresh one exists to evict it; a second bootstrap followed by a drain does. That matters because this is the one id a product is meant to hold across a slow loop with a person in the middle of it: it survives any number of refused attempts, since only success consumes an entry, and it does not survive being superseded. Do not call bootstrapCrossSigning again while an authentication loop is in flight.

What a caller must do about it: resolve a batch before drawing the next. Drain, send in order, markRequestSent each response, and only then call this again.

Within one batch, marking may overlap sending -- nothing in one batch evicts another member of it, so request n need not be marked before request n+1 is sent. The sends themselves stay ordered, which is the half of this that changed after 0.1.0-rc.2; see the ordering rule at the top of this comment. This paragraph used to say sending and marking within a single batch were both safe to do concurrently, which is the sentence that section retracts.

What is not safe is a second drain overlapping unresolved requests from an earlier one: two pumps racing, or a drain on a timer alongside a drain after a write, will produce unknown_request for ids the product still holds.

On unknown_request for an id from an earlier batch, do not retry it. Discard the response that id was going to carry and pump again. Nothing is lost: the need was re-derived rather than dropped, and the request that supersedes it is either already in hand or waiting in the next drain.

to_device, signature_upload and room_message ids are never evicted this way, and two keys_query ids escape it as well. The first three each name one independently deliverable message, so each stays outstanding until markRequestSent resolves it. The other two are standing needs that have to outlive an ordinary drain: the out-of-band query about this account, which only another query of its own kind evicts, and the query a verification finishing by a scanned code queues, which nothing evicts at all, because its answer is the entire product of that verification. Both arrive as keys_query and nothing on this surface tells them apart from an ordinary one, so a product can hold two live keys_query ids at once: a newer one is not evidence that an older one is dead. For every kind, marking is not optional bookkeeping; it is what advances the underlying state machine. A product that calls this but never calls markRequestSent keeps being handed the same requests, including -- for a to-device request the machine could not yet deliver -- a stale m.room_key.withheld notice sitting alongside the actual session key, in no reliable order relative to it (measured across ten runs of the same sequence: six with the notice first, four with the key first). That is not a counter-example to the ordering rule above and is the reason it is scoped as it is: neither of those two requests is order-significant against the other, they are held keyed by transaction id rather than by production order, and a transaction id is random. The measured harm from that specific case is bounded -- that withheld notice carries no scope and no session id of its own, so it names nothing for a recipient to act on, and a matrix-sdk-crypto-based recipient's own add_withheld_info deliberately ignores exactly this notice kind -- but relying on that is not a substitute for calling markRequestSent: it is the only thing that stops the duplication at the source.