Route and Order Without Sleeping on the Lock: subscription + sequence

Partners still drop files into named folders. Downstream still wants decompress, transform, compress, or encrypt. I do not encode that as a C# if/else chain per partner. I treat azure service bus routing as a YAML table: parse the blob subject, take the first matching when/then, write Received on the Cosmos ledger, then dispatch. Order is a different job. sequence only releases the oldest non-terminal row for that interface, and it never sleeps on the peek-lock.

That is the whole job in one sentence: route by config, then wait by scheduling a copy — not by holding the Service Bus lock.

Think of a campus mailroom next to a boarding gate. The mailroom reads the path on the envelope and drops it in the first matching pigeonhole. Clubs do not fork the mailroom because ERP invoices are zipped and the warehouse feed is already JSON. The boarding gate is stricter. Group 2 does not stand in the jetway for thirty minutes hoping group 1 finishes. They take a numbered ticket, sit down, and come back when the previous group has actually boarded.

This is part 6 of the Azure integration series. The catalog lives in microservices integration patterns on Azure Functions. Earlier deep dives: facade, SFTP publish / sender / delivery, PGP at the pipeline boundary, and transformation + API/webhook egress. Names and behaviour come from the public repo github.com/fransiscuss/integration-azure (commit 2539242).

What azure service bus routing looks like as a table

subscription is the pipeline router. The trigger is %INPUT_QUEUE_NAME% on SERVICE_BUS_NAMESPACE. The body is a PipelineEnvelope v1. Happy path: rebuild the Event Grid subject from Source, parse named groups, evaluate the branch table, upsert the ledger, dispatch. then is an app-setting name. The actual queue is resolved at dispatch time, so topology stays a deployment concern.

The grammar is tiny on purpose: path != null, path == null, path == 'literal'. There is no != 'literal'. That would read as set-membership, which this table is not. First match wins. An entry without when is the otherwise fallback and must be last. No match and no otherwise is a dead-letter, never a silent drop.

# functions/subscription/config/subscription.sample.yaml
subjectPattern: "^/blobServices/default/containers/(?<container>[^/]+)/blobs/(?<publisher>[^/]+)/(?<interface>[^/]+)/(?<file>.+)$"

routes:
  - when: "input.compressed != null"
    then: DECOMPRESS_QUEUE_NAME
  - when: "transform != null"
    then: TRANSFORM_QUEUE_NAME
  - when: "output.compressed != null"
    then: COMPRESS_QUEUE_NAME
  - then: POSTPROCESSING_QUEUE_NAME

subscribers:
  erp:
    input:
      compressed: invoices/temporary
    transform:
      name: CsvToJson
      directory: invoices/transformed
  data-warehouse: {}

Predicate context is the subscriber’s free-form YAML, flattened so input.compressed is a path, with the subject’s named groups on top. Groups win name collisions. In the sample, erp has input.compressed set, so those events hit decompress first. data-warehouse is an empty map, so it falls through to the otherwise queue.

Connector	Trigger setting	Job	Settlement
subscription	INPUT_QUEUE_NAME	Parse + route + ledger Received	Host completes on Dispatched
sequence	INPUT_QUEUE	Oldest non-terminal may pass	AutoCompleteMessages = false

That is still the same enterprise integration patterns as facade and SFTP: shared Function code, per-partner YAML. You do not fork C# because partner B’s invoices are zipped.

subscription: parse the subject, write Received, then dispatch

The source repo parsed subjects with positional splits. A dash in a folder name broke routing silently. Here the subject is rebuilt into the canonical Event Grid shape from envelope.Source before the regex runs, so the pattern always sees /blobServices/default/containers/<container>/blobs/<path>. The four well-known groups are container, publisher, interface, file. Extra named groups become FileNameTokens on the outbound envelope.

// functions/subscription/Routing/SubscriptionRouter.cs
var subject = SubjectParser.BuildSubject(envelope.Source);
var parsed = SubjectParser.TryParse(subject, config.SubjectPattern);
if (parsed is null)
    return new RoutingOutcome.Rejected(DeadLetterReasons.UnparseableSubject, ...);

var then = RouteTableEvaluator.Evaluate(config.Routes, context);
var queueName = configuration[then]; // app setting, not a hard-coded queue

await UpsertLedgerAsync(envelope, parsed, ct); // before dispatch
await dispatcher.DispatchAsync(queueName, outbound, ct);

The ledger write is first on purpose. Id is {publisher}:{interface}:{file}, partition key is the interface. Status is Received. TryCreateAsync returning null means the row already exists — treat the delivery as a resume after a crash between write and send, and still dispatch. Duplicate detection on the next queue uses MessageId = envelope.EventId, same instinct as my Service Bus peek-lock guide.

subscription does not call CompleteMessageAsync itself. A Dispatched outcome lets the Functions host complete the trigger message. Rejected outcomes dead-letter with a reason. Transient Cosmos or Service Bus faults throw, and the Function abandons for redelivery up to maxDeliveryCount: 5.

Poison reason	When
InvalidEnvelope	Body is not a PipelineEnvelope
UnparseableSubject	Rebuilt subject misses the regex or a well-known group
UnknownSubscriber	Envelope names a subscriber that is not under subscribers:
NoRoute	No when matched and there is no otherwise
RouteNotConfigured	then names an empty app setting

RBAC is boring and required: Service Bus Data Receiver on the input, Data Sender on each named output, Cosmos data role on the ledger, Storage Blob Data Reader on the config container. Duplicate detection on, 10-minute window, DLQ enabled. Same spine as the rest of the catalog.

sequence: oldest non-terminal wins

sequence sits in front of sequenced delivery. An envelope may pass only when it is the oldest non-terminal ledger record for its interface — every predecessor has reached Delivered or Failed. Trigger is %INPUT_QUEUE% with AutoCompleteMessages = false. Output is Sequence:OutputQueue (sample: delivery-in). Default RetryDelay is 30 seconds.

// functions/sequence/Gate/SequenceGate.cs
record = await ledger.FindAsync(ledgerId, partition, ct);
oldest = record is null ? null : await ledger.FindOldestPendingAsync(envelope.InterfaceName, ct);

if (record is null)
    return GateResult.DeadLetter("LedgerRecordMissing", ...);

if (record.Status is LedgerStatus.Delivered or LedgerStatus.Failed)
    return GateResult.Release; // redelivery of already-released work

return oldest?.Id == record.Id ? GateResult.Release : GateResult.Hold;

The Function maps that verdict onto settlement. Release sends the unchanged envelope downstream with MessageId = EventId, then completes. Hold schedules a delayed copy on the input queue, then completes. Abandon is for transient Cosmos/Service Bus. Dead-letter is for InvalidEnvelope and LedgerRecordMissing.

That last reason is the coupling: subscription must write the ledger before anything reaches sequence. A missing row is poison, not “create it here.” The ids are not the same string today. subscription stores {publisher}:{interface}:{file} where file is the named group (?<file>.+) — nested blob paths keep their slashes. sequence looks up the last segment of Output ?? Source. A path like invoices/2026.csv is Received as that full relative path and queried as 2026.csv, which dead-letters LedgerRecordMissing. Flat blob names work. Nested ones do not until those two sides share one id helper.

Oldest is a Cosmos query: same partition, status not Delivered or Failed, ORDER BY c.fileTimestamp ASC, composite index (interface, fileTimestamp). Honest gap: nothing in this repo assigns FileTimestamp when subscription creates the row — Cosmos only round-trips the property. The gate is still “oldest pending by that field.” Stamp it in UpsertLedgerAsync if you need file-time order. YAML cannot set it.

Don’t sleep on the lock

The source repo waited in-process with retryDelay() while the peek-lock was held. Lock duration is finite. When processing outruns the lock, the broker redelivers and you get a pile-up of workers all “waiting” on the same file. Microsoft’s own guidance is the same: peek-lock, settle explicitly, keep work shorter than the lock, make processing idempotent, and use duplicate detection for send-side retries. See message transfers, locks, and settlement.

// shared/Integration.Core/Messaging/ScheduledRequeue.cs
var copy = new ServiceBusMessage(BinaryData.FromObjectAsJson(envelope))
{
    MessageId = $"{envelope.EventId}:r{deliveryCount}",
    CorrelationId = envelope.CorrelationId,
    ContentType = "application/json",
};
await sender.ScheduleMessageAsync(copy, DateTimeOffset.UtcNow.Add(delay), ct);

The :r{deliveryCount} suffix is not decoration. Duplicate detection includes scheduled messages. If the copy reused the bare EventId, the broker would accept the send and drop the copy because the original — which you just completed — still sits in the detection window. The suffix makes the wait a new message. Metrics split the same way: messages.processed is a release, messages.requeued is a hold, and queue depth on the input is the out-of-order signal.

# functions/sequence/config/sequence.sample.yaml
INPUT_QUEUE: sequence-in
Sequence:
  OutputQueue: delivery-in
  RetryDelay: "00:00:30"

Use sequence when the partner’s SFTP dock is order-sensitive — the same reason delivery exists next to unsequenced sender. Skip it when any order is fine. Do not put a Thread.Sleep in a Service Bus Function and call it an ordering gate.

FAQ: azure service bus routing and ordered delivery

What is azure service bus routing in this catalog?

A declarative when/then table on the subscription connector. The Function parses a rebuilt Event Grid subject, picks the first matching route, writes a Cosmos Received row, and sends the envelope to the queue named by that route’s app setting.

Why not Service Bus topics and SQL filters?

You can. I still want one pipeline envelope, one ledger row, and predicates that read partner YAML (input.compressed, transformer name) rather than broker filter syntax. Topics are a valid bus. They are not the partner config.

Does sequence use sessions?

No. Order is a Cosmos query per interface, not a Service Bus session. Sessions would serialize the whole interface on one receiver. The gate lets other interfaces keep moving.

Why suffix MessageId on a hold?

Scheduled copies participate in duplicate detection. A copy with the same MessageId as the completed original is dropped. {EventId}:r{deliveryCount} is a new id for a new wait.

What if the ledger row is missing?

sequence dead-letters LedgerRecordMissing. Routing is supposed to have created it. Do not invent a row at the gate.

Where does this sit in the series?

After transform and API/webhook egress. Next is composing a full partner flow from integrations/*.yaml without forking connector code.