Compose, Don’t Fork: Wiring Azure Integration Connectors with YAML

A new partner should not mean a new Function App repo. I treat enterprise integration patterns as a catalog of connectors and a YAML list that names them. The C# already lives under functions/. The partner file only says which blocks to stand up, in which environments, with which queue and container names.

That is the whole job in one sentence: compose an integration from existing Azure Function connectors; do not fork the connectors.

Think of a kitchen that already owns a grill, a slicer, and a pass. A new lunch special is a ticket on the rail, not a second kitchen built in the car park. If the ticket asks for a smoker you do not have, you add one smoker to the line — you do not clone the restaurant.

This is part 7, the capstone of the Azure integration series. The catalog is in microservices integration patterns on Azure Functions. Deep dives: facade, SFTP publish / sender / delivery, PGP at the pipeline boundary, transformation + API/webhook egress, and subscription + sequence. Names and behaviour come from the public repo github.com/fransiscuss/integration-azure (commit 2539242).

What enterprise integration patterns look like as a YAML list

Hohpe and Woolf named the moves: content-based router, claim check, message translator, sequencer, request/reply. I do not re-implement those moves per trading partner. Each move is one folder under functions/ with a connector.yaml contract. An integration is the wiring.

The committed composition file is tiny on purpose. This is the whole of integrations/sample-facade.yaml:

name: sample-facade
publisher: sample
subscriber: platform
env:
  - dev
flow:
  - connector: facade

That is a real file, not a sketch. Four keys plus a one-item flow:. Terraform in infra/terraform/envs/dev/main.tf does yamldecode(file(...)) and passes the map into modules/integration. If env does not contain the environment you are planning, flow becomes an empty list and nothing deploys.

Field	What it actually does
name	Goes into the Function App name: func-{name}-{connector}-{env}
publisher / subscriber	Labels on the spec; not C# namespaces
env	Allow-list. contains(spec.env, var.environment) gates deploy
flow[].connector	Must match a catalog name: facade, publish, subscription, preprocessing, decompress, transformation, compress, postprocessing, sequence, delivery, sender, apidelivery, webhookdelivery
flow[].settings	Optional map. Queue, container, and ledger names — looked up with defaults

I call this config driven microservices because behaviour change is a blob of YAML plus app settings, not a branch of FacadeFunction.cs.

How Terraform turns that list into Function Apps

modules/integration/main.tf is a switchboard, not a unique snowflake per partner. Each catalog name has a module "<connector>" { for_each = { for idx, f in local.flow : idx => f if f.connector == "<connector>" } } block. The source is always functions/<connector>/infra. Shared spine comes from modules/platform: resource group, Flex Consumption plan (FC1, Linux), storage account plus config container, Service Bus Standard namespace, serverless Cosmos DB, Log Analytics + Application Insights, Key Vault with RBAC.

locals {
  spec = yamldecode(file("${path.module}/../../../../integrations/sample-facade.yaml"))
}

module "platform" {
  source             = "../../modules/platform"
  environment        = var.environment
  location           = var.location
  name_prefix        = var.name_prefix
  deployer_object_id = var.deployer_object_id
}

module "integration" {
  source      = "../../modules/integration"
  spec        = local.spec
  environment = var.environment
  platform    = module.platform
  artifacts = {
    facade = var.facade_artifact_url
  }
}

That is infra/terraform/envs/dev/main.tf with comments stripped. envs/tst and envs/prd yamldecode the same sample file, but they instantiate module "facade" from functions/facade/infra directly — no integration module, no for_each over flow. And the sample’s env: list is only dev, so even the integration module would skip deploy outside dev. Composition is the intended path; the tst/prd roots are still a facade shortcut. Artifacts are a map of connector name to WEBSITE_RUN_FROM_PACKAGE URL. If you add publish to flow and forget the zip, the integration module still interpolates a placeholder URL — the Function App exists, the code does not. I treat missing artifact keys as a pipeline bug, not an implicit “it will be fine”.

docs/AUTHORING-A-CONNECTOR.md is blunt about the gate I want on every PR:

Adding the connector to a running pipeline is a config change, not a code change: add a flow: entry in the relevant integrations/*.yaml, or create a new one. terraform plan against infra/terraform/envs/<env> should show only new resources — never a diff inside functions/ or shared/.

Move	Compose (what I want)	Fork (what I refuse)
New partner	New integrations/*.yaml	Copy functions/publish to functions/publish-acme
New environment	Add tst / prd to env:	Duplicate the Function project
Queue names	flow[].settings.input_queue_name	Hard-coded strings in C#
Shared behaviour	shared/Integration.Core or Integration.Connectors.*	Paste a helper into two connectors
New capability	scripts/new-connector.sh name template then connector.yaml	If/else inside an existing Function

A partner SFTP path without new C#

The repo currently ships one integration file — the facade sample above. The schema is the same if the partner drops zipped invoices on SFTP. I would add a second YAML next to it, not a second codebase. Teaching composition using the real flow[].connector names and the real settings keys from modules/integration/main.tf:

name: acme-invoices
publisher: acme-erp
subscriber: platform
env:
  - dev
  - tst
  - prd
flow:
  - connector: publish
    settings:
      lock_container_name: publish-locks
  - connector: subscription
    settings:
      input_queue_name: subscription-in
      ledger_container_name: pipeline
  - connector: preprocessing
    settings:
      input_queue_name: preprocessing-in
      output_queue_name: preprocessing-out
      input_container_name: inbound
      output_container_name: cleartext
  - connector: transformation
    settings:
      input_queue_name: transformation-in
  - connector: sequence
    settings:
      input_queue_name: sequence-in
      output_queue_name: sequence-out
      ledger_container_name: pipeline
  - connector: delivery
    settings:
      input_queue_name: delivery-in
      ledger_container_name: pipeline

In dev that names apps such as func-acme-invoices-publish-dev and func-acme-invoices-delivery-dev. Defaults in the module already match most of those queue names, so the settings maps are explicit documentation more than magic. Two honest gaps in the composition layer today:

• route_queue_ids on subscription is empty unless you pass it. The module README says to wire it once downstream queue outputs exist. Until then the router has nowhere to send.
• Per-connector behaviour — SFTP host, filePattern, PGP Key Vault refs, transformer name CsvToJson — still lives in each connector’s own config blob (see functions/publish/config/publish.sample.yaml and functions/subscription/config/subscription.sample.yaml). The integration YAML deploys the apps. It does not replace those files.

That split is the point. Hohpe’s translator is still transformation. The partner’s CSV layout is still a config document on that connector, not a new microservice. Listing connectors in YAML is not the same as wiring their queues; flow: deploys apps. These six names are how I would encode those enterprise integration patterns for an SFTP partner — still not a file the repo ships.

When you actually write code

You write code when the catalog is missing a move. scripts/new-connector.sh sms-delivery webhookdelivery copies the folder shape from a similar trigger. Then you write connector.yaml first — trigger, PipelineEnvelope in/out, infrastructure required — because the contract check fails a connector whose README table does not match requires:, or whose anatomy files are missing. Shared behaviour goes in shared/. Two connectors that both talk SFTP already reference Integration.Connectors.Sftp; they do not copy the client.

If the PR diff is inside functions/publish because Acme wants a different archive folder, I bounce it. That is a publish.sample.yaml change, or a second config blob, not a fork.

Azure Integration Services (Logic Apps, Service Bus, Event Grid, API Management) still sit around this. I use them as the bus and the edge. I do not use them as the place I hide partner-specific C#. This azure integration architecture is YAML composition on a shared platform, not a Logic App per partner. An integration architect review should be able to point at three files — the YAML, the platform module, one connector README — and know where a message dies.

FAQ

Is this just Logic Apps with extra steps?
No. Logic Apps is a hosted designer. This is thirteen independently deployable .NET 10 isolated Function Apps on Flex Consumption, composed by Terraform. You get a real connector.yaml contract, unit tests next to the Function, and a ledger in Cosmos. You also own the failure modes I already wrote down for peek-lock and sequence requeue.

Why is integrations/ almost empty?
Because composition is supposed to be boring. The sample is sample-facade.yaml. Partner files are data. I would rather show the real sample plus the real module keys than invent a production YAML the repo does not ship.

Can I put two of the same connector in one flow?
for_each is keyed by list index, but the Function App name is func-{name}-publish-{env} with no index. Two publish rows would collide on the Azure name. I have not shipped that as a sample. Two YAML files (two integration names) is the operations story.

Where do secrets go?
Not in integrations/*.yaml. Connector samples use Key Vault references such as @Microsoft.KeyVault(SecretUri=...) on SFTP keys. The platform module creates the vault. Managed identity plus RBAC is the path; connection strings in YAML are not.

What if terraform plan touches functions/?
Stop. That PR is a fork wearing a composition costume. Fix the config or add a catalog connector. Do not “just this once” patch Acme into PublishFunction.cs.