PGP Encryption on Azure Functions: Decrypt In, Encrypt Out
Partners still send OpenPGP files. Downstream still wants JSON, zips, and an SFTP drop. I do not hide that work inside one “crypto Function.” I keep pgp encryption azure at two pipeline edges: preprocessing decrypts inbound blobs when config says so; postprocessing encrypts outbound blobs for one recipient, then marks the Cosmos ledger Ready. Sequenced delivery waits on that state. Unsequenced sender does not — it only skips work already marked Delivered.
That is the whole job in one sentence: stream PGP at the blob boundary, pull keys from Key Vault references, and keep cleartext on private storage inside Azure.
Think of an airport. Encrypted bags stay locked until they hit inbound screening (preprocessing). Inside the terminal (transform, compress) they are open. At the outbound gate (postprocessing) they get locked for the destination airline. You do not walk the concourse with someone else’s private key in your pocket — the Function only reads an app setting that Azure already resolved from Key Vault.
This is part 4 of the Azure integration series. The catalog lives in microservices integration patterns on Azure Functions; inbound/outbound files are in SFTP publish, sender, and sequenced delivery. Names and behaviour come from the public repo github.com/fransiscuss/integration-azure (commit 2539242).
What pgp encryption azure means on this pipeline
I treat OpenPGP as a transport concern, not a business process. Connector code stays generic. Config owns queues, containers, and which key setting to read. That is the same azure integration architecture as the rest of the catalog: shared packages, per-partner YAML, managed identity. You do not fork C# because partner B uses a different key.
| Connector | Trigger | Crypto | Key material | Ledger? |
|---|---|---|---|---|
preprocessing |
Service Bus queue | Optional decrypt | Private key (+ optional passphrase) via named app settings | No |
postprocessing |
Service Bus queue | Always encrypt | Recipient public key app setting | Yes — Ready |
One deployed postprocessing instance is one recipient / key / output combination. Adding another partner is config, not a new project. One preprocessing instance covers both “this file is encrypted” and “this file is already clear” because DecryptionRequired defaults to false.
Inbound: decrypt only when config says so
The trigger is %Preprocessing:InputQueue% on the SERVICE_BUS_NAMESPACE managed-identity binding. The body is a PipelineEnvelope v1. Source points at the (possibly encrypted) blob. After work, Output points at cleartext — or the envelope is forwarded unchanged on pass-through. Outbound MessageId is envelope.EventId so duplicate detection on the next queue absorbs redelivery-after-send.
The interesting bit is not the Function glue. It is the pure decider, which I can unit-test without Azurite:
// functions/preprocessing/Processing/PreprocessingDecider.cs
if (!string.Equals(envelope.Source.Container, options.InputContainer, StringComparison.OrdinalIgnoreCase))
return DeadLetter("UnexpectedSourceContainer", "...");
if (!options.DecryptionRequired)
return PassThrough;
return string.IsNullOrWhiteSpace(privateKey)
? DeadLetter("MissingDecryptionKey", "...")
: Decrypt;
Wrong container means the message landed on the wrong instance’s queue. Missing key when decrypt is required is poison, not a retry. Transient storage or Service Bus faults are abandoned so the host redelivers up to maxDeliveryCount — same peek-lock instinct as my Service Bus peek-lock guide. The lock is not held across a sleep.
When decrypt runs, cleartext is not buffered as a byte[]:
// functions/preprocessing/Functions/PreprocessingFunction.cs
await using var cipherText = await sourceBlob.OpenReadAsync(...);
await using (var clearText = await targetBlob.OpenWriteAsync(overwrite: true, ...))
{
await pgp.DecryptAsync(cipherText, clearText, privateKey, passphrase, ct);
}
return envelope.WithOutput(output);
Output keeps the same storage account and the same path. Only the container changes (inbound → cleartext in the sample). That is a boring rule, and boring is what you want when the next connector looks up the file.
# functions/preprocessing/config/preprocessing.sample.yaml
Preprocessing:
InputQueue: preprocessing-in
OutputQueue: subscription-in
InputContainer: inbound
OutputContainer: cleartext
DecryptionRequired: true
PrivateKeySettingName: PGP_PRIVATE_KEY
PassphraseSettingName: PGP_KEY_PASSPHRASE
A per-integration config blob on PipelineEnvelope.ConfigBlobUri may override everything except InputQueue. The trigger binding is fixed at bind time, so you cannot retarget the input queue per message.
Outbound: encrypt, then Ready
Egress is stricter. This is where pgp encryption azure stops being optional: postprocessing always encrypts. It reads the ledger row {publisher}:{interface}:{fileName} (partition /interface), streams the source blob through IPgpService.EncryptAsync, then uploads to a configured destination — Postprocessing:OutputAccountEndpoint + OutputContainer (sample: container encrypted) with path source.Path + EncryptedFileSuffix (default .pgp). It is not an in-place write on the source blob. Then it sets ledger status Ready and forwards the envelope. Sequenced delivery waits on Ready. Unsequenced sender will still upload a non-Ready row; it only short-circuits on Delivered. I covered those two in the SFTP post.
Overwrite is on. The output name is deterministic, so a redelivery that already uploaded is a no-op write. A send that already succeeded is absorbed by duplicate detection. Cosmos ETag conflicts surface as LedgerConcurrencyException and are abandoned, not dead-lettered.
I am honest about one asymmetry: inbound decrypt is blob-to-blob streaming. Outbound encrypt currently collects ciphertext in a MemoryStream before UploadAsync. Large partner files will pressure the Function’s memory on the way out. That is a real gap in PostprocessingProcessor, not a feature.
# functions/postprocessing/README.md (deploy sketch)
Postprocessing__RecipientPublicKey = "@Microsoft.KeyVault(SecretUri=${module.platform.keyvault_uri}secrets/recipient-pgp-public-key)"
Microsoft’s App Service / Functions docs use the same @Microsoft.KeyVault(SecretUri=…) shape. The platform resolves the secret into the app setting; the worker identity needs Key Vault Secrets User. I do not open Key Vault from C#.
The crypto type on purpose
Shared package Integration.Connectors.Crypto wraps PgpCore. Keys arrive already resolved. That keeps tests on throwaway keypairs and avoids the older full-buffer decrypt that held the whole file in memory:
// shared/Integration.Connectors.Crypto/PgpCoreService.cs
public async Task EncryptAsync(Stream input, Stream output, string publicKeyArmored, CancellationToken ct)
{
using var publicKeyStream = ToStream(publicKeyArmored);
var encryptionKeys = new EncryptionKeys(publicKeyStream);
using var pgp = new PGP(encryptionKeys);
await pgp.EncryptAsync(input, output);
}
Decrypt is the same shape with an optional passphrase. Empty passphrase is string.Empty, not null, because PgpCore’s EncryptionKeys constructor wants a string.
Poison vs retry (scoreboard)
| Where | Dead-letter (do not retry) | Abandon (retry) |
|---|---|---|
preprocessing |
InvalidEnvelope, UnexpectedSourceContainer, MissingDecryptionKey, InvalidConfig, ProcessingFailed |
408 / 429 / 5xx, transient Service Bus, I/O, timeout |
postprocessing |
InvalidEnvelope, MissingPublicKey, LedgerRecordMissing, SourceReadFailed, EncryptionFailed, OutputWriteFailed, OutputSendFailed |
408 / 429 / 449 / 5xx, network, LedgerConcurrencyException |
MissingPublicKey is per message on purpose. A late-resolving Key Vault reference or a mid-rotation empty secret should land in the DLQ, not take the host down at startup. LedgerRecordMissing means the envelope was hand-built or misrouted — ingestion is supposed to create the row before this queue sees the file.
RBAC on the identity is least privilege: blob contributor on the containers it writes, blob reader on source accounts for encrypt, Service Bus receiver/sender on the named queues, Cosmos data on the ledger, Key Vault Secrets User. No connection strings in the samples.
When I would not use these two
If every partner file is already TLS-protected SFTP and the receiving system decrypts itself, skip preprocessing decrypt — leave DecryptionRequired: false and keep the connector in the chain so you do not special-case the YAML. If you need CMS/PKCS#7 instead of OpenPGP, this package is the wrong tool; do not pretend PgpCore is a generic encryptor. If files are multi-gigabyte, fix the outbound MemoryStream before you scale out postprocessing.
FAQ
Why two connectors instead of one encrypt/decrypt microservice?
Inbound and outbound fail differently. Decrypt can no-op. Encrypt must stamp Ready or delivery will never fire. One instance per recipient also keeps key material scoped. A single “crypto service” would grow a partner switch statement, which is how this catalog starts to rot.
Where do PGP keys live?
In Key Vault. App settings hold @Microsoft.KeyVault(SecretUri=…) references. PgpCoreService never calls the Key Vault SDK. Locally you can inline a throwaway armored block in local.settings.json.
Does preprocessing always decrypt?
No. DecryptionRequired defaults to false. Same deployed code covers clear and cipher inbound. Config chooses.
Why not fail the host if the public key is missing?
Because Key Vault references can resolve late, and rotation can briefly empty a secret. Dead-letter the message; keep the worker up. You will see MissingPublicKey on the DLQ instead of a cold host.
Is ciphertext streamed end-to-end?
Inbound yes (blob read → decrypt → blob write). Outbound encrypt currently buffers ciphertext in memory, then uploads with overwrite. Treat large-file egress as a known limit.
What does ledger Ready mean?
It is the gate sequenced delivery waits on before SFTP upload. Unsequenced sender does not wait on Ready; it skips only when the row is already Delivered. If you encrypt and never flip Ready, ordered delivery will not ship. Fire-and-go sender still might.
Next in this series is transform-once, deliver-many: transformation, apidelivery, and webhookdelivery. Until then, the source of truth remains integration-azure — read functions/preprocessing/README.md and functions/postprocessing/README.md before you copy snippets into a Function App of your own.
