Skip to content

Crayonic Secure Audio Service

Where this fits

Crayonic Secure Audio Service is Option 4 in the Solution Architecture — it can be added to any of the deployment options, with recordings collected either through a Crayonic Bridge and Agent (Option 4a) or through the Crayonic Mobile ID Wallet as courier (Option 4b).

Overview

Crayonic Secure Audio Service is a self-hosted service for organisations whose staff record spoken content that must never be readable by the infrastructure that stores it — above all clinicians dictating notes or recording consultations. The Crayonic Badge™ captures the audio and encrypts it on the device; the service receives, stores, transcribes and forwards results to the electronic health record (EHR), yet never holds a key that can decrypt a recording.

What the service gives you:

  • End-to-end encryption with the badge as the key. Recordings are encrypted with keys derived from the badge's own FIDO2 credential. Storage, transport, administrators and the service itself only ever see ciphertext.
  • Decryption in the clinician's browser. The clinician signs in to the web console with a passkey and unlocks a recording by using the badge as a security key. The plaintext exists only in that browser session, and only for as long as the clinician chooses.
  • Transcription with the speech-to-text engine you choose — a local Whisper engine shipped with the service, the Praxy Santé medical transcription API, or another provider added through the adapter interface.
  • Delivery to your EHR as HL7 v2 ORU^R01 messages over MLLP.
  • Your infrastructure, your data. Docker images, PostgreSQL and an S3-compatible object store, in your data centre or cloud. Nothing Crayonic-hosted is required.

How it works

  1. Record on the badge. The user presses RECORD on the Crayonic Badge (up to five minutes per recording). Audio is encrypted on the badge as it is written to the badge's storage, under a key derived from the badge's FIDO2 credential using the same hmac-secret / WebAuthn PRF primitive the badge already uses for passkeys.
  2. Upload as ciphertext. The encrypted package leaves the badge by one of the two paths described below and is uploaded to the service over HTTPS. The service stores the package in the object store and its public metadata (owner, size, digest, timestamps) in PostgreSQL.
  3. Unlock in the browser. The clinician opens the recording in the web console. The browser asks the badge, acting as a WebAuthn security key with the PRF extension, for the secret that unlocks this recording and decrypts it locally. The clinician can listen, trim and re-save the recording; edited recordings are re-encrypted in the browser before upload.
  4. Transcribe. On the clinician's action, the decrypted audio is sent to the service, which relays it to the configured speech-to-text provider and returns the transcript. The provider's credentials stay on the server, never in the browser.
  5. Send to the EHR. The clinician reviews the transcript and sends it to the EHR. The service builds an HL7 v2 ORU^R01 observation-result message carrying the transcript and delivers it over MLLP to the configured interface engine or EHR endpoint.
flowchart LR
  BADGE["Crayonic Badge<br/>records + encrypts<br/>(key from its FIDO2 credential)"]
  C["Ciphertext<br/>transport + storage"]
  WEB["Clinician's browser<br/>unlock with the badge<br/>as security key"]
  CSA["Crayonic Secure Audio Service<br/>FastAPI · PostgreSQL · S3 store"]
  STT["Speech-to-text<br/>local Whisper · Praxy Santé · adapter"]
  EHR["EHR<br/>HL7 v2 ORU^R01 over MLLP"]
  BADGE -->|"Bridge + Agent, or<br/>Mobile ID Wallet courier"| C --> CSA
  CSA <-->|"ciphertext only"| WEB
  WEB -->|"clinician action"| CSA
  CSA --> STT
  CSA --> EHR
  classDef plain fill:#ffe0e0,stroke:#c00;
  classDef cipher fill:#e0f0ff,stroke:#06c;
  class WEB plain;
  class C,CSA cipher;

Recordings are compressed before encryption

To keep uploads and storage small, recordings handled by the web application are MP3-compressed at 16 kHz before they are encrypted; the encrypted container carries a marker so the console can open both compressed and raw-PCM recordings transparently. 16 kHz MP3 is also the input format recommended by Praxy Santé, so no server-side transcoding is needed at transcription time.

Two ways to get recordings off the badge

Both paths deliver ciphertext only and can coexist on one service; choose per site.

Bridge + Agent (managed workstations) Mobile ID Wallet as courier (no Bridge, no Agent)
How The badge exposes its recordings as a USB mass-storage volume. At a workstation with a Crayonic Bridge and Crayonic Agent, the Agent imports the packages, uploads them and verifies the service's signed receipt before the badge copy is deleted. The user's own Android phone running the Crayonic Mobile ID Wallet, already paired with the badge, notices when the badge advertises pending audio, pulls the encrypted packages over Bluetooth, uploads them and hands the service's signed receipt back to the badge so it can free the space.
Capture-site footprint Windows PC + Bridge dongle None beyond the phone the user already carries
Who authenticates the uploader Device token issued by an administrator The badge's existing FIDO2 credential, paired once by the user — no administrator step
What a lost uploader can do — The phone holds only an upload-only token and opaque ciphertext: it cannot read, decrypt, list or delete anything
Deletion on the badge Agent verifies the service receipt The badge itself verifies the signed receipt before deleting
Availability Whenever the badge is at a managed workstation Whenever the phone is nearby with Bluetooth on; recordings wait on the badge until then
Platform Windows Android (the protocol is platform-neutral; iOS is not available today)

The clinician can also import an encrypted package file into the web console manually, for example from the badge's mass-storage volume.

What an administrator configures

All operator settings are in the web console's Settings page, protected by an administrator token:

Area Settings
Storage Backend (S3-compatible object store): endpoint URL, region, bucket, access key ID and secret access key; storage quota per deployment (GB).
Transcription (STT) Provider: local Whisper (faster-whisper, with a selectable model) or Praxy Santé (partner username and password, base URL for the partner or production environment). Further providers with an HTTP API are added through a single adapter interface.
LLM Provider, API base URL, API key and model. Reserved for planned transcript post-processing (for example structured clinical notes); no processing uses it yet.
EHR delivery (HL7) MLLP host and port; sending and receiving application and facility identifiers used in the MSH segment.

Deployment

The service is delivered as Docker images and runs in your data centre or cloud:

  • audio-service — the FastAPI control plane (identity registry, uploads and receipts, recordings, transcription and EHR delivery APIs).
  • PostgreSQL — metadata, registry, receipts and job queue.
  • S3-compatible object store — ciphertext packages (Garage is used in the reference stack; MinIO or AWS S3 work equally).
  • web — the clinician and operator console (optional profile; can be served by your own ingress).
  • worker (optional) — a background processing worker with no inbound ports for the Bridge + Agent path; it needs outbound access only to PostgreSQL, the object store and the STT endpoint.

A reference docker-compose stack (PostgreSQL, Garage, one-shot migration, service, worker, optional web profile) is provided; for production, port it to your orchestrator behind your own TLS ingress. Users sign in with passkeys; the badge itself is the passkey.

Trust boundary to note. Speech-to-text is, by definition, a plaintext point: the STT engine must hear the audio. With local Whisper that point stays inside your network; with Praxy Santé or another external provider, audio is sent to that provider over TLS on the clinician's explicit action. Confirm that your data-residency and patient-data policy permit this before enabling an external provider.

Status

To be clear about what is in use and what is specified:

Part Status
Service, web console, passkey sign-in, browser-side encryption and decryption, MP3 compression, S3 storage, quota In use
Local Whisper transcription In use
Praxy Santé transcription adapter Implemented against Praxy Santé's published API; end-to-end use requires a Praxy Santé partner account
HL7 v2 ORU^R01 over MLLP Implemented as a standards-conformant message builder and MLLP client; not yet exercised against a customer's interface engine. The transcript is carried as a free-text OBX segment; other shapes can be agreed with your EHR team
Badge-side recording and on-device encryption Specified (badge firmware specifications DOC/28 and DOC/29); firmware in development, not yet shipping
Bridge + Agent collection path Designed; depends on the badge-side release
Mobile ID Wallet courier path Specified on all three sides (badge, phone, service); in development, not yet shipping
LLM post-processing of transcripts Planned; configuration slot reserved

Until the badge-side release, the web console records directly from the browser microphone or from an uploaded WAV file, applying the same encryption and workflow.

A property to plan for. Because keys derive from one physical badge, a recording can be unlocked only with the badge that made it. There is no recovery with a second badge: a lost badge means its still-encrypted recordings cannot be opened. Transcripts already sent to the EHR are unaffected.