Add end-to-end encryption for workflow user data#950
Draft
TooTallNate wants to merge 7 commits intomainfrom
Draft
Add end-to-end encryption for workflow user data#950TooTallNate wants to merge 7 commits intomainfrom
TooTallNate wants to merge 7 commits intomainfrom
Conversation
🦋 Changeset detectedLatest commit: f95f584 The changes in this PR will be included in the next version bump. This PR includes changesets to release 19 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
Contributor
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests🌍 Community Worlds (169 failed)mongodb (42 failed):
redis (42 failed):
starter (43 failed):
turso (42 failed):
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
❌ 🌍 Community Worlds
✅ 📋 Other
|
Contributor
Contributor
There was a problem hiding this comment.
Pull request overview
This PR implements end-to-end encryption for workflow user data using AES-256-GCM with per-run key derivation via HKDF-SHA256. The implementation requires client-side runId generation to enable encryption before data serialization.
Changes:
- Added encryption module with AES-256-GCM + HKDF-SHA256 key derivation
- Converted all (de)hydration serialization functions to async with encryption support
- Implemented client-side runId generation for encryption context
- Updated workflow execution, steps, hooks, and observability for async serialization
- Added comprehensive encryption test coverage (18 unit tests)
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/world/src/interfaces.ts | Added Encryptor, EncryptionContext, and KeyMaterial interfaces; extended World interface |
| packages/world/src/events.ts | Added optional client-provided runId field to run_created event |
| packages/world-vercel/src/encryption.ts | New encryption implementation with HKDF-based per-run key derivation |
| packages/world-vercel/src/encryption.test.ts | Comprehensive test suite for encryption functionality |
| packages/world-vercel/src/index.ts | Integrated encryptor into Vercel World implementation |
| packages/core/src/serialization.ts | Made all serialization functions async; added encryption/decryption helpers |
| packages/core/src/serialization.test.ts | Updated test wrappers for async serialization |
| packages/core/src/workflow.ts | Added world parameter to runWorkflow; updated hydration calls |
| packages/core/src/workflow.test.ts | Updated test helpers with mock world and async wrappers |
| packages/core/src/step.ts | Updated step hydration for async operation |
| packages/core/src/step.test.ts | Updated test helpers with mock world |
| packages/core/src/workflow/hook.ts | Updated hook payload hydration for async operation |
| packages/core/src/workflow/hook.test.ts | Updated test helpers with mock world |
| packages/core/src/runtime/start.ts | Implemented client-side runId generation for encryption |
| packages/core/src/runtime/start.test.ts | Updated mocks to return client-provided runId |
| packages/core/src/runtime/run.ts | Updated result hydration for async operation |
| packages/core/src/runtime/resume-hook.ts | Updated hook metadata hydration for async operation |
| packages/core/src/runtime/step-handler.ts | Updated step I/O serialization for async operation |
| packages/core/src/runtime/suspension-handler.ts | Updated event creation with async serialization |
| packages/core/src/runtime.ts | Pass world instance to runWorkflow |
| packages/core/src/private.ts | Added runId and world to WorkflowOrchestratorContext |
| packages/core/src/observability.ts | Made hydrateResourceIO async with world parameter |
| packages/core/src/observability.test.ts | Updated test helpers with mock world |
| packages/core/src/writable-stream.test.ts | Removed Promise support tests |
| packages/web-shared/src/api/workflow-server-actions.ts | Updated all hydration calls to async with world parameter |
| packages/cli/src/lib/inspect/output.ts | Updated all hydration calls to async with world parameter |
Comments suppressed due to low confidence (2)
packages/world/src/interfaces.ts:109
- The Streamer interface still allows
runId: string | Promise<string>for writeToStream, writeToStreamMulti, and closeStream, but WorkflowServerWritableStream now only acceptsstring(line 393). This could cause confusion for World implementations.
Since runId is now always generated client-side before serialization (as required for encryption), the Streamer interface should be updated to only accept string for consistency. World implementations (world-local, world-postgres, world-vercel) may need to be updated to match this stricter type.
writeToStream(
name: string,
runId: string | Promise<string>,
chunk: string | Uint8Array
): Promise<void>;
/**
* Write multiple chunks to a stream in a single operation.
* This is an optional optimization for world implementations that can
* batch multiple writes efficiently (e.g., single HTTP request for world-vercel).
*
* If not implemented, the caller should fall back to sequential writeToStream() calls.
*
* @param name - The stream name
* @param runId - The run ID (can be a promise)
* @param chunks - Array of chunks to write, in order
*/
writeToStreamMulti?(
name: string,
runId: string | Promise<string>,
chunks: (string | Uint8Array)[]
): Promise<void>;
closeStream(name: string, runId: string | Promise<string>): Promise<void>;
packages/core/src/runtime/start.ts:143
- The client-generated runId is passed in the event data (line 134) but the server implementations (world-local and world-postgres) check the first parameter of events.create() to decide whether to use a client-provided runId or generate one. Since the client passes
nullas the first parameter (line 130), the server will ignore the client-provided runId in the event data and generate its own runId.
This means encryption will fail because the client encrypted data with one runId but the server will use a different runId when attempting to decrypt.
The fix should be to pass the client-generated runId as the first argument to events.create() instead of null, or update all server implementations to check data.runId from the event data for run_created events.
const result = await world.events.create(
null,
{
eventType: 'run_created',
specVersion,
runId, // Pass client-generated runId to server
eventData: {
deploymentId: deploymentId,
workflowName: workflowName,
input: workflowArguments,
executionContext: { traceCarrier, workflowCoreVersion },
},
},
{ v1Compat }
);
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This implements AES-256-GCM encryption with per-run key derivation via
HKDF-SHA256 for workflow user data.
Key changes:
createEncryptor() and createEncryptorFromEnv() functions
Format: [encr (4 bytes)][nonce (12 bytes)][ciphertext + auth tag]