diff --git a/docs/adr/0015-pii-encryption-at-rest.md b/docs/adr/0015-pii-encryption-at-rest.md new file mode 100644 index 000000000..201f7a271 --- /dev/null +++ b/docs/adr/0015-pii-encryption-at-rest.md @@ -0,0 +1,45 @@ +# 15. PII encryption at rest (AES-256-GCM + key rotation) + +Date: 2026-05-04 + +## Status + +Accepted + +## Context + +The portal persists personally identifiable information (PII) that must be shown again to the user or used in flows after it is stored (for example, email, phone, program identifiers, and date of birth from ID proofing). One-way hashing is appropriate for matching-only fields (for example, SSN via `IIdentifierHasher`), but reversible protection is required for fields that are read back in clear form at the application layer. + +We also need email lookup by equality in SQL without storing a searchable plaintext email. A deterministic lookup fingerprint (HMAC over the normalized address) supports indexed lookup while ciphertext remains non-deterministic under AES-GCM. + +## Decision + +1. **Algorithm and envelope.** Use AES-256-GCM with a random nonce per encryption, prefixed by a stable ASCII sentinel (`sep-pii:v1:`). The ciphertext column stores UTF-8 text that embeds **key id**, nonce, tag, and payload so decryption can resolve the correct key from a configured ring. + +2. **Key management.** Configure `PiiEncryption:ActiveKeyId` and `PiiEncryption:Keys[]` (`KeyId` + Base64-encoded 256-bit material). Implementations resolve keys via `PiiEncryptionSettings.ResolveKeyRing()`. New columns or environments add keys without schema changes. + +3. **Rotation vs. backfill.** **Key rotation** (moving ciphertext from an older key id to the current `ActiveKeyId`) is done with `IPiiSymmetricEncryption.ReSealWithActiveEncryptor(...)` in an operational or batch job — decrypt using the key id embedded in the envelope, then encrypt with the active key. **Startup backfill** (`PiiPlaintextEncryptionBackfill`) is separate: it runs after migrations to encrypt legacy plaintext at rest and to attach `EmailHash` where the email column is already an envelope but `EmailHash` was missing — it does not, by itself, re-wrap every row when only `ActiveKeyId` changes. + +4. **Email lookup.** Persist `Email` as ciphertext and `EmailHash` as `IEmailLookupHasher.HashNormalized(...)`. Queries use equality on `EmailHash`, with transitional fallbacks where legacy plaintext rows remain. The unique index applies to non-null `EmailHash` values. Bulk adds via `IDataSeeder.AddUsers` / `AddUsersAsync` skip users whose normalized email is already recognized by `GetExistingUserEmails*` / `GetExistingUserEmailsAsync`, so a legacy plaintext row cannot sit beside a duplicate insert for the same address. + +5. **Profile update vs. legacy plaintext.** User updates encrypt email and populate `EmailHash`. Before saving, `DatabaseUserRepository` rejects updates when another user (different Id) still stores the target address as plaintext (EmailHash null and Email column equals the normalized address without the envelope prefix), avoiding silent duplicates alongside the filtered unique index on `EmailHash`. + +6. **Failure semantics.** Authenticated decryption failures surface as `PiiDecryptException` (wrapping the underlying crypto error) so corruption or tampering is explicit rather than returning empty strings. + +7. **Operational guardrails.** Logging for encryption backfill and seed paths must not include decrypted or plaintext PII; messages use counts and generic conflict text only. + +## Consequences + +- **Pros:** Strong confidentiality at rest for reversible PII, forward-compatible key rotation, indexed email lookup without plaintext email in the database, and a single encryption path for future columns that need the same pattern. +- **Cons:** Key material must be managed like other secrets (rotation runbook, secure distribution). Decryption is required on every read path touched by repositories; performance impact is small relative to I/O but must be kept in hot paths. Production startup rejects placeholder `PiiEncryption` keys (`PiiEncryptionGuard`), mirroring `IdentifierHasherGuard`. +- **Migration:** SQL Server `date`-typed `Users.DateOfBirth` is converted to `nvarchar` via a migration batch that uses dynamic SQL so column renames and copies parse correctly. Down-migration is intentionally unsupported once ciphertext is written. +- **Email lookup keying:** `EmailLookupHasher` derives its HMAC key from the same `IdentifierHasher:SecretKey` used for other deterministic hashes, with a distinct domain prefix so message formats do not collide. Rotating that secret affects **both** identifier hashing and email lookup hashes — plan coordinated rotation or introduce a dedicated secret later if isolation is required. +- **Backfill failures:** If `PiiPlaintextEncryptionBackfill` throws during startup, the failure is logged at **error** severity (structured logs / Datadog). The API process still starts so transient DB issues do not brick the service; **operations should alert on that log** and re-run or fix the underlying issue until backfill completes, since plaintext-at-rest rows may remain until then. + +## References + +- `SEBT.Portal.Core.Services.IPiiSymmetricEncryption`, `IEmailLookupHasher` +- `SEBT.Portal.Infrastructure.Services.PiiAesGcmSymmetricEncryption`, `PiiPlaintextEncryptionBackfill` +- `SEBT.Portal.Infrastructure.Repositories.DatabaseUserRepository`, `DatabaseDocVerificationChallengeRepository` +- Unit tests: `PiiAesGcmSymmetricEncryptionTests`, `PiiPlaintextEncryptionBackfillTests`, `PiiEncryptionGuardTests`, `PiiEncryptionSettingsValidatorTests` +- Production guard: `SEBT.Portal.Api.Startup.PiiEncryptionGuard`; options coherence: `SEBT.Portal.Infrastructure.Configuration.PiiEncryptionSettingsValidator` diff --git a/src/SEBT.Portal.Api/Program.cs b/src/SEBT.Portal.Api/Program.cs index bcf0b8c6a..279d522cd 100644 --- a/src/SEBT.Portal.Api/Program.cs +++ b/src/SEBT.Portal.Api/Program.cs @@ -1,5 +1,6 @@ using System.Text; using System.Threading.RateLimiting; +using System.Data.Common; using Microsoft.AspNetCore.Authentication.JwtBearer; using Microsoft.AspNetCore.Diagnostics.HealthChecks; using Microsoft.AspNetCore.HttpOverrides; @@ -16,6 +17,7 @@ using SEBT.Portal.Api.Options; using SEBT.Portal.Api.Services; using SEBT.Portal.Core.AppSettings; +using SEBT.Portal.Core.Exceptions; using SEBT.Portal.Core.Services; using SEBT.Portal.Infrastructure.Configuration; using SEBT.Portal.Infrastructure.Services; @@ -413,6 +415,10 @@ await context.HttpContext.Response.WriteAsJsonAsync( if (app.Environment.IsProduction()) { IdentifierHasherGuard.ValidateForProduction(app.Configuration["IdentifierHasher:SecretKey"]); + + var piiEncryptionSettings = app.Configuration.GetSection(PiiEncryptionSettings.SectionName) + .Get(); + PiiEncryptionGuard.ValidateForProduction(piiEncryptionSettings); } // HMAC-SHA256 requires ≥256-bit (32-byte) key. Fail fast if configured but too short. @@ -431,6 +437,32 @@ await context.HttpContext.Response.WriteAsJsonAsync( var databaseMigrator = scope.ServiceProvider.GetRequiredService(); await databaseMigrator.MigrateAsync(); + try + { + var piiBackfill = scope.ServiceProvider.GetRequiredService(); + await piiBackfill.ApplyAsync(CancellationToken.None); + } + catch (PiiDecryptException backfillEx) + { + Log.Error( + backfillEx, + "PII ciphertext backfill failed due to decryption/authentication error. " + + "Startup continues, but legacy plaintext may remain until this is resolved."); + } + catch (DbException backfillEx) + { + Log.Warning( + backfillEx, + "PII ciphertext backfill hit a database error (likely transient). " + + "Startup continues; backfill should be retried."); + } + catch (Exception backfillEx) + { + // Error severity so deploy logs / Datadog can alert; app still starts so a transient DB + // blip does not take down the service — see docs/adr/0015-pii-encryption-at-rest.md. + Log.Error(backfillEx, "PII ciphertext backfill step failed."); + } + var seedingSettings = app.Configuration.GetSection(SeedingSettings.SectionName).Get(); if (app.Environment.IsDevelopment() || seedingSettings?.Enabled == true) { diff --git a/src/SEBT.Portal.Api/Startup/PiiEncryptionGuard.cs b/src/SEBT.Portal.Api/Startup/PiiEncryptionGuard.cs new file mode 100644 index 000000000..49c7b5dcb --- /dev/null +++ b/src/SEBT.Portal.Api/Startup/PiiEncryptionGuard.cs @@ -0,0 +1,75 @@ +using SEBT.Portal.Core.AppSettings; + +namespace SEBT.Portal.Api.Startup; + +/// +/// Validates that PII encryption keys are not default or placeholder values in production. +/// +public static class PiiEncryptionGuard +{ + /// Matches appsettings.json sample ActiveKeyId — not safe for production. + public const string ForbiddenDevelopmentActiveKeyId = "local-dev-primary"; + + /// 32× ASCII 'a' (256-bit) — sample key in repo; must not be used in production. + public const string ForbiddenPlaceholderKeyMaterialBase64 = "YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWE="; + + /// + /// Validates configured PII encryption for production. Throws if missing, empty, or known placeholders. + /// + public static void ValidateForProduction(PiiEncryptionSettings? settings) + { + if (settings == null) + { + throw new InvalidOperationException( + "PiiEncryption configuration section is missing. Configure PiiEncryption:ActiveKeyId and PiiEncryption:Keys."); + } + + if (string.IsNullOrWhiteSpace(settings.ActiveKeyId)) + { + throw new InvalidOperationException( + "PiiEncryption:ActiveKeyId must be set in production (e.g. PIIENCRYPTION__ACTIVEKEYID)."); + } + + if (settings.Keys == null || settings.Keys.Count == 0) + { + throw new InvalidOperationException( + "PiiEncryption:Keys must contain at least one key entry in production."); + } + + if (string.Equals(settings.ActiveKeyId.Trim(), ForbiddenDevelopmentActiveKeyId, StringComparison.OrdinalIgnoreCase)) + { + throw new InvalidOperationException( + $"PiiEncryption:ActiveKeyId must not be '{ForbiddenDevelopmentActiveKeyId}' in production. " + + "Use a deployment-specific key id and secrets management."); + } + + foreach (var entry in settings.Keys) + { + if (entry == null) + { + throw new InvalidOperationException( + "PiiEncryption:Keys must not contain null entries in production."); + } + + if (string.IsNullOrWhiteSpace(entry.KeyId)) + { + throw new InvalidOperationException( + "Each PiiEncryption:Keys entry must have a non-empty KeyId in production."); + } + + if (string.IsNullOrWhiteSpace(entry.KeyMaterialBase64)) + { + throw new InvalidOperationException( + "Each PiiEncryption:Keys entry must have KeyMaterialBase64 set in production."); + } + + var material = entry.KeyMaterialBase64.Trim(); + if (string.Equals(material, ForbiddenPlaceholderKeyMaterialBase64, StringComparison.Ordinal)) + { + throw new InvalidOperationException( + "PiiEncryption key material must not use the repository placeholder Base64 value in production. " + + "Generate random 256-bit keys and store them in secrets (e.g. PIIENCRYPTION__KEYS__0__KEYMATERIALBASE64)."); + } + } + } +} diff --git a/src/SEBT.Portal.Api/appsettings.Development.example.json b/src/SEBT.Portal.Api/appsettings.Development.example.json index 5b13387c2..57244cba7 100644 --- a/src/SEBT.Portal.Api/appsettings.Development.example.json +++ b/src/SEBT.Portal.Api/appsettings.Development.example.json @@ -31,6 +31,15 @@ "IdentifierHasher": { "SecretKey": "YOUR_IDENTIFIER_HASHER_KEY_AT_LEAST_32_CHARS" }, + "PiiEncryption": { + "ActiveKeyId": "local-dev-primary", + "Keys": [ + { + "KeyId": "local-dev-primary", + "KeyMaterialBase64": "YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWE=" + } + ] + }, "StateHouseholdId": { "PreferredHouseholdIdTypes": ["Phone"] }, diff --git a/src/SEBT.Portal.Api/appsettings.json b/src/SEBT.Portal.Api/appsettings.json deleted file mode 100644 index 1e46a1a93..000000000 --- a/src/SEBT.Portal.Api/appsettings.json +++ /dev/null @@ -1,168 +0,0 @@ -{ - "Serilog": { - "Using": ["Serilog.Sinks.Console", "Serilog.Sinks.File"], - "MinimumLevel": { - "Default": "Information", - "Override": { - "Microsoft": "Warning", - "Microsoft.AspNetCore": "Warning", - "System": "Warning" - } - }, - "WriteTo": [ - { - "Name": "File", - "Args": { - "path": "logs/sebt-portal-.log", - "rollingInterval": "Day", - "retainedFileCountLimit": 7, - "outputTemplate": "[{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} {Level:u3}] {Message:lj} {Properties:j}{NewLine}{Exception}" - } - } - ], - "Enrich": ["FromLogContext"] - }, - "Logging": { - "LogLevel": { - "Default": "Information", - "Microsoft.AspNetCore": "Warning" - } - }, - "Otel": { - "UseTracingExporter": "otlp", - "UseMetricsExporter": "otlp", - "UseLogExporter": "otlp", - "HistogramAggregation": "explicit", - "Otlp": { - "Endpoint": "http://localhost:4317" - }, - "AspNetCoreInstrumentation": { - "RecordException": "true" - } - }, - "AllowedHosts": "*", - "ConnectionStrings": { - "DefaultConnection": "Server=localhost,1433;Database=SebtPortal;User Id=sa;Password=YourStrong@Passw0rd;" - }, - "EmailOtpSenderServiceSettings": { - "SenderEmail": "noreply@sunbucks.dc.gov", - "SenderName": "DC SUN Bucks", - "Subject": "Your DC SUN Bucks Login Code", - "ProgramName": "DC SUN Bucks", - "StateName": "DC SUN Bucks", - "ExpiryMinutes": 10, - "Language": "en" - }, - "SmtpClientSettings": { - "SmtpServer": "localhost", - "SmtpPort": 1025, - "EnableSsl": false - }, - "OtpRateLimitSettings": { - "PermitLimit": 5, - "WindowMinutes": 1.0 - }, - "EnrollmentCheckRateLimitSettings": { - "PermitLimit": 10, - "WindowMinutes": 1.0 - }, - "WebhookRateLimitSettings": { - "PermitLimit": 60, - "WindowMinutes": 1.0 - }, - "IdentifierHasher": { - "SecretKey": "OverrideInProductionUseEnvVarIDENTIFIERHASHER__SECRETKEY" - }, - "JwtSettings": { - // Intentionally empty — overridden by Secrets Manager via Tofu in deployed environments - "SecretKey": "", - "Issuer": "SEBT.Portal.Api", - "Audience": "SEBT.Portal.Web", - // Idle and absolute timeouts together implement OWASP / NIST SP 800-63B IAL2 session policy. - "ExpirationMinutes": 15, - "AbsoluteExpirationMinutes": 60 - }, - "IdProofingRequirements": { - "address+view": "IAL1plus", - "address+write": "IAL1plus", - "email+view": "IAL1", - "phone+view": "IAL1", - "household+view": "IAL1plus", - "card+write": "IAL1plus" - }, - "Oidc": { - "CompleteLoginSigningKey": "AT_LEAST_32_CHARACTERS_FOR_HMAC_SHA256_SIGNING" - }, - "StateHouseholdId": { - "PreferredHouseholdIdTypes": ["Email"] - }, - "AppConfig": { - "Agent": { - "BaseUrl": "http://localhost:2772", - "ApplicationId": "", - "EnvironmentId": "", - "ReloadAfterSeconds": 90 - }, - "FeatureFlags": { - "ProfileId": "" - }, - "AppSettings": { - "ProfileId": "" - } - }, - "CoLoadedCohortFilter": { - "SuppressCoLoadedCasesForExcludedCohort": true - }, - "SelfServiceRules": { - "AddressUpdate": { - "Enabled": true, - "DisabledMessageKey": "actionNavigationSelfServiceUnavailable", - "ByIssuanceType": { - "SummerEbt": { "Enabled": true, "AllowedCardStatuses": [] }, - "TanfEbtCard": { "Enabled": false }, - "SnapEbtCard": { "Enabled": false }, - "Unknown": { "Enabled": false } - } - }, - "CardReplacement": { - "Enabled": true, - "DisabledMessageKey": "actionNavigationSelfServiceUnavailable", - "ByIssuanceType": { - "SummerEbt": { "Enabled": true, "AllowedCardStatuses": [] }, - "TanfEbtCard": { "Enabled": false }, - "SnapEbtCard": { "Enabled": false }, - "Unknown": { "Enabled": false } - } - } - }, - "Smarty": { - "Enabled": false, - "AuthId": "", - "AuthToken": "", - "BaseUrl": "https://us-street.api.smartystreets.com", - "TimeoutSeconds": 20 - }, - "AddressValidationPolicy": { - "AllowGeneralDelivery": true - }, - "AddressValidationData": { - "BlockedAddresses": [], - "StreetAbbreviations": {}, - "MaxStreetAddressLength": 0 - }, - "FeatureManagement": { - "email_dob_opt_in": false, - "show_application_number": true, - "show_case_number": true, - "show_card_last4": true, - "enable_beta_banner": false, - "show_contact_preferences": false, - "AppConfig": { - "Enabled": false, - "ApplicationId": "", - "EnvironmentId": "", - "ConfigurationProfileId": "" - }, - "bypass_otp": false - } -} diff --git a/src/SEBT.Portal.Core/AppSettings/PiiEncryptionSettings.cs b/src/SEBT.Portal.Core/AppSettings/PiiEncryptionSettings.cs new file mode 100644 index 000000000..00623fe28 --- /dev/null +++ b/src/SEBT.Portal.Core/AppSettings/PiiEncryptionSettings.cs @@ -0,0 +1,73 @@ +using System.ComponentModel.DataAnnotations; + +namespace SEBT.Portal.Core.AppSettings; + +/// +/// AES-256-GCM key material configuration for reversible PII column encryption (see ADR). +/// +public class PiiEncryptionSettings +{ + public const string SectionName = "PiiEncryption"; + + /// Decoded key material must be exactly this many bytes (256-bit AES only). + public const int RequiredKeyMaterialLengthBytes = 32; + + /// The key id entries are encrypted or re-encrypted with at write time. + [Required(ErrorMessage = "PiiEncryption:ActiveKeyId is required.")] + [MinLength(1)] + public string ActiveKeyId { get; set; } = ""; + + /// Historical + active symmetric keys keyed by logical id. + [Required(ErrorMessage = "PiiEncryption:Keys is required.")] + [MinLength(1, ErrorMessage = "PiiEncryption requires at least one key entry.")] + public List Keys { get; set; } = []; + + public IReadOnlyDictionary ResolveKeyRing() + { + var dictionary = new Dictionary(StringComparer.OrdinalIgnoreCase); + foreach (var k in Keys) + { + var id = k.KeyId.Trim(); + if (dictionary.ContainsKey(id)) + { + throw new InvalidOperationException($"Duplicate PII encryption KeyId '{id}'."); + } + + dictionary[id] = Convert.FromBase64String(k.KeyMaterialBase64.Trim()); + } + + foreach (var (id, raw) in dictionary) + { + if (raw.Length != RequiredKeyMaterialLengthBytes) + { + throw new InvalidOperationException( + $"PII encryption key '{id}' must decode to exactly {RequiredKeyMaterialLengthBytes} bytes (256-bit AES-GCM); actual length was {raw.Length}."); + } + } + + var active = ActiveKeyId.Trim(); + if (active.Length == 0 || + !dictionary.Keys.Any(k => string.Equals(k, active, StringComparison.OrdinalIgnoreCase))) + { + throw new InvalidOperationException( + $"PiiEncryption:ActiveKeyId '{ActiveKeyId}' was not found in PiiEncryption:Keys."); + } + + return dictionary; + } +} + +public class PiiEncryptionKeySetting +{ + /// + /// Logical identifier embedded in ciphertext (stable across deployments for rotation/decrypt). + /// + [Required(ErrorMessage = "Each PII encryption key entry requires KeyId.")] + [MinLength(1)] + public string KeyId { get; set; } = ""; + + /// Raw AES-256 key bytes (store Base64 in configuration; decoded length must be 32). + [Required(ErrorMessage = "Each PII encryption key entry requires KeyMaterialBase64.")] + [MinLength(1)] + public string KeyMaterialBase64 { get; set; } = ""; +} diff --git a/src/SEBT.Portal.Core/Exceptions/PiiDecryptException.cs b/src/SEBT.Portal.Core/Exceptions/PiiDecryptException.cs new file mode 100644 index 000000000..be1080594 --- /dev/null +++ b/src/SEBT.Portal.Core/Exceptions/PiiDecryptException.cs @@ -0,0 +1,22 @@ +namespace SEBT.Portal.Core.Exceptions; + +/// +/// Thrown when an encrypted payload cannot be decrypted (tampering, corruption, unknown key/version). +/// Plaintext-at-rest transitional values () do not trigger this exception. +/// +public class PiiDecryptException : Exception +{ + public PiiDecryptException() + { + } + + public PiiDecryptException(string message) + : base(message) + { + } + + public PiiDecryptException(string message, Exception innerException) + : base(message, innerException) + { + } +} diff --git a/src/SEBT.Portal.Core/Services/IEmailLookupHasher.cs b/src/SEBT.Portal.Core/Services/IEmailLookupHasher.cs new file mode 100644 index 000000000..7ec801eb9 --- /dev/null +++ b/src/SEBT.Portal.Core/Services/IEmailLookupHasher.cs @@ -0,0 +1,16 @@ +namespace SEBT.Portal.Core.Services; + +/// +/// Computes a deterministic 64-character hex MAC for normalized email lookups (equality in SQL via persisted email-hash column). +/// Separate from identifier hashing normalization rules used for SNAP/TANF/SSN. +/// +public interface IEmailLookupHasher +{ + /// + /// Returns null when is null/whitespace or cannot be normalized; otherwise returns the same MAC as for that normalized address. + /// + string? NormalizeAndHash(string? email); + + /// Returns null when email is null/whitespace; otherwise trims + lowercases then MACs UTF-8. + string? HashNormalized(string? normalizedLowercaseTrimmedEmail); +} diff --git a/src/SEBT.Portal.Core/Services/IPiiSymmetricEncryption.cs b/src/SEBT.Portal.Core/Services/IPiiSymmetricEncryption.cs new file mode 100644 index 000000000..d506c0556 --- /dev/null +++ b/src/SEBT.Portal.Core/Services/IPiiSymmetricEncryption.cs @@ -0,0 +1,31 @@ +namespace SEBT.Portal.Core.Services; + +/// +/// Encrypts/decrypts short UTF-8 strings at rest using an authenticated envelope bound to an explicit key id. +/// +public interface IPiiSymmetricEncryption +{ + /// + /// True when the stored value was produced by (case-sensitive prefix). + /// + bool IsEnvelope(string? storedValue); + + /// Returns null when is null or empty. + string? Encrypt(string? plaintext); + + /// + /// Requires a valid authenticated envelope produced by this service — throws on mismatch. + /// + string Decrypt(string storedValue); + + /// + /// Converts stored column text to plaintext. Envelopes decrypt; non-envelope values are returned verbatim (migration / legacy plaintext). + /// Throws for envelopes that decrypt but fail AEAD verification. + /// + string? DecryptOrPassThroughLegacy(string? storedValue); + + /// + /// Decrypts ciphertext ( true), then seals it again using the configured active encryptor (key rotation / re-pack helper). + /// + string ReSealWithActiveEncryptor(string envelopeCiphertext); +} diff --git a/src/SEBT.Portal.Infrastructure/Configuration/PiiEncryptionSettingsValidator.cs b/src/SEBT.Portal.Infrastructure/Configuration/PiiEncryptionSettingsValidator.cs new file mode 100644 index 000000000..361fcdd73 --- /dev/null +++ b/src/SEBT.Portal.Infrastructure/Configuration/PiiEncryptionSettingsValidator.cs @@ -0,0 +1,30 @@ +using Microsoft.Extensions.Options; +using SEBT.Portal.Core.AppSettings; + +namespace SEBT.Portal.Infrastructure.Configuration; + +/// +/// Ensures binds coherently: key ring parses, 256-bit key lengths, and ActiveKeyId resolves. +/// +public sealed class PiiEncryptionSettingsValidator : IValidateOptions +{ + /// + public ValidateOptionsResult Validate(string? name, PiiEncryptionSettings options) + { + if (options == null) + { + return ValidateOptionsResult.Fail("PiiEncryption configuration section is not present."); + } + + try + { + _ = options.ResolveKeyRing(); + } + catch (Exception ex) + { + return ValidateOptionsResult.Fail($"PiiEncryption configuration is invalid: {ex.Message}"); + } + + return ValidateOptionsResult.Success; + } +} diff --git a/src/SEBT.Portal.Infrastructure/Data/Entities/UserEntity.cs b/src/SEBT.Portal.Infrastructure/Data/Entities/UserEntity.cs index 5020ddbf2..e7570cf9d 100644 --- a/src/SEBT.Portal.Infrastructure/Data/Entities/UserEntity.cs +++ b/src/SEBT.Portal.Infrastructure/Data/Entities/UserEntity.cs @@ -11,10 +11,13 @@ public class UserEntity public Guid Id { get; set; } = Guid.CreateVersion7(); /// - /// The user's email address, used as a unique identifier. + /// AES-GCM ciphertext for the canonical (lowercase) email. Equality lookup uses . /// public string? Email { get; set; } + /// HMAC-derived lookup fingerprint for canonical email equality in SQL. + public string? EmailHash { get; set; } + /// /// The subject identifier from the external identity provider (e.g., PingOne sub claim). /// Null for OTP-authenticated users. @@ -22,9 +25,9 @@ public class UserEntity public string? ExternalProviderId { get; set; } /// - /// The user's date of birth, when collected. + /// Encrypted yyyy-MM-dd payload (or transitional ISO plaintext during migration/backfill). /// - public DateOnly? DateOfBirth { get; set; } + public string? DateOfBirth { get; set; } /// /// Workflow state of ID proofing (NotStarted, InProgress, Completed, Failed, Expired) diff --git a/src/SEBT.Portal.Infrastructure/Data/PortalDbContext.cs b/src/SEBT.Portal.Infrastructure/Data/PortalDbContext.cs index 962d8e1f3..7b33a9c1b 100644 --- a/src/SEBT.Portal.Infrastructure/Data/PortalDbContext.cs +++ b/src/SEBT.Portal.Infrastructure/Data/PortalDbContext.cs @@ -76,11 +76,19 @@ protected override void OnModelCreating(ModelBuilder modelBuilder) entity.HasKey(e => e.Id); entity.Property(e => e.Id).ValueGeneratedNever(); entity.Property(e => e.Email) - .HasMaxLength(255); - entity.HasIndex(e => e.Email) + .HasMaxLength(512); + entity.Property(e => e.EmailHash) + .HasMaxLength(64); + entity.HasIndex(e => e.EmailHash) .IsUnique() - .HasDatabaseName("IX_Users_Email") - .HasFilter("[Email] IS NOT NULL"); + .HasDatabaseName("IX_Users_EmailHash") + .HasFilter("[EmailHash] IS NOT NULL"); + + // Non-unique filtered index for transitional plaintext rows (EmailHash null): keeps OR-branch lookups in + // DatabaseUserRepository from scanning Users after IX_Users_Email was dropped for ciphertext Email column widths. + entity.HasIndex(e => e.Email) + .HasDatabaseName("IX_Users_Email_LegacyLookup") + .HasFilter("[EmailHash] IS NULL AND [Email] IS NOT NULL"); entity.Property(e => e.IdProofingStatus) .IsRequired() .HasDefaultValue(0); // 0 = NotStarted @@ -106,9 +114,10 @@ protected override void OnModelCreating(ModelBuilder modelBuilder) .HasDatabaseName("IX_Users_IdProofingSessionId"); // Household identifier fields - entity.Property(e => e.Phone).HasMaxLength(64); - entity.Property(e => e.SnapId).HasMaxLength(64); - entity.Property(e => e.TanfId).HasMaxLength(64); + entity.Property(e => e.Phone).HasMaxLength(512); + entity.Property(e => e.SnapId).HasMaxLength(512); + entity.Property(e => e.TanfId).HasMaxLength(512); + entity.Property(e => e.DateOfBirth).HasMaxLength(512); entity.Property(e => e.Ssn).HasMaxLength(64); // OIDC external provider identifier — nullable, filtered unique index @@ -179,11 +188,11 @@ protected override void OnModelCreating(ModelBuilder modelBuilder) .HasMaxLength(255); entity.Property(e => e.ProofingDateOfBirth) - .HasMaxLength(32); + .HasMaxLength(512); entity.Property(e => e.ProofingIdType) - .HasMaxLength(64); + .HasMaxLength(512); entity.Property(e => e.ProofingIdValue) - .HasMaxLength(255); + .HasMaxLength(512); entity.Property(e => e.DocvTokenIssuedAt) .HasColumnType("datetime2"); diff --git a/src/SEBT.Portal.Infrastructure/Dependencies.cs b/src/SEBT.Portal.Infrastructure/Dependencies.cs index e5ed7913c..2b80f7007 100644 --- a/src/SEBT.Portal.Infrastructure/Dependencies.cs +++ b/src/SEBT.Portal.Infrastructure/Dependencies.cs @@ -106,6 +106,8 @@ public static IServiceCollection AddPortalInfrastructureServices(this IServiceCo services.AddTransient(); services.AddSingleton(); services.AddSingleton(); + services.AddSingleton(); + services.AddSingleton(); // Expose SocureSettings directly for use case injection (avoids IOptions dependency in UseCases layer). // Scoped so each request gets a consistent snapshot, supporting live AppConfig reload. @@ -250,6 +252,7 @@ public static IServiceCollection AddPortalDbContext( configureOptions?.Invoke(options); }); + services.AddScoped(); services.AddScoped(); services.AddScoped(); @@ -273,6 +276,10 @@ public static IServiceCollection AddPortalInfrastructureAppSettings(this IServic .ValidateDataAnnotations(); services.AddOptions() .BindConfiguration(StateHouseholdIdSettings.SectionName); + services.AddSingleton, PiiEncryptionSettingsValidator>(); + services.AddOptionsWithValidateOnStart() + .BindConfiguration(PiiEncryptionSettings.SectionName) + .ValidateDataAnnotations(); services.AddOptionsWithValidateOnStart() .BindConfiguration(IdentifierHasherSettings.SectionName) .ValidateDataAnnotations(); diff --git a/src/SEBT.Portal.Infrastructure/Helpers/UserFactory.cs b/src/SEBT.Portal.Infrastructure/Helpers/UserFactory.cs index 39c8fff96..4ada1ae65 100644 --- a/src/SEBT.Portal.Infrastructure/Helpers/UserFactory.cs +++ b/src/SEBT.Portal.Infrastructure/Helpers/UserFactory.cs @@ -1,3 +1,4 @@ +using System.Globalization; using SEBT.Portal.Core.Models.Auth; using SEBT.Portal.Core.Utilities; using SEBT.Portal.Infrastructure.Data.Entities; @@ -91,12 +92,13 @@ private static UserEntity MapToEntity(User user) { Id = user.Id, Email = user.Email != null ? EmailNormalizer.Normalize(user.Email) : null, + EmailHash = null, IdProofingStatus = (int)user.IdProofingStatus, IalLevel = (int)user.IalLevel, IdProofingSessionId = user.IdProofingSessionId, IdProofingCompletedAt = user.IdProofingCompletedAt, IdProofingExpiresAt = user.IdProofingExpiresAt, - DateOfBirth = user.DateOfBirth, + DateOfBirth = user.DateOfBirth?.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture), IsCoLoaded = user.IsCoLoaded, CoLoadedLastUpdated = user.CoLoadedLastUpdated, IdProofingAttemptCount = user.IdProofingAttemptCount, diff --git a/src/SEBT.Portal.Infrastructure/Migrations/20260504204427_EncryptPiiAtRestColumnsAndEmailHash.Designer.cs b/src/SEBT.Portal.Infrastructure/Migrations/20260504204427_EncryptPiiAtRestColumnsAndEmailHash.Designer.cs new file mode 100644 index 000000000..5387848a5 --- /dev/null +++ b/src/SEBT.Portal.Infrastructure/Migrations/20260504204427_EncryptPiiAtRestColumnsAndEmailHash.Designer.cs @@ -0,0 +1,380 @@ +// +using System; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Infrastructure; +using Microsoft.EntityFrameworkCore.Metadata; +using Microsoft.EntityFrameworkCore.Migrations; +using Microsoft.EntityFrameworkCore.Storage.ValueConversion; +using SEBT.Portal.Infrastructure.Data; + +#nullable disable + +namespace SEBT.Portal.Infrastructure.Migrations +{ + [DbContext(typeof(PortalDbContext))] + [Migration("20260504204427_EncryptPiiAtRestColumnsAndEmailHash")] + partial class EncryptPiiAtRestColumnsAndEmailHash + { + /// + protected override void BuildTargetModel(ModelBuilder modelBuilder) + { +#pragma warning disable 612, 618 + modelBuilder + .HasAnnotation("ProductVersion", "10.0.0") + .HasAnnotation("Relational:MaxIdentifierLength", 128); + + SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.CardReplacementRequestEntity", b => + { + b.Property("Id") + .HasColumnType("uniqueidentifier"); + + b.Property("CaseIdHash") + .IsRequired() + .HasMaxLength(64) + .HasColumnType("nvarchar(64)"); + + b.Property("HouseholdIdentifierHash") + .IsRequired() + .HasMaxLength(64) + .HasColumnType("nvarchar(64)"); + + b.Property("RequestedAt") + .HasColumnType("datetime2"); + + b.Property("RequestedByUserId") + .HasColumnType("uniqueidentifier"); + + b.HasKey("Id"); + + b.HasIndex("RequestedByUserId"); + + b.HasIndex("HouseholdIdentifierHash", "CaseIdHash", "RequestedAt") + .HasDatabaseName("IX_CardReplacementRequests_Household_Case_RequestedAt"); + + b.ToTable("CardReplacementRequests", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.DeidentifiedChildResultEntity", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("uniqueidentifier"); + + b.Property("BirthYear") + .HasColumnType("int"); + + b.Property("EligibilityType") + .HasMaxLength(50) + .HasColumnType("nvarchar(50)"); + + b.Property("SchoolName") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("Status") + .IsRequired() + .HasMaxLength(50) + .HasColumnType("nvarchar(50)"); + + b.Property("SubmissionId") + .HasColumnType("uniqueidentifier"); + + b.HasKey("Id"); + + b.HasIndex("SubmissionId"); + + b.ToTable("DeidentifiedChildResults", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.DocVerificationChallengeEntity", b => + { + b.Property("Id") + .HasColumnType("uniqueidentifier"); + + b.Property("AllowIdRetry") + .ValueGeneratedOnAdd() + .HasColumnType("bit") + .HasDefaultValue(true); + + b.Property("CreatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.Property("DocvTokenIssuedAt") + .HasColumnType("datetime2"); + + b.Property("DocvTransactionToken") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("DocvUrl") + .HasMaxLength(1024) + .HasColumnType("nvarchar(1024)"); + + b.Property("EvalId") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("ExpiresAt") + .HasColumnType("datetime2"); + + b.Property("OffboardingReason") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("ProofingDateOfBirth") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("ProofingIdType") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("ProofingIdValue") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("PublicId") + .HasColumnType("uniqueidentifier"); + + b.Property("RowVersion") + .IsConcurrencyToken() + .ValueGeneratedOnAddOrUpdate() + .HasColumnType("rowversion"); + + b.Property("SocureEventId") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("SocureReferenceId") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("Status") + .ValueGeneratedOnAdd() + .HasColumnType("int") + .HasDefaultValue(0); + + b.Property("UpdatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.Property("UserId") + .HasColumnType("uniqueidentifier"); + + b.HasKey("Id"); + + b.HasIndex("EvalId") + .HasDatabaseName("IX_DocVerificationChallenges_EvalId"); + + b.HasIndex("PublicId") + .IsUnique() + .HasDatabaseName("IX_DocVerificationChallenges_PublicId"); + + b.HasIndex("SocureReferenceId") + .HasDatabaseName("IX_DocVerificationChallenges_SocureReferenceId"); + + b.HasIndex("UserId") + .HasDatabaseName("IX_DocVerificationChallenges_UserId"); + + b.ToTable("DocVerificationChallenges", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.EnrollmentCheckSubmissionEntity", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("uniqueidentifier"); + + b.Property("CheckedAtUtc") + .HasColumnType("datetime2"); + + b.Property("ChildrenChecked") + .HasColumnType("int"); + + b.Property("IpAddressHash") + .HasMaxLength(128) + .HasColumnType("nvarchar(128)"); + + b.HasKey("Id"); + + b.ToTable("EnrollmentCheckSubmissions", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.UserEntity", b => + { + b.Property("Id") + .HasColumnType("uniqueidentifier"); + + b.Property("CoLoadedLastUpdated") + .HasColumnType("datetime2"); + + b.Property("CreatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.Property("DateOfBirth") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("Email") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("EmailHash") + .HasMaxLength(64) + .HasColumnType("nvarchar(64)"); + + b.Property("ExternalProviderId") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("IalLevel") + .ValueGeneratedOnAdd() + .HasColumnType("int") + .HasDefaultValue(0); + + b.Property("IdProofingAttemptCount") + .ValueGeneratedOnAdd() + .HasColumnType("int") + .HasDefaultValue(0); + + b.Property("IdProofingCompletedAt") + .HasColumnType("datetime2"); + + b.Property("IdProofingExpiresAt") + .HasColumnType("datetime2"); + + b.Property("IdProofingSessionId") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("IdProofingStatus") + .ValueGeneratedOnAdd() + .HasColumnType("int") + .HasDefaultValue(0); + + b.Property("IsCoLoaded") + .ValueGeneratedOnAdd() + .HasColumnType("bit") + .HasDefaultValue(false); + + b.Property("Phone") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("SnapId") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("Ssn") + .HasMaxLength(64) + .HasColumnType("nvarchar(64)"); + + b.Property("TanfId") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("UpdatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.HasKey("Id"); + + b.HasIndex("EmailHash") + .IsUnique() + .HasDatabaseName("IX_Users_EmailHash") + .HasFilter("[EmailHash] IS NOT NULL"); + + b.HasIndex("ExternalProviderId") + .IsUnique() + .HasDatabaseName("IX_Users_ExternalProviderId") + .HasFilter("[ExternalProviderId] IS NOT NULL"); + + b.HasIndex("IdProofingSessionId") + .HasDatabaseName("IX_Users_IdProofingSessionId"); + + b.ToTable("Users", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.UserOptInEntity", b => + { + b.Property("Id") + .HasColumnType("uniqueidentifier"); + + b.Property("CreatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.Property("DobOptIn") + .HasColumnType("bit"); + + b.Property("Email") + .IsRequired() + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("EmailOptIn") + .HasColumnType("bit"); + + b.Property("UpdatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.HasKey("Id"); + + b.HasIndex("Email") + .IsUnique(); + + b.ToTable("UserOptIns", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.CardReplacementRequestEntity", b => + { + b.HasOne("SEBT.Portal.Infrastructure.Data.Entities.UserEntity", "RequestedByUser") + .WithMany() + .HasForeignKey("RequestedByUserId") + .OnDelete(DeleteBehavior.Cascade) + .IsRequired(); + + b.Navigation("RequestedByUser"); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.DeidentifiedChildResultEntity", b => + { + b.HasOne("SEBT.Portal.Infrastructure.Data.Entities.EnrollmentCheckSubmissionEntity", "Submission") + .WithMany("ChildResults") + .HasForeignKey("SubmissionId") + .OnDelete(DeleteBehavior.Cascade) + .IsRequired(); + + b.Navigation("Submission"); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.DocVerificationChallengeEntity", b => + { + b.HasOne("SEBT.Portal.Infrastructure.Data.Entities.UserEntity", "User") + .WithMany() + .HasForeignKey("UserId") + .OnDelete(DeleteBehavior.Cascade) + .IsRequired(); + + b.Navigation("User"); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.EnrollmentCheckSubmissionEntity", b => + { + b.Navigation("ChildResults"); + }); +#pragma warning restore 612, 618 + } + } +} diff --git a/src/SEBT.Portal.Infrastructure/Migrations/20260504204427_EncryptPiiAtRestColumnsAndEmailHash.cs b/src/SEBT.Portal.Infrastructure/Migrations/20260504204427_EncryptPiiAtRestColumnsAndEmailHash.cs new file mode 100644 index 000000000..b377d80ee --- /dev/null +++ b/src/SEBT.Portal.Infrastructure/Migrations/20260504204427_EncryptPiiAtRestColumnsAndEmailHash.cs @@ -0,0 +1,150 @@ +using Microsoft.EntityFrameworkCore.Migrations; + +#nullable disable + +namespace SEBT.Portal.Infrastructure.Migrations; + +/// +public partial class EncryptPiiAtRestColumnsAndEmailHash : Migration +{ + /// + protected override void Up(MigrationBuilder migrationBuilder) + { + migrationBuilder.DropIndex( + name: "IX_Users_Email", + table: "Users"); + + migrationBuilder.AlterColumn( + name: "TanfId", + table: "Users", + type: "nvarchar(512)", + maxLength: 512, + nullable: true, + oldClrType: typeof(string), + oldType: "nvarchar(64)", + oldMaxLength: 64, + oldNullable: true); + + migrationBuilder.AlterColumn( + name: "SnapId", + table: "Users", + type: "nvarchar(512)", + maxLength: 512, + nullable: true, + oldClrType: typeof(string), + oldType: "nvarchar(64)", + oldMaxLength: 64, + oldNullable: true); + + migrationBuilder.AlterColumn( + name: "Phone", + table: "Users", + type: "nvarchar(512)", + maxLength: 512, + nullable: true, + oldClrType: typeof(string), + oldType: "nvarchar(64)", + oldMaxLength: 64, + oldNullable: true); + + migrationBuilder.AlterColumn( + name: "Email", + table: "Users", + type: "nvarchar(512)", + maxLength: 512, + nullable: true, + oldClrType: typeof(string), + oldType: "nvarchar(255)", + oldMaxLength: 255, + oldNullable: true); + + MoveUsersDateOfBirthFromDateTypeToNvarchar(migrationBuilder); + + migrationBuilder.AddColumn( + name: "EmailHash", + table: "Users", + type: "nvarchar(64)", + maxLength: 64, + nullable: true); + + migrationBuilder.AlterColumn( + name: "ProofingIdValue", + table: "DocVerificationChallenges", + type: "nvarchar(512)", + maxLength: 512, + nullable: true, + oldClrType: typeof(string), + oldType: "nvarchar(255)", + oldMaxLength: 255, + oldNullable: true); + + migrationBuilder.AlterColumn( + name: "ProofingIdType", + table: "DocVerificationChallenges", + type: "nvarchar(512)", + maxLength: 512, + nullable: true, + oldClrType: typeof(string), + oldType: "nvarchar(64)", + oldMaxLength: 64, + oldNullable: true); + + migrationBuilder.AlterColumn( + name: "ProofingDateOfBirth", + table: "DocVerificationChallenges", + type: "nvarchar(512)", + maxLength: 512, + nullable: true, + oldClrType: typeof(string), + oldType: "nvarchar(32)", + oldMaxLength: 32, + oldNullable: true); + + migrationBuilder.CreateIndex( + name: "IX_Users_EmailHash", + table: "Users", + column: "EmailHash", + unique: true, + filter: "[EmailHash] IS NOT NULL"); + } + + /// + protected override void Down(MigrationBuilder migrationBuilder) => + throw new NotSupportedException( + "Reverting EncryptPiiAtRestColumnsAndEmailHash drops ciphertext columns and would lose data — not supported."); + + /// + /// SQL Server cannot always widen date → ciphertext nvarchar in one ALTER; copy via temp column (idempotent). + /// + private static void MoveUsersDateOfBirthFromDateTypeToNvarchar(MigrationBuilder migrationBuilder) + { + migrationBuilder.Sql( + """ + IF COL_LENGTH(N'dbo.Users', N'DateOfBirth') IS NOT NULL + BEGIN + DECLARE @dtype sysname = + (SELECT DATA_TYPE FROM INFORMATION_SCHEMA.COLUMNS + WHERE TABLE_SCHEMA = N'dbo' AND TABLE_NAME = N'Users' AND COLUMN_NAME = N'DateOfBirth'); + IF @dtype IN (N'date', N'datetime2', N'datetime') + BEGIN + IF COL_LENGTH(N'dbo.Users', N'__pii_legacy_dob_nvarchar') IS NULL + ALTER TABLE [dbo].[Users] ADD [__pii_legacy_dob_nvarchar] nvarchar(512) NULL; + EXEC(N' + UPDATE [dbo].[Users] + SET [__pii_legacy_dob_nvarchar] = CONVERT(char(10), [DateOfBirth], 126) + WHERE [DateOfBirth] IS NOT NULL + '); + IF COL_LENGTH(N'dbo.Users', N'DateOfBirth') IS NOT NULL + ALTER TABLE [dbo].[Users] DROP COLUMN [DateOfBirth]; + IF COL_LENGTH(N'dbo.Users', N'__pii_legacy_dob_nvarchar') IS NOT NULL + AND COL_LENGTH(N'dbo.Users', N'DateOfBirth') IS NULL + EXEC sp_rename N'dbo.Users.__pii_legacy_dob_nvarchar', N'DateOfBirth', N'COLUMN'; + END + ELSE IF @dtype IS NOT NULL AND @dtype NOT IN (N'nvarchar', N'varchar', N'nchar', N'char') + BEGIN + THROW 50001, 'Unexpected Users.DateOfBirth type during PII encryption migration.', 1; + END + END + """); + } +} diff --git a/src/SEBT.Portal.Infrastructure/Migrations/20260508210456_AddUsersEmailLegacyLookupIndex.Designer.cs b/src/SEBT.Portal.Infrastructure/Migrations/20260508210456_AddUsersEmailLegacyLookupIndex.Designer.cs new file mode 100644 index 000000000..3e67f9a63 --- /dev/null +++ b/src/SEBT.Portal.Infrastructure/Migrations/20260508210456_AddUsersEmailLegacyLookupIndex.Designer.cs @@ -0,0 +1,384 @@ +// +using System; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Infrastructure; +using Microsoft.EntityFrameworkCore.Metadata; +using Microsoft.EntityFrameworkCore.Migrations; +using Microsoft.EntityFrameworkCore.Storage.ValueConversion; +using SEBT.Portal.Infrastructure.Data; + +#nullable disable + +namespace SEBT.Portal.Infrastructure.Migrations +{ + [DbContext(typeof(PortalDbContext))] + [Migration("20260508210456_AddUsersEmailLegacyLookupIndex")] + partial class AddUsersEmailLegacyLookupIndex + { + /// + protected override void BuildTargetModel(ModelBuilder modelBuilder) + { +#pragma warning disable 612, 618 + modelBuilder + .HasAnnotation("ProductVersion", "10.0.0") + .HasAnnotation("Relational:MaxIdentifierLength", 128); + + SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.CardReplacementRequestEntity", b => + { + b.Property("Id") + .HasColumnType("uniqueidentifier"); + + b.Property("CaseIdHash") + .IsRequired() + .HasMaxLength(64) + .HasColumnType("nvarchar(64)"); + + b.Property("HouseholdIdentifierHash") + .IsRequired() + .HasMaxLength(64) + .HasColumnType("nvarchar(64)"); + + b.Property("RequestedAt") + .HasColumnType("datetime2"); + + b.Property("RequestedByUserId") + .HasColumnType("uniqueidentifier"); + + b.HasKey("Id"); + + b.HasIndex("RequestedByUserId"); + + b.HasIndex("HouseholdIdentifierHash", "CaseIdHash", "RequestedAt") + .HasDatabaseName("IX_CardReplacementRequests_Household_Case_RequestedAt"); + + b.ToTable("CardReplacementRequests", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.DeidentifiedChildResultEntity", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("uniqueidentifier"); + + b.Property("BirthYear") + .HasColumnType("int"); + + b.Property("EligibilityType") + .HasMaxLength(50) + .HasColumnType("nvarchar(50)"); + + b.Property("SchoolName") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("Status") + .IsRequired() + .HasMaxLength(50) + .HasColumnType("nvarchar(50)"); + + b.Property("SubmissionId") + .HasColumnType("uniqueidentifier"); + + b.HasKey("Id"); + + b.HasIndex("SubmissionId"); + + b.ToTable("DeidentifiedChildResults", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.DocVerificationChallengeEntity", b => + { + b.Property("Id") + .HasColumnType("uniqueidentifier"); + + b.Property("AllowIdRetry") + .ValueGeneratedOnAdd() + .HasColumnType("bit") + .HasDefaultValue(true); + + b.Property("CreatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.Property("DocvTokenIssuedAt") + .HasColumnType("datetime2"); + + b.Property("DocvTransactionToken") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("DocvUrl") + .HasMaxLength(1024) + .HasColumnType("nvarchar(1024)"); + + b.Property("EvalId") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("ExpiresAt") + .HasColumnType("datetime2"); + + b.Property("OffboardingReason") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("ProofingDateOfBirth") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("ProofingIdType") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("ProofingIdValue") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("PublicId") + .HasColumnType("uniqueidentifier"); + + b.Property("RowVersion") + .IsConcurrencyToken() + .ValueGeneratedOnAddOrUpdate() + .HasColumnType("rowversion"); + + b.Property("SocureEventId") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("SocureReferenceId") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("Status") + .ValueGeneratedOnAdd() + .HasColumnType("int") + .HasDefaultValue(0); + + b.Property("UpdatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.Property("UserId") + .HasColumnType("uniqueidentifier"); + + b.HasKey("Id"); + + b.HasIndex("EvalId") + .HasDatabaseName("IX_DocVerificationChallenges_EvalId"); + + b.HasIndex("PublicId") + .IsUnique() + .HasDatabaseName("IX_DocVerificationChallenges_PublicId"); + + b.HasIndex("SocureReferenceId") + .HasDatabaseName("IX_DocVerificationChallenges_SocureReferenceId"); + + b.HasIndex("UserId") + .HasDatabaseName("IX_DocVerificationChallenges_UserId"); + + b.ToTable("DocVerificationChallenges", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.EnrollmentCheckSubmissionEntity", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("uniqueidentifier"); + + b.Property("CheckedAtUtc") + .HasColumnType("datetime2"); + + b.Property("ChildrenChecked") + .HasColumnType("int"); + + b.Property("IpAddressHash") + .HasMaxLength(128) + .HasColumnType("nvarchar(128)"); + + b.HasKey("Id"); + + b.ToTable("EnrollmentCheckSubmissions", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.UserEntity", b => + { + b.Property("Id") + .HasColumnType("uniqueidentifier"); + + b.Property("CoLoadedLastUpdated") + .HasColumnType("datetime2"); + + b.Property("CreatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.Property("DateOfBirth") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("Email") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("EmailHash") + .HasMaxLength(64) + .HasColumnType("nvarchar(64)"); + + b.Property("ExternalProviderId") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("IalLevel") + .ValueGeneratedOnAdd() + .HasColumnType("int") + .HasDefaultValue(0); + + b.Property("IdProofingAttemptCount") + .ValueGeneratedOnAdd() + .HasColumnType("int") + .HasDefaultValue(0); + + b.Property("IdProofingCompletedAt") + .HasColumnType("datetime2"); + + b.Property("IdProofingExpiresAt") + .HasColumnType("datetime2"); + + b.Property("IdProofingSessionId") + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("IdProofingStatus") + .ValueGeneratedOnAdd() + .HasColumnType("int") + .HasDefaultValue(0); + + b.Property("IsCoLoaded") + .ValueGeneratedOnAdd() + .HasColumnType("bit") + .HasDefaultValue(false); + + b.Property("Phone") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("SnapId") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("Ssn") + .HasMaxLength(64) + .HasColumnType("nvarchar(64)"); + + b.Property("TanfId") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("UpdatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.HasKey("Id"); + + b.HasIndex("Email") + .HasDatabaseName("IX_Users_Email_LegacyLookup") + .HasFilter("[EmailHash] IS NULL AND [Email] IS NOT NULL"); + + b.HasIndex("EmailHash") + .IsUnique() + .HasDatabaseName("IX_Users_EmailHash") + .HasFilter("[EmailHash] IS NOT NULL"); + + b.HasIndex("ExternalProviderId") + .IsUnique() + .HasDatabaseName("IX_Users_ExternalProviderId") + .HasFilter("[ExternalProviderId] IS NOT NULL"); + + b.HasIndex("IdProofingSessionId") + .HasDatabaseName("IX_Users_IdProofingSessionId"); + + b.ToTable("Users", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.UserOptInEntity", b => + { + b.Property("Id") + .HasColumnType("uniqueidentifier"); + + b.Property("CreatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.Property("DobOptIn") + .HasColumnType("bit"); + + b.Property("Email") + .IsRequired() + .HasMaxLength(255) + .HasColumnType("nvarchar(255)"); + + b.Property("EmailOptIn") + .HasColumnType("bit"); + + b.Property("UpdatedAt") + .ValueGeneratedOnAdd() + .HasColumnType("datetime2") + .HasDefaultValueSql("GETUTCDATE()"); + + b.HasKey("Id"); + + b.HasIndex("Email") + .IsUnique(); + + b.ToTable("UserOptIns", (string)null); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.CardReplacementRequestEntity", b => + { + b.HasOne("SEBT.Portal.Infrastructure.Data.Entities.UserEntity", "RequestedByUser") + .WithMany() + .HasForeignKey("RequestedByUserId") + .OnDelete(DeleteBehavior.Cascade) + .IsRequired(); + + b.Navigation("RequestedByUser"); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.DeidentifiedChildResultEntity", b => + { + b.HasOne("SEBT.Portal.Infrastructure.Data.Entities.EnrollmentCheckSubmissionEntity", "Submission") + .WithMany("ChildResults") + .HasForeignKey("SubmissionId") + .OnDelete(DeleteBehavior.Cascade) + .IsRequired(); + + b.Navigation("Submission"); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.DocVerificationChallengeEntity", b => + { + b.HasOne("SEBT.Portal.Infrastructure.Data.Entities.UserEntity", "User") + .WithMany() + .HasForeignKey("UserId") + .OnDelete(DeleteBehavior.Cascade) + .IsRequired(); + + b.Navigation("User"); + }); + + modelBuilder.Entity("SEBT.Portal.Infrastructure.Data.Entities.EnrollmentCheckSubmissionEntity", b => + { + b.Navigation("ChildResults"); + }); +#pragma warning restore 612, 618 + } + } +} diff --git a/src/SEBT.Portal.Infrastructure/Migrations/20260508210456_AddUsersEmailLegacyLookupIndex.cs b/src/SEBT.Portal.Infrastructure/Migrations/20260508210456_AddUsersEmailLegacyLookupIndex.cs new file mode 100644 index 000000000..693ec18a1 --- /dev/null +++ b/src/SEBT.Portal.Infrastructure/Migrations/20260508210456_AddUsersEmailLegacyLookupIndex.cs @@ -0,0 +1,28 @@ +using Microsoft.EntityFrameworkCore.Migrations; + +#nullable disable + +namespace SEBT.Portal.Infrastructure.Migrations +{ + /// + public partial class AddUsersEmailLegacyLookupIndex : Migration + { + /// + protected override void Up(MigrationBuilder migrationBuilder) + { + migrationBuilder.CreateIndex( + name: "IX_Users_Email_LegacyLookup", + table: "Users", + column: "Email", + filter: "[EmailHash] IS NULL AND [Email] IS NOT NULL"); + } + + /// + protected override void Down(MigrationBuilder migrationBuilder) + { + migrationBuilder.DropIndex( + name: "IX_Users_Email_LegacyLookup", + table: "Users"); + } + } +} diff --git a/src/SEBT.Portal.Infrastructure/Migrations/PortalDbContextModelSnapshot.cs b/src/SEBT.Portal.Infrastructure/Migrations/PortalDbContextModelSnapshot.cs index b573b995e..1ed3ec9e6 100644 --- a/src/SEBT.Portal.Infrastructure/Migrations/PortalDbContextModelSnapshot.cs +++ b/src/SEBT.Portal.Infrastructure/Migrations/PortalDbContextModelSnapshot.cs @@ -123,16 +123,16 @@ protected override void BuildModel(ModelBuilder modelBuilder) .HasColumnType("nvarchar(255)"); b.Property("ProofingDateOfBirth") - .HasMaxLength(32) - .HasColumnType("nvarchar(32)"); + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); b.Property("ProofingIdType") - .HasMaxLength(64) - .HasColumnType("nvarchar(64)"); + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); b.Property("ProofingIdValue") - .HasMaxLength(255) - .HasColumnType("nvarchar(255)"); + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); b.Property("PublicId") .HasColumnType("uniqueidentifier"); @@ -215,12 +215,17 @@ protected override void BuildModel(ModelBuilder modelBuilder) .HasColumnType("datetime2") .HasDefaultValueSql("GETUTCDATE()"); - b.Property("DateOfBirth") - .HasColumnType("date"); + b.Property("DateOfBirth") + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); b.Property("Email") - .HasMaxLength(255) - .HasColumnType("nvarchar(255)"); + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); + + b.Property("EmailHash") + .HasMaxLength(64) + .HasColumnType("nvarchar(64)"); b.Property("ExternalProviderId") .HasMaxLength(255) @@ -257,20 +262,20 @@ protected override void BuildModel(ModelBuilder modelBuilder) .HasDefaultValue(false); b.Property("Phone") - .HasMaxLength(64) - .HasColumnType("nvarchar(64)"); + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); b.Property("SnapId") - .HasMaxLength(64) - .HasColumnType("nvarchar(64)"); + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); b.Property("Ssn") .HasMaxLength(64) .HasColumnType("nvarchar(64)"); b.Property("TanfId") - .HasMaxLength(64) - .HasColumnType("nvarchar(64)"); + .HasMaxLength(512) + .HasColumnType("nvarchar(512)"); b.Property("UpdatedAt") .ValueGeneratedOnAdd() @@ -280,9 +285,13 @@ protected override void BuildModel(ModelBuilder modelBuilder) b.HasKey("Id"); b.HasIndex("Email") + .HasDatabaseName("IX_Users_Email_LegacyLookup") + .HasFilter("[EmailHash] IS NULL AND [Email] IS NOT NULL"); + + b.HasIndex("EmailHash") .IsUnique() - .HasDatabaseName("IX_Users_Email") - .HasFilter("[Email] IS NOT NULL"); + .HasDatabaseName("IX_Users_EmailHash") + .HasFilter("[EmailHash] IS NOT NULL"); b.HasIndex("ExternalProviderId") .IsUnique() diff --git a/src/SEBT.Portal.Infrastructure/Repositories/DatabaseDocVerificationChallengeRepository.cs b/src/SEBT.Portal.Infrastructure/Repositories/DatabaseDocVerificationChallengeRepository.cs index 1f3e00968..5de4fc494 100644 --- a/src/SEBT.Portal.Infrastructure/Repositories/DatabaseDocVerificationChallengeRepository.cs +++ b/src/SEBT.Portal.Infrastructure/Repositories/DatabaseDocVerificationChallengeRepository.cs @@ -4,6 +4,7 @@ using SEBT.Portal.Core.Repositories; using SEBT.Portal.Infrastructure.Data; using SEBT.Portal.Infrastructure.Data.Entities; +using SEBT.Portal.Core.Services; namespace SEBT.Portal.Infrastructure.Repositories; @@ -11,7 +12,9 @@ namespace SEBT.Portal.Infrastructure.Repositories; /// Database-backed implementation of using Entity Framework Core. /// All read operations are scoped by userId to enforce ownership. /// -public class DatabaseDocVerificationChallengeRepository(PortalDbContext dbContext) +public class DatabaseDocVerificationChallengeRepository( + PortalDbContext dbContext, + IPiiSymmetricEncryption piiSymmetricEncryption) : IDocVerificationChallengeRepository { private const string OneActivePerUserIndex = "IX_DocVerificationChallenges_OneActivePerUser"; @@ -27,7 +30,7 @@ public class DatabaseDocVerificationChallengeRepository(PortalDbContext dbContex c => c.PublicId == publicId && c.UserId == userId, cancellationToken); - return entity == null ? null : MapToDomainModel(entity); + return entity == null ? null : MapToDomainModel(entity, piiSymmetricEncryption); } public async Task GetActiveByUserIdAsync( @@ -44,7 +47,7 @@ public class DatabaseDocVerificationChallengeRepository(PortalDbContext dbContex && (c.ExpiresAt == null || c.ExpiresAt > DateTime.UtcNow), cancellationToken); - return entity == null ? null : MapToDomainModel(entity); + return entity == null ? null : MapToDomainModel(entity, piiSymmetricEncryption); } public async Task GetBySocureReferenceIdAsync( @@ -60,7 +63,7 @@ public class DatabaseDocVerificationChallengeRepository(PortalDbContext dbContex .AsNoTracking() .FirstOrDefaultAsync(c => c.SocureReferenceId == referenceId, cancellationToken); - return entity == null ? null : MapToDomainModel(entity); + return entity == null ? null : MapToDomainModel(entity, piiSymmetricEncryption); } public async Task GetByEvalIdAsync( @@ -76,7 +79,7 @@ public class DatabaseDocVerificationChallengeRepository(PortalDbContext dbContex .AsNoTracking() .FirstOrDefaultAsync(c => c.EvalId == evalId, cancellationToken); - return entity == null ? null : MapToDomainModel(entity); + return entity == null ? null : MapToDomainModel(entity, piiSymmetricEncryption); } public async Task CreateAsync( @@ -112,7 +115,7 @@ await dbContext.DocVerificationChallenges .SetProperty(e => e.UpdatedAt, now), cancellationToken); - var entity = MapToEntity(challenge); + var entity = MapToEntity(challenge, piiSymmetricEncryption); dbContext.DocVerificationChallenges.Add(entity); try @@ -161,9 +164,11 @@ public async Task UpdateAsync( entity.OffboardingReason = challenge.OffboardingReason; entity.AllowIdRetry = challenge.AllowIdRetry; entity.ExpiresAt = challenge.ExpiresAt; - entity.ProofingDateOfBirth = challenge.ProofingDateOfBirth; - entity.ProofingIdType = challenge.ProofingIdType; - entity.ProofingIdValue = challenge.ProofingIdValue; + entity.ProofingDateOfBirth = + piiSymmetricEncryption.Encrypt(challenge.ProofingDateOfBirth); + entity.ProofingIdType = piiSymmetricEncryption.Encrypt(challenge.ProofingIdType); + entity.ProofingIdValue = + piiSymmetricEncryption.Encrypt(challenge.ProofingIdValue); entity.DocvTokenIssuedAt = challenge.DocvTokenIssuedAt; entity.UpdatedAt = DateTime.UtcNow; @@ -178,7 +183,9 @@ public async Task UpdateAsync( } } - private static DocVerificationChallenge MapToDomainModel(DocVerificationChallengeEntity entity) + private static DocVerificationChallenge MapToDomainModel( + DocVerificationChallengeEntity entity, + IPiiSymmetricEncryption crypto) { return DocVerificationChallenge.Reconstitute( id: entity.Id, @@ -195,13 +202,17 @@ private static DocVerificationChallenge MapToDomainModel(DocVerificationChalleng createdAt: entity.CreatedAt, updatedAt: entity.UpdatedAt, expiresAt: entity.ExpiresAt, - proofingDateOfBirth: entity.ProofingDateOfBirth, - proofingIdType: entity.ProofingIdType, - proofingIdValue: entity.ProofingIdValue, + proofingDateOfBirth: + crypto.DecryptOrPassThroughLegacy(entity.ProofingDateOfBirth), + proofingIdType: crypto.DecryptOrPassThroughLegacy(entity.ProofingIdType), + proofingIdValue: + crypto.DecryptOrPassThroughLegacy(entity.ProofingIdValue), docvTokenIssuedAt: entity.DocvTokenIssuedAt); } - private static DocVerificationChallengeEntity MapToEntity(DocVerificationChallenge challenge) + private static DocVerificationChallengeEntity MapToEntity( + DocVerificationChallenge challenge, + IPiiSymmetricEncryption crypto) { return new DocVerificationChallengeEntity { @@ -219,9 +230,9 @@ private static DocVerificationChallengeEntity MapToEntity(DocVerificationChallen CreatedAt = challenge.CreatedAt, UpdatedAt = challenge.UpdatedAt, ExpiresAt = challenge.ExpiresAt, - ProofingDateOfBirth = challenge.ProofingDateOfBirth, - ProofingIdType = challenge.ProofingIdType, - ProofingIdValue = challenge.ProofingIdValue, + ProofingDateOfBirth = crypto.Encrypt(challenge.ProofingDateOfBirth), + ProofingIdType = crypto.Encrypt(challenge.ProofingIdType), + ProofingIdValue = crypto.Encrypt(challenge.ProofingIdValue), DocvTokenIssuedAt = challenge.DocvTokenIssuedAt }; } diff --git a/src/SEBT.Portal.Infrastructure/Repositories/DatabaseUserRepository.cs b/src/SEBT.Portal.Infrastructure/Repositories/DatabaseUserRepository.cs index e5e97af61..7cd87484f 100644 --- a/src/SEBT.Portal.Infrastructure/Repositories/DatabaseUserRepository.cs +++ b/src/SEBT.Portal.Infrastructure/Repositories/DatabaseUserRepository.cs @@ -2,29 +2,45 @@ using SEBT.Portal.Core.Models.Auth; using SEBT.Portal.Core.Repositories; using SEBT.Portal.Core.Services; +using SEBT.Portal.Core.Utilities; using SEBT.Portal.Infrastructure.Data; using SEBT.Portal.Infrastructure.Data.Entities; +using SEBT.Portal.Infrastructure.Services; namespace SEBT.Portal.Infrastructure.Repositories; /// /// Database-backed implementation of using Entity Framework Core. /// -public class DatabaseUserRepository(PortalDbContext dbContext, IIdentifierHasher identifierHasher) : IUserRepository +public class DatabaseUserRepository( + PortalDbContext dbContext, + IIdentifierHasher identifierHasher, + IPiiSymmetricEncryption piiEncryption, + IEmailLookupHasher emailLookupHasher) : IUserRepository { public async Task GetUserByEmailAsync(string email, CancellationToken cancellationToken = default) { - if (string.IsNullOrWhiteSpace(email)) + var normalizedEmail = NormalizeEmail(email); + if (normalizedEmail == null) + { + return null; + } + + var lookupHash = emailLookupHasher.HashNormalized(normalizedEmail); + if (lookupHash == null) { return null; } - var normalizedEmail = NormalizeEmail(email); var entity = await dbContext.Users .AsNoTracking() - .FirstOrDefaultAsync(u => u.Email == normalizedEmail, cancellationToken); + .FirstOrDefaultAsync( + u => + u.EmailHash == lookupHash || + u.EmailHash == null && u.Email != null && u.Email == normalizedEmail, + cancellationToken); - return entity == null ? null : MapToDomainModel(entity); + return entity == null ? null : UserEncryptedFieldMapper.ToDomain(entity, piiEncryption); } public async Task CreateUserAsync(User user, CancellationToken cancellationToken = default) @@ -34,20 +50,17 @@ public async Task CreateUserAsync(User user, CancellationToken cancellationToken throw new ArgumentNullException(nameof(user)); } - // OTP users must have an email; OIDC users must have an ExternalProviderId. - // At least one identifier is required. if (string.IsNullOrWhiteSpace(user.Email) && string.IsNullOrWhiteSpace(user.ExternalProviderId)) { throw new ArgumentException( "Either Email or ExternalProviderId must be provided.", nameof(user)); } - var entity = MapToEntity(user); - // Normalize email to lowercase for consistent storage (when present) - if (entity.Email != null) - { - entity.Email = NormalizeEmail(entity.Email); - } + var entity = NewTrackedEntityStructural(user); + UserEncryptedFieldMapper.EncryptIdentifiers( + entity, user, piiEncryption, emailLookupHasher, includeEmailColumns: true); + entity.Ssn = identifierHasher.HashForStorage(user.Ssn); + dbContext.Users.Add(entity); await dbContext.SaveChangesAsync(cancellationToken); } @@ -72,29 +85,43 @@ public async Task UpdateUserAsync(User user, CancellationToken cancellationToken throw new InvalidOperationException($"User with Id {user.Id} not found."); } - // Update email only when the caller provides one (OTP users). - // OIDC users have null email — leave the DB value unchanged in that case. - if (user.Email != null) + if (NormalizeEmail(user.Email) != null) { - var normalizedEmail = NormalizeEmail(user.Email); - if (entity.Email != normalizedEmail) + var normalizedIncoming = NormalizeEmail(user.Email)!; + var envelopePrefix = PiiAesGcmSymmetricEncryption.EnvelopePrefix; + var plaintextDupe = await dbContext.Users.AnyAsync( + u => + u.Id != user.Id && + u.EmailHash == null && + u.Email != null && + !u.Email.StartsWith(envelopePrefix) && + u.Email == normalizedIncoming, + cancellationToken); + + if (plaintextDupe) { - entity.Email = normalizedEmail; + throw new InvalidOperationException("A user with this email address already exists."); } } - // Update properties + if (user.Email != null) + { + UserEncryptedFieldMapper.EncryptIdentifiers( + entity, user, piiEncryption, emailLookupHasher, includeEmailColumns: true); + } + else + { + UserEncryptedFieldMapper.EncryptIdentifiers( + entity, user, piiEncryption, emailLookupHasher, includeEmailColumns: false); + } + entity.IdProofingStatus = (int)user.IdProofingStatus; entity.IalLevel = (int)user.IalLevel; entity.IdProofingSessionId = user.IdProofingSessionId; entity.IdProofingCompletedAt = user.IdProofingCompletedAt; entity.IdProofingExpiresAt = user.IdProofingExpiresAt; - entity.DateOfBirth = user.DateOfBirth; entity.IsCoLoaded = user.IsCoLoaded; entity.CoLoadedLastUpdated = user.CoLoadedLastUpdated; - entity.Phone = user.Phone; - entity.SnapId = user.SnapId; - entity.TanfId = user.TanfId; entity.Ssn = identifierHasher.HashForStorage(user.Ssn); entity.IdProofingAttemptCount = user.IdProofingAttemptCount; entity.UpdatedAt = DateTime.UtcNow; @@ -105,47 +132,55 @@ public async Task UpdateUserAsync(User user, CancellationToken cancellationToken } catch (DbUpdateException ex) { - // Handle unique constraint violation for email (race condition or duplicate email) if (ex.InnerException?.Message.Contains("UNIQUE") == true || ex.InnerException?.Message.Contains("duplicate key") == true || - ex.InnerException?.Message.Contains("IX_Users_Email") == true) + ex.InnerException?.Message.Contains("IX_Users_EmailHash") == true) { - throw new InvalidOperationException($"A user with email {user.Email} already exists.", ex); + throw new InvalidOperationException("A user with this email address already exists.", ex); } - // Re-throw if it's not a unique constraint violation throw; } } - public async Task<(User user, bool isNewUser)> GetOrCreateUserAsync(string email, CancellationToken cancellationToken = default) + public async Task<(User user, bool isNewUser)> GetOrCreateUserAsync( + string email, + CancellationToken cancellationToken = default) { if (string.IsNullOrWhiteSpace(email)) { throw new ArgumentException("Email cannot be null or empty.", nameof(email)); } - var normalizedEmail = NormalizeEmail(email); + var normalizedEmail = NormalizeEmail(email)!; + var lookupHash = emailLookupHasher.HashNormalized(normalizedEmail)!; + var entity = await dbContext.Users - .FirstOrDefaultAsync(u => u.Email == normalizedEmail, cancellationToken); + .FirstOrDefaultAsync( + u => + u.EmailHash == lookupHash || + u.EmailHash == null && u.Email != null && u.Email == normalizedEmail, + cancellationToken); if (entity != null) { - return (MapToDomainModel(entity), false); + return (UserEncryptedFieldMapper.ToDomain(entity, piiEncryption), false); } - // Create new user with normalized email - // (Id defaults to Guid.CreateVersion7() on the entity; no need to set it explicitly.) - var newEntity = new UserEntity + var draftUser = new User { Email = normalizedEmail, - IdProofingStatus = (int)IdProofingStatus.NotStarted, - IalLevel = (int)UserIalLevel.None, + IdProofingStatus = IdProofingStatus.NotStarted, + IalLevel = UserIalLevel.None, IsCoLoaded = false, CreatedAt = DateTime.UtcNow, UpdatedAt = DateTime.UtcNow }; + var newEntity = NewTrackedEntityStructural(draftUser); + UserEncryptedFieldMapper.EncryptIdentifiers( + newEntity, draftUser, piiEncryption, emailLookupHasher, includeEmailColumns: true); + dbContext.Users.Add(newEntity); try @@ -154,27 +189,26 @@ public async Task UpdateUserAsync(User user, CancellationToken cancellationToken } catch (DbUpdateException ex) { - // Handle race condition: if another request created the user between our check and save, - // retry by fetching the existing user if (ex.InnerException?.Message.Contains("PRIMARY KEY") == true || ex.InnerException?.Message.Contains("UNIQUE") == true || ex.InnerException?.Message.Contains("duplicate key") == true) { - // User was created by another request, fetch it - entity = await dbContext.Users - .FirstOrDefaultAsync(u => u.Email == normalizedEmail, cancellationToken); + entity = await dbContext.Users.FirstOrDefaultAsync( + u => + u.EmailHash == lookupHash || + u.EmailHash == null && u.Email != null && u.Email == normalizedEmail, + cancellationToken); if (entity != null) { - return (MapToDomainModel(entity), false); + return (UserEncryptedFieldMapper.ToDomain(entity, piiEncryption), false); } } - // Re-throw if it's not a duplicate key violation throw; } - return (MapToDomainModel(newEntity), true); + return (UserEncryptedFieldMapper.ToDomain(newEntity, piiEncryption), true); } public async Task GetUserBySessionIdAsync(string sessionId, CancellationToken cancellationToken = default) @@ -188,7 +222,7 @@ public async Task UpdateUserAsync(User user, CancellationToken cancellationToken .AsNoTracking() .FirstOrDefaultAsync(u => u.IdProofingSessionId == sessionId, cancellationToken); - return entity == null ? null : MapToDomainModel(entity); + return entity == null ? null : UserEncryptedFieldMapper.ToDomain(entity, piiEncryption); } public async Task GetUserByIdAsync(Guid id, CancellationToken cancellationToken = default) @@ -202,11 +236,10 @@ public async Task UpdateUserAsync(User user, CancellationToken cancellationToken .AsNoTracking() .FirstOrDefaultAsync(u => u.Id == id, cancellationToken); - return entity == null ? null : MapToDomainModel(entity); + return entity == null ? null : UserEncryptedFieldMapper.ToDomain(entity, piiEncryption); } - public async Task GetUserByExternalIdAsync( - string externalProviderId, CancellationToken cancellationToken = default) + public async Task GetUserByExternalIdAsync(string externalProviderId, CancellationToken cancellationToken = default) { if (string.IsNullOrWhiteSpace(externalProviderId)) { @@ -217,7 +250,7 @@ public async Task UpdateUserAsync(User user, CancellationToken cancellationToken .AsNoTracking() .FirstOrDefaultAsync(u => u.ExternalProviderId == externalProviderId, cancellationToken); - return entity == null ? null : MapToDomainModel(entity); + return entity == null ? null : UserEncryptedFieldMapper.ToDomain(entity, piiEncryption); } public async Task<(User user, bool isNewUser)> GetOrCreateUserByExternalIdAsync( @@ -231,46 +264,51 @@ public async Task UpdateUserAsync(User user, CancellationToken cancellationToken "External provider ID cannot be null or empty.", nameof(externalProviderId)); } - // Primary lookup: by ExternalProviderId (the steady-state path) - var entity = await dbContext.Users - .FirstOrDefaultAsync(u => u.ExternalProviderId == externalProviderId, cancellationToken); + var entity = await dbContext.Users.FirstOrDefaultAsync( + u => u.ExternalProviderId == externalProviderId, + cancellationToken); if (entity != null) { - return (MapToDomainModel(entity), false); + return (UserEncryptedFieldMapper.ToDomain(entity, piiEncryption), false); } - // Migration fallback: if an email hint is provided, check for a legacy - // email-only record and adopt it by setting ExternalProviderId. - // TODO: Remove this fallback once all existing users have logged in - // under the new sub-based identity flow. if (!string.IsNullOrWhiteSpace(email)) { var normalizedEmail = NormalizeEmail(email); - var legacyEntity = await dbContext.Users - .FirstOrDefaultAsync( - u => u.Email == normalizedEmail && u.ExternalProviderId == null, - cancellationToken); + var lookupHash = normalizedEmail != null ? emailLookupHasher.HashNormalized(normalizedEmail) : null; - if (legacyEntity != null) + if (normalizedEmail != null && lookupHash != null) { - legacyEntity.ExternalProviderId = externalProviderId; - legacyEntity.Email = null; // OIDC users derive email from IdP claims, not DB - legacyEntity.UpdatedAt = DateTime.UtcNow; - await dbContext.SaveChangesAsync(cancellationToken); - return (MapToDomainModel(legacyEntity), false); + var legacyEntity = await dbContext.Users.FirstOrDefaultAsync( + u => + u.ExternalProviderId == null && + (u.EmailHash == lookupHash || + u.EmailHash == null && u.Email != null && u.Email == normalizedEmail), + cancellationToken); + + if (legacyEntity != null) + { + legacyEntity.ExternalProviderId = externalProviderId; + UserEncryptedFieldMapper.ClearEmailColumns(legacyEntity); + legacyEntity.UpdatedAt = DateTime.UtcNow; + await dbContext.SaveChangesAsync(cancellationToken); + return (UserEncryptedFieldMapper.ToDomain(legacyEntity, piiEncryption), false); + } } } - // No existing record found — create a new minimal one - // (Id defaults to Guid.CreateVersion7() on the entity; no need to set it explicitly.) - var newEntity = new UserEntity + var draftOidcUser = new User { ExternalProviderId = externalProviderId, CreatedAt = DateTime.UtcNow, UpdatedAt = DateTime.UtcNow }; + var newEntity = NewTrackedEntityStructural(draftOidcUser); + UserEncryptedFieldMapper.EncryptIdentifiers( + newEntity, draftOidcUser, piiEncryption, emailLookupHasher, includeEmailColumns: true); + dbContext.Users.Add(newEntity); try @@ -282,75 +320,36 @@ public async Task UpdateUserAsync(User user, CancellationToken cancellationToken if (ex.InnerException?.Message.Contains("UNIQUE") == true || ex.InnerException?.Message.Contains("duplicate key") == true) { - entity = await dbContext.Users - .FirstOrDefaultAsync( - u => u.ExternalProviderId == externalProviderId, cancellationToken); + entity = await dbContext.Users.FirstOrDefaultAsync( + u => u.ExternalProviderId == externalProviderId, cancellationToken); if (entity != null) { - return (MapToDomainModel(entity), false); + return (UserEncryptedFieldMapper.ToDomain(entity, piiEncryption), false); } } + throw; } - return (MapToDomainModel(newEntity), true); - } - - /// - /// Normalizes an email address to lowercase for consistent storage and comparison. - /// - /// The email address to normalize. - /// The normalized (lowercase) email address. - private static string NormalizeEmail(string email) - { - return email.Trim().ToLowerInvariant(); + return (UserEncryptedFieldMapper.ToDomain(newEntity, piiEncryption), true); } - private static User MapToDomainModel(UserEntity entity) - { - return new User - { - Id = entity.Id, - Email = entity.Email, - ExternalProviderId = entity.ExternalProviderId, - IdProofingStatus = (IdProofingStatus)entity.IdProofingStatus, - IalLevel = (UserIalLevel)entity.IalLevel, - IdProofingSessionId = entity.IdProofingSessionId, - IdProofingCompletedAt = entity.IdProofingCompletedAt, - IdProofingExpiresAt = entity.IdProofingExpiresAt, - DateOfBirth = entity.DateOfBirth, - IsCoLoaded = entity.IsCoLoaded, - CoLoadedLastUpdated = entity.CoLoadedLastUpdated, - Phone = entity.Phone, - SnapId = entity.SnapId, - TanfId = entity.TanfId, - Ssn = entity.Ssn, - IdProofingAttemptCount = entity.IdProofingAttemptCount, - CreatedAt = entity.CreatedAt, - UpdatedAt = entity.UpdatedAt - }; - } + private static string? NormalizeEmail(string? email) => EmailNormalizer.NormalizeOrNull(email); - private UserEntity MapToEntity(User user) + private UserEntity NewTrackedEntityStructural(User user) { return new UserEntity { Id = user.Id, - Email = user.Email, // Will be normalized in calling method ExternalProviderId = user.ExternalProviderId, IdProofingStatus = (int)user.IdProofingStatus, IalLevel = (int)user.IalLevel, IdProofingSessionId = user.IdProofingSessionId, IdProofingCompletedAt = user.IdProofingCompletedAt, IdProofingExpiresAt = user.IdProofingExpiresAt, - DateOfBirth = user.DateOfBirth, IsCoLoaded = user.IsCoLoaded, CoLoadedLastUpdated = user.CoLoadedLastUpdated, - Phone = user.Phone, - SnapId = user.SnapId, - TanfId = user.TanfId, - Ssn = identifierHasher.HashForStorage(user.Ssn), IdProofingAttemptCount = user.IdProofingAttemptCount, CreatedAt = user.CreatedAt, UpdatedAt = user.UpdatedAt diff --git a/src/SEBT.Portal.Infrastructure/Repositories/UserEncryptedFieldMapper.cs b/src/SEBT.Portal.Infrastructure/Repositories/UserEncryptedFieldMapper.cs new file mode 100644 index 000000000..36c018330 --- /dev/null +++ b/src/SEBT.Portal.Infrastructure/Repositories/UserEncryptedFieldMapper.cs @@ -0,0 +1,109 @@ +using System.Globalization; +using SEBT.Portal.Core.Models.Auth; +using SEBT.Portal.Core.Services; +using SEBT.Portal.Core.Utilities; +using SEBT.Portal.Infrastructure.Data.Entities; + +namespace SEBT.Portal.Infrastructure.Repositories; + +internal static class UserEncryptedFieldMapper +{ + internal static User ToDomain(UserEntity entity, IPiiSymmetricEncryption crypto) + { + return new User + { + Id = entity.Id, + Email = DecryptNormalizedEmail(entity, crypto), + ExternalProviderId = entity.ExternalProviderId, + IdProofingStatus = (IdProofingStatus)entity.IdProofingStatus, + IalLevel = (UserIalLevel)entity.IalLevel, + IdProofingSessionId = entity.IdProofingSessionId, + IdProofingCompletedAt = entity.IdProofingCompletedAt, + IdProofingExpiresAt = entity.IdProofingExpiresAt, + DateOfBirth = DecodeDateOnlySafe(crypto, entity.DateOfBirth), + IsCoLoaded = entity.IsCoLoaded, + CoLoadedLastUpdated = entity.CoLoadedLastUpdated, + Phone = crypto.DecryptOrPassThroughLegacy(entity.Phone), + SnapId = crypto.DecryptOrPassThroughLegacy(entity.SnapId), + TanfId = crypto.DecryptOrPassThroughLegacy(entity.TanfId), + Ssn = entity.Ssn, + IdProofingAttemptCount = entity.IdProofingAttemptCount, + CreatedAt = entity.CreatedAt, + UpdatedAt = entity.UpdatedAt + }; + } + + /// + /// When false, OTP leave-behind semantics: callers already loaded the tracked entity and only want household + DOB fields encrypted. + /// + internal static void EncryptIdentifiers( + UserEntity entity, + User user, + IPiiSymmetricEncryption crypto, + IEmailLookupHasher lookup, + bool includeEmailColumns) + { + if (includeEmailColumns) + { + var normalizedEmail = EmailNormalizer.NormalizeOrNull(user.Email); + entity.Email = normalizedEmail == null ? null : crypto.Encrypt(normalizedEmail); + entity.EmailHash = lookup.NormalizeAndHash(user.Email); + } + + entity.Phone = crypto.Encrypt(user.Phone); + entity.SnapId = crypto.Encrypt(user.SnapId); + entity.TanfId = crypto.Encrypt(user.TanfId); + entity.DateOfBirth = crypto.Encrypt( + user.DateOfBirth?.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)); + } + + internal static void ClearEmailColumns(UserEntity entity) + { + entity.Email = null; + entity.EmailHash = null; + } + + private static string? DecryptNormalizedEmail(UserEntity entity, IPiiSymmetricEncryption crypto) + { + if (string.IsNullOrEmpty(entity.Email)) + { + return null; + } + + var plain = crypto.DecryptOrPassThroughLegacy(entity.Email); + return EmailNormalizer.NormalizeOrNull(plain); + } + + private static DateOnly? DecodeDateOnlySafe(IPiiSymmetricEncryption crypto, string? stored) + { + if (string.IsNullOrEmpty(stored)) + { + return null; + } + + var plain = crypto.DecryptOrPassThroughLegacy(stored); + if (string.IsNullOrWhiteSpace(plain)) + { + return null; + } + + plain = plain.Trim(); + if (DateOnly.TryParseExact( + plain, + "yyyy-MM-dd", + CultureInfo.InvariantCulture, + DateTimeStyles.None, + out var exact)) + { + return exact; + } + + return DateOnly.TryParse( + plain, + CultureInfo.InvariantCulture, + DateTimeStyles.AllowWhiteSpaces, + out var general) + ? general + : null; + } +} diff --git a/src/SEBT.Portal.Infrastructure/Services/DataSeeder.cs b/src/SEBT.Portal.Infrastructure/Services/DataSeeder.cs index 26b7687e0..6f16202af 100644 --- a/src/SEBT.Portal.Infrastructure/Services/DataSeeder.cs +++ b/src/SEBT.Portal.Infrastructure/Services/DataSeeder.cs @@ -1,9 +1,11 @@ using Microsoft.EntityFrameworkCore; +using SEBT.Portal.Core.Exceptions; using SEBT.Portal.Core.Models.Auth; using SEBT.Portal.Core.Services; using SEBT.Portal.Core.Utilities; using SEBT.Portal.Infrastructure.Data; using SEBT.Portal.Infrastructure.Data.Entities; +using SEBT.Portal.Infrastructure.Repositories; namespace SEBT.Portal.Infrastructure.Services; @@ -15,74 +17,177 @@ public class DataSeeder : IDataSeeder { private readonly PortalDbContext _dbContext; private readonly IIdentifierHasher _identifierHasher; + private readonly IPiiSymmetricEncryption _piiSymmetricEncryption; + private readonly IEmailLookupHasher _emailLookupHasher; - public DataSeeder(PortalDbContext dbContext, IIdentifierHasher identifierHasher) + public DataSeeder( + PortalDbContext dbContext, + IIdentifierHasher identifierHasher, + IPiiSymmetricEncryption piiSymmetricEncryption, + IEmailLookupHasher emailLookupHasher) { _dbContext = dbContext ?? throw new ArgumentNullException(nameof(dbContext)); _identifierHasher = identifierHasher ?? throw new ArgumentNullException(nameof(identifierHasher)); + _piiSymmetricEncryption = + piiSymmetricEncryption ?? throw new ArgumentNullException(nameof(piiSymmetricEncryption)); + _emailLookupHasher = emailLookupHasher ?? throw new ArgumentNullException(nameof(emailLookupHasher)); + } + + private List FilterOutUsersMatchingExistingEmails(List usersList) + { + var keyed = usersList + .Select(u => (User: u, Normalized: EmailNormalizer.NormalizeOrNull(u.Email))) + .Where(x => x.Normalized != null) + .ToList(); + + if (keyed.Count == 0) + { + return []; + } + + var distinct = keyed.Select(x => x.Normalized!).Distinct().ToList(); + var existing = GetExistingUserEmails(distinct); + return keyed + .Where(x => !existing.Contains(x.Normalized!, StringComparer.Ordinal)) + .Select(x => x.User) + .ToList(); + } + + private async Task> FilterOutUsersMatchingExistingEmailsAsync( + List usersList, + CancellationToken cancellationToken) + { + var keyed = usersList + .Select(u => (User: u, Normalized: EmailNormalizer.NormalizeOrNull(u.Email))) + .Where(x => x.Normalized != null) + .ToList(); + + if (keyed.Count == 0) + { + return []; + } + + var distinct = keyed.Select(x => x.Normalized!).Distinct().ToList(); + var existing = await GetExistingUserEmailsAsync(distinct, cancellationToken); + return keyed + .Where(x => !existing.Contains(x.Normalized!, StringComparer.Ordinal)) + .Select(x => x.User) + .ToList(); } - /// - /// Maps a User domain model to a UserEntity for database persistence. - /// private UserEntity MapToEntity(User user) { ArgumentNullException.ThrowIfNull(user); - var normalizedEmail = user.Email != null ? EmailNormalizer.Normalize(user.Email) : null; - return new UserEntity + var entity = new UserEntity { - Id = user.Id, // Will be 0 for new users, set by database - Email = normalizedEmail, + Id = user.Id, ExternalProviderId = user.ExternalProviderId, IdProofingStatus = (int)user.IdProofingStatus, IalLevel = (int)user.IalLevel, IdProofingSessionId = user.IdProofingSessionId, IdProofingCompletedAt = user.IdProofingCompletedAt, IdProofingExpiresAt = user.IdProofingExpiresAt, - DateOfBirth = user.DateOfBirth, IsCoLoaded = user.IsCoLoaded, CoLoadedLastUpdated = user.CoLoadedLastUpdated, - Phone = user.Phone, - SnapId = user.SnapId, - TanfId = user.TanfId, - Ssn = _identifierHasher.HashForStorage(user.Ssn), + IdProofingAttemptCount = user.IdProofingAttemptCount, CreatedAt = user.CreatedAt, - UpdatedAt = user.UpdatedAt + UpdatedAt = user.UpdatedAt, + Ssn = _identifierHasher.HashForStorage(user.Ssn) }; + + UserEncryptedFieldMapper.EncryptIdentifiers( + entity, + user, + _piiSymmetricEncryption, + _emailLookupHasher, + includeEmailColumns: true); + + return entity; } - /// - /// Handles DbUpdateException for unique constraint violations that may occur during seeding. - /// Returns true if the exception was handled (duplicate key violation), false otherwise. - /// private static bool HandleDuplicateKeyException(DbUpdateException ex) { if (ex.InnerException?.Message.Contains("UNIQUE") == true || ex.InnerException?.Message.Contains("duplicate key") == true || + ex.InnerException?.Message.Contains("IX_Users_EmailHash") == true || ex.InnerException?.Message.Contains("IX_Users_Email") == true) { - // Users may have been created by another process return true; } + return false; } + /// Decrypt failures are swallowed — callers rely on hashing + legacy plaintext lookups. + private string? TryHydrateNormalizedEmail(string? ciphertext) + { + if (string.IsNullOrEmpty(ciphertext)) + { + return null; + } + + try + { + var plain = _piiSymmetricEncryption.DecryptOrPassThroughLegacy(ciphertext); + return EmailNormalizer.NormalizeOrNull(plain); + } + catch (PiiDecryptException) + { + return null; + } + } + public async Task AnyUsersExistAsync(CancellationToken cancellationToken = default) { return await _dbContext.Users.AnyAsync(cancellationToken); } - public async Task> GetExistingUserEmailsAsync(IEnumerable emails, CancellationToken cancellationToken = default) + public async Task> GetExistingUserEmailsAsync( + IEnumerable emails, + CancellationToken cancellationToken = default) { ArgumentNullException.ThrowIfNull(emails); - var normalizedEmails = emails.Select(EmailNormalizer.Normalize).ToList(); - var existingEmails = await _dbContext.Users - .Where(u => u.Email != null && normalizedEmails.Contains(u.Email)) + var normalizedEmails = emails.Select(EmailNormalizer.Normalize).Distinct().ToList(); + var prefix = PiiAesGcmSymmetricEncryption.EnvelopePrefix; + + var matches = new HashSet(); + + foreach (var normalized in normalizedEmails) + { + var fingerprint = _emailLookupHasher.HashNormalized(normalized); + if (fingerprint != null && + await _dbContext.Users.AnyAsync(u => u.EmailHash == fingerprint, cancellationToken)) + { + matches.Add(normalized); + } + } + + var plaintextMatches = await _dbContext.Users + .Where(u => u.EmailHash == null && u.Email != null && normalizedEmails.Contains(u.Email)) .Select(u => u.Email!) .ToListAsync(cancellationToken); - return existingEmails.ToHashSet(); + foreach (var hit in plaintextMatches) + { + matches.Add(hit); + } + + var envelopeOrphans = await _dbContext.Users + .AsNoTracking() + .Where(u => u.Email != null && u.EmailHash == null && u.Email.StartsWith(prefix)) + .ToListAsync(cancellationToken); + + foreach (var row in envelopeOrphans) + { + var normalized = TryHydrateNormalizedEmail(row.Email); + if (normalized != null && normalizedEmails.Contains(normalized)) + { + matches.Add(normalized); + } + } + + return matches; } public async Task AddUsersAsync(IEnumerable users, CancellationToken cancellationToken = default) @@ -95,7 +200,13 @@ public async Task AddUsersAsync(IEnumerable users, CancellationToken cance return; } - var entities = usersList.Select(u => MapToEntity(u)).ToList(); + var toInsert = await FilterOutUsersMatchingExistingEmailsAsync(usersList, cancellationToken); + if (toInsert.Count == 0) + { + return; + } + + var entities = toInsert.Select(MapToEntity).ToList(); _dbContext.Users.AddRange(entities); try @@ -111,26 +222,76 @@ public async Task AddUsersAsync(IEnumerable users, CancellationToken cance } } - public async Task> GetUserEmailsByDomainAsync(string emailDomain, CancellationToken cancellationToken = default) + public async Task> GetUserEmailsByDomainAsync( + string emailDomain, + CancellationToken cancellationToken = default) { ArgumentNullException.ThrowIfNull(emailDomain); - return await _dbContext.Users - .Where(u => u.Email != null && u.Email.EndsWith(emailDomain)) + var suffix = emailDomain.StartsWith("@", StringComparison.Ordinal) + ? emailDomain + : "@" + emailDomain; + + var rows = await _dbContext.Users + .Where(u => u.Email != null) .Select(u => u.Email!) .ToListAsync(cancellationToken); + + var results = new List(); + foreach (var ciphertext in rows) + { + var normalized = TryHydrateNormalizedEmail(ciphertext); + if (!string.IsNullOrEmpty(normalized) && normalized.EndsWith(suffix, StringComparison.Ordinal)) + { + results.Add(normalized); + } + } + + return results; } public async Task RemoveUsersByEmailAsync(IEnumerable emails, CancellationToken cancellationToken = default) { ArgumentNullException.ThrowIfNull(emails); - var normalizedEmails = emails.Select(EmailNormalizer.Normalize).ToList(); - var usersToRemove = await _dbContext.Users - .Where(u => u.Email != null && normalizedEmails.Contains(u.Email)) + var normalizedEmails = emails.Select(EmailNormalizer.Normalize).Distinct().ToList(); + + var usersToRemove = new List(); + + foreach (var normalized in normalizedEmails) + { + var fingerprint = _emailLookupHasher.HashNormalized(normalized); + if (fingerprint != null) + { + usersToRemove.AddRange( + await _dbContext.Users.Where(u => u.EmailHash == fingerprint).ToListAsync(cancellationToken)); + } + } + + usersToRemove.AddRange(await _dbContext.Users + .Where(u => + u.EmailHash == null && u.Email != null && normalizedEmails.Contains(u.Email)) + .ToListAsync(cancellationToken)); + + var orphans = await _dbContext.Users + .Where(u => + u.Email != null && + u.Email.StartsWith(PiiAesGcmSymmetricEncryption.EnvelopePrefix) && + u.EmailHash == null) .ToListAsync(cancellationToken); + foreach (var orphan in orphans) + { + var normalizedOrphanEmail = TryHydrateNormalizedEmail(orphan.Email); + if (normalizedOrphanEmail != null && normalizedEmails.Contains(normalizedOrphanEmail)) + { + usersToRemove.Add(orphan); + } + } - _dbContext.Users.RemoveRange(usersToRemove); + foreach (var user in usersToRemove.DistinctBy(u => u.Id)) + { + _dbContext.Users.Remove(user); + } } public async Task RemoveUserOptInsByEmailAsync(IEnumerable emails, CancellationToken cancellationToken = default) @@ -154,12 +315,43 @@ public HashSet GetExistingUserEmails(IEnumerable emails) { ArgumentNullException.ThrowIfNull(emails); - var normalizedEmails = emails.Select(EmailNormalizer.Normalize).ToList(); - var existingEmails = _dbContext.Users - .Where(u => u.Email != null && normalizedEmails.Contains(u.Email)) - .Select(u => u.Email!) - .ToList(); - return existingEmails.ToHashSet(); + var normalizedEmails = emails.Select(EmailNormalizer.Normalize).Distinct().ToList(); + var prefix = PiiAesGcmSymmetricEncryption.EnvelopePrefix; + + var matches = new HashSet(); + + foreach (var normalized in normalizedEmails) + { + var fingerprint = _emailLookupHasher.HashNormalized(normalized); + if (fingerprint != null && _dbContext.Users.Any(u => u.EmailHash == fingerprint)) + { + matches.Add(normalized); + } + } + + foreach (var row in _dbContext.Users.Where(u => + u.EmailHash == null && + u.Email != null && + normalizedEmails.Contains(u.Email)) + .Select(u => u.Email!)) + { + matches.Add(row); + } + + foreach (var row in _dbContext.Users.Where(u => + u.Email != null && + u.EmailHash == null && + u.Email.StartsWith(prefix)) + .Select(u => u.Email!)) + { + var normalized = TryHydrateNormalizedEmail(row); + if (normalized != null && normalizedEmails.Contains(normalized)) + { + matches.Add(normalized); + } + } + + return matches; } public bool AnyUsersExist() @@ -177,7 +369,13 @@ public void AddUsers(IEnumerable users) return; } - var entities = usersList.Select(u => MapToEntity(u)).ToList(); + var toInsert = FilterOutUsersMatchingExistingEmails(usersList); + if (toInsert.Count == 0) + { + return; + } + + var entities = toInsert.Select(MapToEntity).ToList(); _dbContext.Users.AddRange(entities); try diff --git a/src/SEBT.Portal.Infrastructure/Services/EmailLookupHasher.cs b/src/SEBT.Portal.Infrastructure/Services/EmailLookupHasher.cs new file mode 100644 index 000000000..4b302494d --- /dev/null +++ b/src/SEBT.Portal.Infrastructure/Services/EmailLookupHasher.cs @@ -0,0 +1,52 @@ +using System.Security.Cryptography; +using System.Text; +using Microsoft.Extensions.Options; +using SEBT.Portal.Core.AppSettings; +using SEBT.Portal.Core.Services; +using SEBT.Portal.Core.Utilities; + +namespace SEBT.Portal.Infrastructure.Services; + +/// HMAC-SHA256 over normalized-email bytes for deterministic SQL equality lookups. +public sealed class EmailLookupHasher : IEmailLookupHasher +{ + private const string DomainSeparatorUtf8 = "sep|portal|email:v1|"; + + private readonly byte[] _keyBytes; + + public EmailLookupHasher(IOptions options) + { + var secretKey = options?.Value?.SecretKey + ?? throw new InvalidOperationException("IdentifierHasher:SecretKey must be configured."); + _keyBytes = Encoding.UTF8.GetBytes(secretKey); + if (_keyBytes.Length < 32) + { + throw new InvalidOperationException("IdentifierHasher:SecretKey must be at least 32 characters."); + } + } + + /// + public string? NormalizeAndHash(string? email) + { + var normalized = EmailNormalizer.NormalizeOrNull(email); + return HashNormalized(normalized); + } + + /// + public string? HashNormalized(string? normalizedLowercaseTrimmedEmail) + { + if (string.IsNullOrWhiteSpace(normalizedLowercaseTrimmedEmail)) + { + return null; + } + + var prefixBytes = Encoding.UTF8.GetBytes(DomainSeparatorUtf8); + var normalizedBytes = Encoding.UTF8.GetBytes(normalizedLowercaseTrimmedEmail); + var combined = new byte[prefixBytes.Length + normalizedBytes.Length]; + Buffer.BlockCopy(prefixBytes, 0, combined, 0, prefixBytes.Length); + Buffer.BlockCopy(normalizedBytes, 0, combined, prefixBytes.Length, normalizedBytes.Length); + var mac = HMACSHA256.HashData(_keyBytes, combined); + + return Convert.ToHexString(mac); + } +} diff --git a/src/SEBT.Portal.Infrastructure/Services/PiiAesGcmSymmetricEncryption.cs b/src/SEBT.Portal.Infrastructure/Services/PiiAesGcmSymmetricEncryption.cs new file mode 100644 index 000000000..c6d0a5fd1 --- /dev/null +++ b/src/SEBT.Portal.Infrastructure/Services/PiiAesGcmSymmetricEncryption.cs @@ -0,0 +1,262 @@ +using System.Security.Cryptography; +using System.Text; +using Microsoft.Extensions.Options; +using SEBT.Portal.Core.AppSettings; +using SEBT.Portal.Core.Exceptions; +using SEBT.Portal.Core.Services; + +namespace SEBT.Portal.Infrastructure.Services; + +/// +/// AEAD envelopes for reversible PII: AES-GCM / 96-bit nonce / 128-bit tag. Format is stable + versioned (). +/// +public sealed class PiiAesGcmSymmetricEncryption : IPiiSymmetricEncryption +{ + public const byte EnvelopeVersion = 1; + public const string EnvelopePrefix = "sep-pii:v1:"; + + private const int TagSizeBytes = 16; + private const int NonceSizeBytes = 12; + + private readonly IReadOnlyDictionary _keys; + private readonly string _encryptKeyId; + private readonly byte[] _encryptKeyRaw; + + public PiiAesGcmSymmetricEncryption(IOptions options) + { + ArgumentNullException.ThrowIfNull(options); + var snapshot = options.Value ?? throw new InvalidOperationException("PiiEncryption settings are missing."); + + var ring = snapshot.ResolveKeyRing(); + foreach (var k in ring.Keys.Where(k => + string.Equals(k, snapshot.ActiveKeyId.Trim(), StringComparison.OrdinalIgnoreCase))) + { + _encryptKeyId = k; + _encryptKeyRaw = ring[k]; + _keys = ring; + return; + } + + throw new InvalidOperationException( + $"PiiEncryption ActiveKeyId '{snapshot.ActiveKeyId}' is not present after validation."); + } + + public bool IsEnvelope(string? storedValue) => + !string.IsNullOrEmpty(storedValue) + && storedValue.StartsWith(EnvelopePrefix, StringComparison.Ordinal); + + public string? Encrypt(string? plaintext) + { + if (string.IsNullOrEmpty(plaintext)) + { + return null; + } + + ReadOnlySpan trimmed = plaintext.AsSpan().Trim(); + if (trimmed.IsEmpty) + { + return null; + } + + var plainBytesCount = Encoding.UTF8.GetByteCount(trimmed); + Span plainBytes = plainBytesCount <= 4096 + ? stackalloc byte[plainBytesCount] + : new byte[plainBytesCount]; + + Encoding.UTF8.GetBytes(trimmed, plainBytes); + + Span nonce = stackalloc byte[NonceSizeBytes]; + RandomNumberGenerator.Fill(nonce); + + Span ciphertext = plainBytes.Length <= 8192 ? stackalloc byte[plainBytes.Length] : new byte[plainBytes.Length]; + Span tag = stackalloc byte[TagSizeBytes]; + + using (var aes = new AesGcm(_encryptKeyRaw, TagSizeBytes)) + { + aes.Encrypt(nonce, plainBytes, ciphertext, tag); + } + + var keyIdUtf8 = Encoding.UTF8.GetBytes(_encryptKeyId); + if (keyIdUtf8.Length > byte.MaxValue) + { + throw new InvalidOperationException("PII encryption key id is too long (max 255 UTF-8 bytes)."); + } + + var envelopeLength = + sizeof(byte) + + sizeof(byte) + + keyIdUtf8.Length + + nonce.Length + + ciphertext.Length + + tag.Length; + + Span envelope = envelopeLength <= 16384 + ? stackalloc byte[envelopeLength] + : new byte[envelopeLength]; + + var offset = 0; + envelope[offset++] = EnvelopeVersion; + envelope[offset++] = (byte)keyIdUtf8.Length; + keyIdUtf8.AsSpan().CopyTo(envelope[offset..]); + offset += keyIdUtf8.Length; + nonce.CopyTo(envelope[offset..]); + offset += nonce.Length; + ciphertext.CopyTo(envelope[offset..]); + offset += ciphertext.Length; + tag.CopyTo(envelope[offset..]); + + return EnvelopePrefix + Convert.ToBase64String(envelope); + } + + /// + public string Decrypt(string storedValue) + { + ArgumentException.ThrowIfNullOrWhiteSpace(storedValue, nameof(storedValue)); + + try + { + return DecodeAndDecrypt(storedValue); + } + catch (CryptographicException ex) + { + throw WrapDecrypt(ex); + } + catch (FormatException ex) + { + throw WrapDecrypt(ex); + } + catch (ArgumentException ex) + { + throw WrapDecrypt(ex); + } + catch (PiiDecryptException) + { + throw; + } + } + + public string? DecryptOrPassThroughLegacy(string? storedValue) + { + if (string.IsNullOrEmpty(storedValue)) + { + return storedValue; + } + + return IsEnvelope(storedValue) + ? Decrypt(storedValue) + : storedValue.Trim(); + } + + public string ReSealWithActiveEncryptor(string envelopeCiphertext) + { + ArgumentException.ThrowIfNullOrWhiteSpace(envelopeCiphertext); + + try + { + var plaintext = DecodeAndDecrypt(envelopeCiphertext); + return EncryptNonEmptyPlaintext(plaintext); + } + catch (CryptographicException ex) + { + throw WrapDecrypt(ex); + } + catch (FormatException ex) + { + throw WrapDecrypt(ex); + } + catch (PiiDecryptException) + { + throw; + } + } + + private string EncryptNonEmptyPlaintext(string plaintext) + { + var sealedValue = Encrypt(plaintext); + if (sealedValue == null) + { + throw new PiiDecryptException( + "PII envelope round-trip decryption produced whitespace-only payloads (unexpected)."); + } + + return sealedValue; + } + + private static PiiDecryptException WrapDecrypt(Exception inner) => + new( + "PII ciphertext decryption failed — data may have been corrupted or altered while at rest.", inner); + + private string DecodeAndDecrypt(string storedValue) + { + if (!IsEnvelope(storedValue)) + { + throw new PiiDecryptException( + $"PII decrypt expected envelope prefix '{EnvelopePrefix}'. Decrypt(...) does not decode legacy plaintext."); + } + + var base64Chars = storedValue.AsSpan()[EnvelopePrefix.Length..]; + if (base64Chars.IsEmpty) + { + throw new PiiDecryptException("PII ciphertext is empty after prefix."); + } + + byte[] envelopeBytes; + try + { + envelopeBytes = Convert.FromBase64String(base64Chars.ToString()); + } + catch (FormatException ex) + { + throw new PiiDecryptException("PII ciphertext base64 decoding failed.", ex); + } + + if (envelopeBytes.Length < 2 + 1 + NonceSizeBytes + TagSizeBytes) + { + throw new PiiDecryptException("PII ciphertext truncated."); + } + + var offset = 0; + if (envelopeBytes[offset++] != EnvelopeVersion) + { + throw new PiiDecryptException("Unsupported or unknown PII encryption envelope version."); + } + + var keyIdLength = envelopeBytes[offset++]; + if (keyIdLength == 0 + || offset + keyIdLength + NonceSizeBytes + TagSizeBytes > envelopeBytes.Length) + { + throw new PiiDecryptException("PII ciphertext malformed (key identifier length invalid)."); + } + + var keyId = Encoding.UTF8.GetString(envelopeBytes, offset, keyIdLength); + offset += keyIdLength; + + var nonceSlice = envelopeBytes.AsSpan(offset, NonceSizeBytes); + offset += NonceSizeBytes; + + var authTagCipherSpan = envelopeBytes.AsSpan(offset); + if (authTagCipherSpan.Length <= TagSizeBytes) + { + throw new PiiDecryptException("PII ciphertext malformed (truncated ciphertext)."); + } + + var cipherSpan = authTagCipherSpan[..^TagSizeBytes]; + var tagSpan = authTagCipherSpan[^TagSizeBytes..]; + + if (!_keys.TryGetValue(keyId, out var keyRaw)) + { + throw new PiiDecryptException($"No configured PII key material matches stored key id '{keyId}'."); + } + + Span plaintext = cipherSpan.Length <= 8192 + ? stackalloc byte[cipherSpan.Length] + : new byte[cipherSpan.Length]; + + using (var aes = new AesGcm(keyRaw, TagSizeBytes)) + { + aes.Decrypt(nonceSlice, cipherSpan, tagSpan, plaintext); + } + + return Encoding.UTF8.GetString(plaintext); + } +} diff --git a/src/SEBT.Portal.Infrastructure/Services/PiiPlaintextEncryptionBackfill.cs b/src/SEBT.Portal.Infrastructure/Services/PiiPlaintextEncryptionBackfill.cs new file mode 100644 index 000000000..a84712d65 --- /dev/null +++ b/src/SEBT.Portal.Infrastructure/Services/PiiPlaintextEncryptionBackfill.cs @@ -0,0 +1,179 @@ +using Microsoft.EntityFrameworkCore; +using Microsoft.Extensions.Logging; +using SEBT.Portal.Core.Services; +using SEBT.Portal.Core.Utilities; +using SEBT.Portal.Infrastructure.Data; +using SEBT.Portal.Infrastructure.Data.Entities; + +namespace SEBT.Portal.Infrastructure.Services; + +/// +/// Migrates plaintext-at-rest (or ciphertext lacking the email MAC column) into authenticated AES-GCM envelopes. +/// Executes after EF migrations — idempotent skips rows that already have coherent ciphertext. +/// +public sealed class PiiPlaintextEncryptionBackfill +{ + private readonly PortalDbContext _dbContext; + private readonly IPiiSymmetricEncryption _crypto; + private readonly IEmailLookupHasher _emailLookupHasher; + private readonly ILogger _logger; + + public PiiPlaintextEncryptionBackfill( + PortalDbContext dbContext, + IPiiSymmetricEncryption crypto, + IEmailLookupHasher emailLookupHasher, + ILogger logger) + { + _dbContext = dbContext; + _crypto = crypto; + _emailLookupHasher = emailLookupHasher; + _logger = logger; + } + + public async Task ApplyAsync(CancellationToken cancellationToken = default) + { + await MigrateUsersLoopAsync(cancellationToken); + await MigrateDocVerificationLoopAsync(cancellationToken); + } + + private async Task MigrateUsersLoopAsync(CancellationToken cancellationToken) + { + int migrated; + do + { + migrated = await MigrateOneUsersBatchAsync(cancellationToken); + } + while (migrated > 0); + } + + private async Task MigrateOneUsersBatchAsync(CancellationToken cancellationToken) + { + var prefix = PiiAesGcmSymmetricEncryption.EnvelopePrefix; + var batch = await _dbContext.Users + .Where(user => + user.Email != null && !(user.Email.StartsWith(prefix) && user.EmailHash != null) + || user.Phone != null && !user.Phone.StartsWith(prefix) + || user.SnapId != null && !user.SnapId.StartsWith(prefix) + || user.TanfId != null && !user.TanfId.StartsWith(prefix) + || user.DateOfBirth != null && !user.DateOfBirth.StartsWith(prefix)) + .OrderBy(u => u.UpdatedAt) + .Take(500) + .ToListAsync(cancellationToken); + + foreach (var user in batch) + { + BackfillEmail(prefix, user); + BackfillScalar(prefix, () => user.Phone, value => user.Phone = value); + BackfillScalar(prefix, () => user.SnapId, value => user.SnapId = value); + BackfillScalar(prefix, () => user.TanfId, value => user.TanfId = value); + BackfillScalar(prefix, () => user.DateOfBirth, value => user.DateOfBirth = value); + user.UpdatedAt = DateTime.UtcNow; + } + + if (batch.Count == 0) + { + return 0; + } + + await SaveChangesAndDetachBatchAsync(cancellationToken); + + _logger.LogInformation("PII ciphertext backfill: encrypted {Rows} Users row batch.", batch.Count); + return batch.Count; + } + + private void BackfillEmail(string envelopePrefix, UserEntity entity) + { + if (entity.Email == null) + { + entity.EmailHash = null; + return; + } + + if (entity.Email.StartsWith(envelopePrefix, StringComparison.Ordinal) && entity.EmailHash != null) + { + return; + } + + var plaintextCandidate = (_crypto.DecryptOrPassThroughLegacy(entity.Email) ?? string.Empty).Trim(); + var normalized = EmailNormalizer.NormalizeOrNull(plaintextCandidate); + if (normalized == null) + { + return; + } + + entity.Email = _crypto.Encrypt(normalized); + entity.EmailHash = _emailLookupHasher.HashNormalized(normalized); + } + + private void BackfillScalar(string prefix, Func read, Action write) + { + var current = read(); + if (string.IsNullOrEmpty(current)) + { + return; + } + + if (current.StartsWith(prefix, StringComparison.Ordinal)) + { + return; + } + + var plaintext = (_crypto.DecryptOrPassThroughLegacy(current) ?? string.Empty).Trim(); + write(string.IsNullOrEmpty(plaintext) ? null : _crypto.Encrypt(plaintext)); + } + + /// + /// Persists then clears the change tracker. Not redundant with — + /// saved entities stay tracked as ; without , + /// each batch would retain prior rows until backfill completes (unbounded tracker growth). + /// + private async Task SaveChangesAndDetachBatchAsync(CancellationToken cancellationToken) + { + await _dbContext.SaveChangesAsync(cancellationToken); + _dbContext.ChangeTracker.Clear(); + } + + private async Task MigrateDocVerificationLoopAsync(CancellationToken cancellationToken) + { + int migrated; + do + { + migrated = await MigrateOneDocVerificationBatchAsync(cancellationToken); + } + while (migrated > 0); + } + + private async Task MigrateOneDocVerificationBatchAsync(CancellationToken cancellationToken) + { + var prefix = PiiAesGcmSymmetricEncryption.EnvelopePrefix; + var batch = await _dbContext.DocVerificationChallenges + .Where(row => + row.ProofingDateOfBirth != null && !row.ProofingDateOfBirth.StartsWith(prefix) + || row.ProofingIdType != null && !row.ProofingIdType.StartsWith(prefix) + || row.ProofingIdValue != null && !row.ProofingIdValue.StartsWith(prefix)) + .OrderBy(row => row.UpdatedAt) + .Take(500) + .ToListAsync(cancellationToken); + + foreach (var challenge in batch) + { + BackfillScalar(prefix, () => challenge.ProofingDateOfBirth, value => challenge.ProofingDateOfBirth = value); + BackfillScalar(prefix, () => challenge.ProofingIdType, value => challenge.ProofingIdType = value); + BackfillScalar(prefix, () => challenge.ProofingIdValue, value => challenge.ProofingIdValue = value); + challenge.UpdatedAt = DateTime.UtcNow; + } + + if (batch.Count == 0) + { + return 0; + } + + await SaveChangesAndDetachBatchAsync(cancellationToken); + + _logger.LogInformation( + "PII ciphertext backfill: encrypted {Rows} DocVerificationChallenge row batch.", + batch.Count); + + return batch.Count; + } +} diff --git a/test/SEBT.Portal.Tests/Unit/Configuration/PiiEncryptionSettingsValidatorTests.cs b/test/SEBT.Portal.Tests/Unit/Configuration/PiiEncryptionSettingsValidatorTests.cs new file mode 100644 index 000000000..d3b92d4db --- /dev/null +++ b/test/SEBT.Portal.Tests/Unit/Configuration/PiiEncryptionSettingsValidatorTests.cs @@ -0,0 +1,86 @@ +using Microsoft.Extensions.Options; +using SEBT.Portal.Core.AppSettings; +using SEBT.Portal.Infrastructure.Configuration; +using Xunit; + +namespace SEBT.Portal.Tests.Unit.Configuration; + +public class PiiEncryptionSettingsValidatorTests +{ + private readonly PiiEncryptionSettingsValidator _validator = new(); + + [Fact] + public void Validate_WhenCoherentRing_ReturnsSuccess() + { + var options = new PiiEncryptionSettings + { + ActiveKeyId = "primary", + Keys = + [ + new PiiEncryptionKeySetting + { + KeyId = "primary", + KeyMaterialBase64 = "YjJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmI=" + } + ] + }; + + var result = _validator.Validate(Options.DefaultName, options); + + Assert.True(result.Succeeded); + } + + [Fact] + public void Validate_WhenKeyMaterialNot256Bits_ReturnsFail() + { + var options = new PiiEncryptionSettings + { + ActiveKeyId = "short-key", + Keys = + [ + new PiiEncryptionKeySetting + { + KeyId = "short-key", + // Decodes to 16 bytes — AES-128-sized; only 256-bit keys are allowed. + KeyMaterialBase64 = Convert.ToBase64String(new byte[16]) + } + ] + }; + + var result = _validator.Validate(Options.DefaultName, options); + + Assert.True(result.Failed); + Assert.Contains("256-bit", result.FailureMessage, StringComparison.Ordinal); + Assert.Contains("32", result.FailureMessage, StringComparison.Ordinal); + } + + [Fact] + public void Validate_WhenActiveKeyMissingFromRing_ReturnsFail() + { + var options = new PiiEncryptionSettings + { + ActiveKeyId = "missing", + Keys = + [ + new PiiEncryptionKeySetting + { + KeyId = "primary", + KeyMaterialBase64 = "YjJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmI=" + } + ] + }; + + var result = _validator.Validate(Options.DefaultName, options); + + Assert.True(result.Failed); + Assert.Contains("ActiveKeyId", result.FailureMessage, StringComparison.Ordinal); + } + + [Fact] + public void Validate_WhenNullOptions_ReturnsFail() + { + var result = _validator.Validate(Options.DefaultName, null!); + + Assert.True(result.Failed); + } +} diff --git a/test/SEBT.Portal.Tests/Unit/Data/PortalDbContextTests.cs b/test/SEBT.Portal.Tests/Unit/Data/PortalDbContextTests.cs index c8c4fb7b6..657e2d17c 100644 --- a/test/SEBT.Portal.Tests/Unit/Data/PortalDbContextTests.cs +++ b/test/SEBT.Portal.Tests/Unit/Data/PortalDbContextTests.cs @@ -394,10 +394,9 @@ public void Users_Email_ShouldBeNullable() } [Fact] - public void Users_Email_ShouldHaveFilteredUniqueIndex() + public void Users_EmailHash_ShouldHaveFilteredUniqueIndex() { - // The filtered index ensures uniqueness only among non-null emails, - // allowing multiple OIDC users with no email address. + // Primary equality lookup uses EmailHash; ciphertext rows omit IX_Users_Email_LegacyLookup (filtered out). // Arrange var options = new DbContextOptionsBuilder() .UseInMemoryDatabase(databaseName: Guid.NewGuid().ToString()) @@ -407,14 +406,36 @@ public void Users_Email_ShouldHaveFilteredUniqueIndex() // Act var entityType = context.Model.FindEntityType(typeof(UserEntity)); - var emailIndex = entityType!.GetIndexes() + var emailHashIndex = entityType!.GetIndexes() + .FirstOrDefault(i => i.Properties.Count == 1 && i.Properties[0].Name == "EmailHash"); + + // Assert + Assert.NotNull(emailHashIndex); + Assert.True(emailHashIndex!.IsUnique); + Assert.Equal("IX_Users_EmailHash", emailHashIndex.GetDatabaseName()); + Assert.Equal("[EmailHash] IS NOT NULL", emailHashIndex.GetFilter()); + } + + [Fact] + public void Users_Email_ShouldHaveFilteredNonUniqueIndex_ForLegacyPlaintextLookup() + { + // Arrange + var options = new DbContextOptionsBuilder() + .UseInMemoryDatabase(databaseName: Guid.NewGuid().ToString()) + .Options; + + using var context = new PortalDbContext(options); + + // Act + var entityType = context.Model.FindEntityType(typeof(UserEntity)); + var legacyEmailIndex = entityType!.GetIndexes() .FirstOrDefault(i => i.Properties.Count == 1 && i.Properties[0].Name == "Email"); // Assert - Assert.NotNull(emailIndex); - Assert.True(emailIndex!.IsUnique); - Assert.Equal("IX_Users_Email", emailIndex.GetDatabaseName()); - Assert.Equal("[Email] IS NOT NULL", emailIndex.GetFilter()); + Assert.NotNull(legacyEmailIndex); + Assert.False(legacyEmailIndex!.IsUnique); + Assert.Equal("IX_Users_Email_LegacyLookup", legacyEmailIndex.GetDatabaseName()); + Assert.Equal("[EmailHash] IS NULL AND [Email] IS NOT NULL", legacyEmailIndex.GetFilter()); } [Fact] @@ -462,8 +483,9 @@ public void Users_ExternalProviderId_ShouldHaveFilteredUniqueIndex() } [Fact] - public void Users_Email_ShouldHaveMaxLength255() + public void Users_Email_ShouldHaveMaxLength512() { + // Email column holds AES-GCM envelopes (much wider than plaintext addresses). // Arrange var options = new DbContextOptionsBuilder() .UseInMemoryDatabase(databaseName: Guid.NewGuid().ToString()) @@ -477,7 +499,7 @@ public void Users_Email_ShouldHaveMaxLength255() // Assert Assert.NotNull(emailProperty); - Assert.Equal(255, emailProperty!.GetMaxLength()); + Assert.Equal(512, emailProperty!.GetMaxLength()); } [Fact] diff --git a/test/SEBT.Portal.Tests/Unit/Infrastructure/Services/PiiAesGcmSymmetricEncryptionTests.cs b/test/SEBT.Portal.Tests/Unit/Infrastructure/Services/PiiAesGcmSymmetricEncryptionTests.cs new file mode 100644 index 000000000..f1b70cab6 --- /dev/null +++ b/test/SEBT.Portal.Tests/Unit/Infrastructure/Services/PiiAesGcmSymmetricEncryptionTests.cs @@ -0,0 +1,66 @@ +using Microsoft.Extensions.Options; +using SEBT.Portal.Core.AppSettings; +using SEBT.Portal.Core.Exceptions; +using SEBT.Portal.Infrastructure.Services; +using Xunit; + +namespace SEBT.Portal.Tests.Unit.Infrastructure.Services; + +public class PiiAesGcmSymmetricEncryptionTests +{ + private static PiiAesGcmSymmetricEncryption CreateEncryption( + string activeKeyId, + params (string KeyId, string Base64Material)[] keys) + { + var settings = new PiiEncryptionSettings + { + ActiveKeyId = activeKeyId, + Keys = keys.Select(k => new PiiEncryptionKeySetting + { + KeyId = k.KeyId, + KeyMaterialBase64 = k.Base64Material + }).ToList() + }; + + return new PiiAesGcmSymmetricEncryption(Options.Create(settings)); + } + + [Fact] + public void EncryptThenDecrypt_roundTrips_utf8_plaintext() + { + var enc = CreateEncryption("a", ("a", "YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWE=")); + var cipher = enc.Encrypt(" parent+child@example.com"); + Assert.True(enc.IsEnvelope(cipher)); + Assert.Equal("parent+child@example.com", enc.DecryptOrPassThroughLegacy(cipher)); + } + + [Fact] + public void ReSealWithActiveEncryptor_decryptsUsingEmbeddedKeyId_thenRewrapsWithActiveKey() + { + var keyA = "YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWE="; // 32 x 'a' + var keyB = "YmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmI="; // 32 x 'b' + + var writerA = CreateEncryption("key-a", ("key-a", keyA), ("key-b", keyB)); + var cipher = writerA.Encrypt("rotate-me@example.com")!; + + var writerB = CreateEncryption("key-b", ("key-a", keyA), ("key-b", keyB)); + var resealed = writerB.ReSealWithActiveEncryptor(cipher); + + Assert.NotEqual(cipher, resealed); + Assert.Equal("rotate-me@example.com", writerB.DecryptOrPassThroughLegacy(resealed)); + } + + [Fact] + public void Decrypt_whenTagIsTampered_throws_PiiDecryptException() + { + var enc = CreateEncryption("k", ("k", "YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWE=")); + var cipher = enc.Encrypt("victim@example.com")!; + + // Flip a stable position inside the Base64 payload (not the ASCII prefix). + var i = cipher.Length - 4; + var tampered = cipher[..i] + (cipher[i] == 'A' ? 'B' : 'A') + cipher[(i + 1)..]; + + var ex = Assert.Throws(() => enc.Decrypt(tampered)); + Assert.NotNull(ex.InnerException); + } +} diff --git a/test/SEBT.Portal.Tests/Unit/Repositories/DatabaseDocVerificationChallengeRepositoryTests.cs b/test/SEBT.Portal.Tests/Unit/Repositories/DatabaseDocVerificationChallengeRepositoryTests.cs index e3482383d..a9c3d960c 100644 --- a/test/SEBT.Portal.Tests/Unit/Repositories/DatabaseDocVerificationChallengeRepositoryTests.cs +++ b/test/SEBT.Portal.Tests/Unit/Repositories/DatabaseDocVerificationChallengeRepositoryTests.cs @@ -5,6 +5,7 @@ using SEBT.Portal.Infrastructure.Data.Entities; using SEBT.Portal.Infrastructure.Helpers; using SEBT.Portal.Infrastructure.Repositories; +using SEBT.Portal.Tests.Unit.TestSupport; namespace SEBT.Portal.Tests.Unit.Repositories; @@ -56,7 +57,7 @@ public async Task GetActiveByUserIdAsync_ShouldExcludeExpiredPendingChallenge() status: (int)DocVerificationStatus.Pending, expiresAt: DateTime.UtcNow.AddMinutes(-5)); - var repo = new DatabaseDocVerificationChallengeRepository(context); + var repo = new DatabaseDocVerificationChallengeRepository(context, TestPortalCryptography.PiiSymmetricEncryption); var result = await repo.GetActiveByUserIdAsync(userId); Assert.Null(result); @@ -70,7 +71,7 @@ public async Task GetActiveByUserIdAsync_ShouldIncludeNonExpiredPendingChallenge status: (int)DocVerificationStatus.Pending, expiresAt: DateTime.UtcNow.AddMinutes(25)); - var repo = new DatabaseDocVerificationChallengeRepository(context); + var repo = new DatabaseDocVerificationChallengeRepository(context, TestPortalCryptography.PiiSymmetricEncryption); var result = await repo.GetActiveByUserIdAsync(userId); Assert.NotNull(result); @@ -84,7 +85,7 @@ public async Task GetActiveByUserIdAsync_ShouldIncludeCreatedChallengeWithNullEx status: (int)DocVerificationStatus.Created, expiresAt: null); - var repo = new DatabaseDocVerificationChallengeRepository(context); + var repo = new DatabaseDocVerificationChallengeRepository(context, TestPortalCryptography.PiiSymmetricEncryption); var result = await repo.GetActiveByUserIdAsync(userId); Assert.NotNull(result); @@ -98,7 +99,7 @@ public async Task GetActiveByUserIdAsync_ShouldExcludeExpiredCreatedChallenge() status: (int)DocVerificationStatus.Created, expiresAt: DateTime.UtcNow.AddMinutes(-10)); - var repo = new DatabaseDocVerificationChallengeRepository(context); + var repo = new DatabaseDocVerificationChallengeRepository(context, TestPortalCryptography.PiiSymmetricEncryption); var result = await repo.GetActiveByUserIdAsync(userId); Assert.Null(result); @@ -139,7 +140,7 @@ public async Task CreateAsync_ShouldExpireTimeElapsedRow_ThenInsertNew() status: (int)DocVerificationStatus.Created, expiresAt: DateTime.UtcNow.AddMinutes(-10)); - var repo = new DatabaseDocVerificationChallengeRepository(context); + var repo = new DatabaseDocVerificationChallengeRepository(context, TestPortalCryptography.PiiSymmetricEncryption); var challenge = new DocVerificationChallenge { UserId = userId, @@ -167,7 +168,7 @@ public async Task CreateAsync_ShouldTranslateUniqueIndexViolation_ToDuplicateRec status: (int)DocVerificationStatus.Created, expiresAt: DateTime.UtcNow.AddMinutes(30)); - var repo = new DatabaseDocVerificationChallengeRepository(context); + var repo = new DatabaseDocVerificationChallengeRepository(context, TestPortalCryptography.PiiSymmetricEncryption); var duplicate = new DocVerificationChallenge { UserId = userId, diff --git a/test/SEBT.Portal.Tests/Unit/Repositories/DatabaseUserRepositoryTests.cs b/test/SEBT.Portal.Tests/Unit/Repositories/DatabaseUserRepositoryTests.cs index 15ba368b8..f2db3b535 100644 --- a/test/SEBT.Portal.Tests/Unit/Repositories/DatabaseUserRepositoryTests.cs +++ b/test/SEBT.Portal.Tests/Unit/Repositories/DatabaseUserRepositoryTests.cs @@ -4,9 +4,11 @@ using SEBT.Portal.Core.Models.Auth; using SEBT.Portal.Core.Services; using SEBT.Portal.Infrastructure.Data; +using SEBT.Portal.Infrastructure.Data.Entities; using SEBT.Portal.Infrastructure.Helpers; using SEBT.Portal.Infrastructure.Repositories; using SEBT.Portal.Infrastructure.Services; +using SEBT.Portal.Tests.Unit.TestSupport; namespace SEBT.Portal.Tests.Unit.Repositories; @@ -29,7 +31,11 @@ private PortalDbContext CreateContext() } private static DatabaseUserRepository CreateRepository(PortalDbContext context) => - new(context, TestHasher); + new( + context, + TestHasher, + TestPortalCryptography.PiiSymmetricEncryption, + TestPortalCryptography.EmailLookupHasher); [Fact] public async Task GetUserByEmailAsync_WhenUserExists_ShouldReturnUser() @@ -142,7 +148,6 @@ public async Task CreateUserAsync_ShouldStoreUserInDatabase() IalLevel = UserIalLevel.IAL1, IdProofingSessionId = "session-123", IdProofingCompletedAt = null, - IdProofingExpiresAt = null, CreatedAt = DateTime.UtcNow, UpdatedAt = DateTime.UtcNow }; @@ -152,9 +157,9 @@ public async Task CreateUserAsync_ShouldStoreUserInDatabase() // Assert var normalizedEmail = uniqueEmail.ToLowerInvariant(); - var stored = await context.Users.FirstOrDefaultAsync(u => u.Email == normalizedEmail); + var stored = await FindStoredUserWithLegacyPlaintextFallbackAsync(context, normalizedEmail); Assert.NotNull(stored); - Assert.Equal(normalizedEmail, stored!.Email); + AssertPersistedNormalizedEmailCipher(normalizedEmail, stored!.Email); Assert.Equal((int)UserIalLevel.IAL1, stored.IalLevel); Assert.Equal("session-123", stored.IdProofingSessionId); } @@ -177,9 +182,9 @@ public async Task CreateUserAsync_ShouldNormalizeEmailToLowercase() // Assert var normalizedEmail = $"user-{uniqueId}@example.com"; - var stored = await context.Users.FirstOrDefaultAsync(u => u.Email == normalizedEmail); + var stored = await FindStoredUserWithLegacyPlaintextFallbackAsync(context, normalizedEmail); Assert.NotNull(stored); - Assert.Equal(normalizedEmail, stored!.Email); + AssertPersistedNormalizedEmailCipher(normalizedEmail, stored!.Email); } [Fact] @@ -230,7 +235,6 @@ public async Task UpdateUserAsync_ShouldUpdateExistingUser() u.IalLevel = UserIalLevel.IAL1plus; u.IdProofingSessionId = "new-session-456"; u.IdProofingCompletedAt = DateTime.UtcNow; - u.IdProofingExpiresAt = DateTime.UtcNow.AddYears(1); }); // Set init-only properties using reflection var idProperty = typeof(User).GetProperty(nameof(User.Id)); @@ -242,12 +246,11 @@ public async Task UpdateUserAsync_ShouldUpdateExistingUser() await repository.UpdateUserAsync(user, CancellationToken.None); // Assert - var updated = await context.Users.FirstOrDefaultAsync(u => u.Email == uniqueEmail); + var updated = await FindStoredUserWithLegacyPlaintextFallbackAsync(context, uniqueEmail); Assert.NotNull(updated); Assert.Equal((int)UserIalLevel.IAL1plus, updated!.IalLevel); Assert.Equal("new-session-456", updated.IdProofingSessionId); Assert.NotNull(updated.IdProofingCompletedAt); - Assert.NotNull(updated.IdProofingExpiresAt); } [Fact] @@ -285,7 +288,7 @@ public async Task UpdateUserAsync_ShouldUpdateUpdatedAtTimestamp() await repository.UpdateUserAsync(user, CancellationToken.None); // Assert - var updated = await context.Users.FirstOrDefaultAsync(u => u.Email == uniqueEmail); + var updated = await FindStoredUserWithLegacyPlaintextFallbackAsync(context, uniqueEmail); Assert.NotNull(updated); Assert.True(updated!.UpdatedAt > originalTime); } @@ -351,7 +354,7 @@ public async Task UpdateUserAsync_ShouldBeCaseInsensitive() await repository.UpdateUserAsync(user, CancellationToken.None); // Assert - var updated = await context.Users.FirstOrDefaultAsync(u => u.Email == baseEmail); + var updated = await FindStoredUserWithLegacyPlaintextFallbackAsync(context, baseEmail); Assert.NotNull(updated); Assert.Equal((int)UserIalLevel.IAL1plus, updated!.IalLevel); } @@ -389,7 +392,7 @@ public async Task UpdateUserAsync_ShouldUpdateEmail() // Assert var updated = await context.Users.FirstOrDefaultAsync(u => u.Id == entity.Id); Assert.NotNull(updated); - Assert.Equal(newEmail.ToLowerInvariant(), updated!.Email); + AssertPersistedNormalizedEmailCipher(newEmail.ToLowerInvariant(), updated!.Email); Assert.Equal((int)UserIalLevel.IAL1plus, updated.IalLevel); } @@ -461,7 +464,7 @@ public async Task UpdateUserAsync_WhenUserHasSsn_ShouldStoreSsnAsHash() await repository.UpdateUserAsync(user, CancellationToken.None); // Assert - SSN stored as HMAC-SHA256 hash, not plaintext - var updated = await context.Users.FirstOrDefaultAsync(u => u.Email == uniqueEmail); + var updated = await FindStoredUserWithLegacyPlaintextFallbackAsync(context, uniqueEmail); Assert.NotNull(updated); Assert.NotNull(updated!.Ssn); Assert.Equal(64, updated.Ssn.Length); @@ -498,7 +501,7 @@ public async Task GetOrCreateUserAsync_WhenUserExists_ShouldReturnExistingUser() Assert.Equal(entity.CreatedAt, result.CreatedAt); // Verify only one user exists with this email - var count = await context.Users.CountAsync(u => u.Email == uniqueEmail); + var count = await CountNormalizedEmailMatchesAsync(context, uniqueEmail); Assert.Equal(1, count); } @@ -523,7 +526,7 @@ public async Task GetOrCreateUserAsync_WhenUserDoesNotExist_ShouldCreateNewUser( Assert.NotEqual(default(DateTime), result.UpdatedAt); // Verify user was saved - var stored = await context.Users.FirstOrDefaultAsync(u => u.Email == uniqueEmail); + var stored = await FindStoredUserWithLegacyPlaintextFallbackAsync(context, uniqueEmail); Assert.NotNull(stored); } @@ -547,7 +550,7 @@ public async Task GetOrCreateUserAsync_ShouldNormalizeEmailToLowercase() Assert.Equal(expectedEmail, result.Email); // Verify stored with lowercase - var stored = await context.Users.FirstOrDefaultAsync(u => u.Email == expectedEmail); + var stored = await FindStoredUserWithLegacyPlaintextFallbackAsync(context, expectedEmail); Assert.NotNull(stored); } @@ -744,7 +747,9 @@ public async Task MapToDomainModel_ShouldCorrectlyMapAllProperties() Assert.Equal(UserIalLevel.IAL1plus, result.IalLevel); Assert.Equal("test-session", result.IdProofingSessionId); Assert.Equal(completedAt, result.IdProofingCompletedAt); +#pragma warning disable CS0618 // User.IdProofingExpiresAt — verifying EF ↔ domain mapping until column retired Assert.Equal(expiresAt, result.IdProofingExpiresAt); +#pragma warning restore CS0618 Assert.True(result.IsCoLoaded); Assert.Equal(coLoadedUpdated, result.CoLoadedLastUpdated); Assert.Equal(createdAt, result.CreatedAt); @@ -752,7 +757,7 @@ public async Task MapToDomainModel_ShouldCorrectlyMapAllProperties() } [Fact] - public async Task CreateUserAsync_WhenUserHasIdentifierFields_ShouldStoreSsnAsHashAndOthersAsPlaintext() + public async Task CreateUserAsync_WhenUserHasIdentifierFields_ShouldStoreSsnAsHashAndOthersEncrypted() { // Arrange using var context = CreateContext(); @@ -770,12 +775,18 @@ public async Task CreateUserAsync_WhenUserHasIdentifierFields_ShouldStoreSsnAsHa // Act await repository.CreateUserAsync(user, CancellationToken.None); - // Assert - Phone, SnapId, TanfId stored as plaintext; SSN stored as HMAC-SHA256 hash - var stored = await context.Users.FirstOrDefaultAsync(u => u.Email == uniqueEmail); + // Assert - reversible identifiers persist as ciphertext; SSN stored as HMAC-SHA256 hash + var stored = await FindStoredUserWithLegacyPlaintextFallbackAsync(context, uniqueEmail); Assert.NotNull(stored); - Assert.Equal("8185558439", stored!.Phone); - Assert.Equal("SNAP123", stored.SnapId); - Assert.Equal("TANF456", stored.TanfId); + Assert.Equal( + "8185558439", + TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(stored!.Phone)); + Assert.Equal( + "SNAP123", + TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(stored.SnapId)); + Assert.Equal( + "TANF456", + TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(stored.TanfId)); Assert.NotNull(stored.Ssn); Assert.Equal(64, stored.Ssn.Length); Assert.NotEqual("123456789", stored.Ssn); @@ -798,11 +809,13 @@ public async Task CreateUserAsync_ShouldPreserveAllUserProperties() u.IalLevel = UserIalLevel.None; u.IdProofingSessionId = "full-session"; u.IdProofingCompletedAt = completedAt; - u.IdProofingExpiresAt = expiresAt; u.IsCoLoaded = true; u.CoLoadedLastUpdated = coLoadedUpdated; u.UpdatedAt = DateTime.UtcNow.AddDays(-5); }); +#pragma warning disable CS0618 // User.IdProofingExpiresAt — persistence round-trip for legacy column + user.IdProofingExpiresAt = expiresAt; +#pragma warning restore CS0618 // Set init-only CreatedAt using reflection var createdAtProperty = typeof(User).GetProperty(nameof(User.CreatedAt)); createdAtProperty?.SetValue(user, DateTime.UtcNow.AddDays(-10)); @@ -811,7 +824,7 @@ public async Task CreateUserAsync_ShouldPreserveAllUserProperties() await repository.CreateUserAsync(user, CancellationToken.None); // Assert - var stored = await context.Users.FirstOrDefaultAsync(u => u.Email == uniqueEmail); + var stored = await FindStoredUserWithLegacyPlaintextFallbackAsync(context, uniqueEmail); Assert.NotNull(stored); Assert.Equal((int)UserIalLevel.None, stored!.IalLevel); Assert.Equal("full-session", stored.IdProofingSessionId); @@ -820,5 +833,40 @@ public async Task CreateUserAsync_ShouldPreserveAllUserProperties() Assert.True(stored.IsCoLoaded); Assert.Equal(coLoadedUpdated, stored.CoLoadedLastUpdated); } + + private static async Task FindStoredUserWithLegacyPlaintextFallbackAsync( + PortalDbContext ctx, + string normalizedEmailPlaintext) + { + var fingerprint = + TestPortalCryptography.EmailLookupHasher.HashNormalized(normalizedEmailPlaintext)!; + + return await ctx.Users.FirstOrDefaultAsync( + u => + u.EmailHash == fingerprint || + u.EmailHash == null && u.Email != null && u.Email == normalizedEmailPlaintext, + CancellationToken.None); + } + + private static async Task CountNormalizedEmailMatchesAsync(PortalDbContext ctx, string normalizedEmail) + { + var fingerprint = + TestPortalCryptography.EmailLookupHasher.HashNormalized(normalizedEmail)!; + + return await ctx.Users.CountAsync( + u => + u.EmailHash == fingerprint || + u.EmailHash == null && u.Email != null && u.Email == normalizedEmail, + CancellationToken.None); + } + + private static void AssertPersistedNormalizedEmailCipher(string expectedNormalizedPlaintext, string? ciphertextEmail) + { + Assert.False(string.IsNullOrEmpty(ciphertextEmail)); + var plain = + TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(ciphertextEmail!); + Assert.False(string.IsNullOrEmpty(plain)); + Assert.Equal(expectedNormalizedPlaintext, plain); + } } diff --git a/test/SEBT.Portal.Tests/Unit/Services/DataSeederTests.cs b/test/SEBT.Portal.Tests/Unit/Services/DataSeederTests.cs index 48b47e712..6435ff6ce 100644 --- a/test/SEBT.Portal.Tests/Unit/Services/DataSeederTests.cs +++ b/test/SEBT.Portal.Tests/Unit/Services/DataSeederTests.cs @@ -1,12 +1,14 @@ using Microsoft.EntityFrameworkCore; using NSubstitute; using SEBT.Portal.Core.Models.Auth; +using SEBT.Portal.Core.Utilities; using SEBT.Portal.Core.Services; using SEBT.Portal.Infrastructure.Data; using SEBT.Portal.Infrastructure.Data.Entities; using SEBT.Portal.Infrastructure.Services; using SEBT.Portal.TestUtilities.Helpers; using SEBT.Portal.Tests.Unit.Repositories; +using SEBT.Portal.Tests.Unit.TestSupport; using UserEntityFactory = SEBT.Portal.Infrastructure.Helpers.UserFactory; namespace SEBT.Portal.Tests.Unit.Services; @@ -28,7 +30,31 @@ private static DataSeeder CreateDataSeeder(PortalDbContext context) { var identifierHasher = Substitute.For(); identifierHasher.HashForStorage(Arg.Any()).Returns(c => c.Arg()); - return new DataSeeder(context, identifierHasher); + return new DataSeeder( + context, + identifierHasher, + TestPortalCryptography.PiiSymmetricEncryption, + TestPortalCryptography.EmailLookupHasher); + } + + private static string NormalizeEmailStrict(string plaintext) => + EmailNormalizer.Normalize(plaintext); + + private static string EmailFingerprint(string plaintextNormalizedOrAnyCase) => + TestPortalCryptography.EmailLookupHasher.HashNormalized(NormalizeEmailStrict(plaintextNormalizedOrAnyCase))!; + + private static string StoredEmailPlaintext(UserEntity row) => + TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(row.Email!)!; + + private static async Task LoadUserRawRowAsync(PortalDbContext ctx, string emailAnyCase) + { + var norm = NormalizeEmailStrict(emailAnyCase); + var fp = EmailFingerprint(emailAnyCase); + return await ctx.Users.FirstOrDefaultAsync( + u => + u.EmailHash == fp || + u.EmailHash == null && u.Email != null && u.Email == norm, + CancellationToken.None); } private async Task CleanupDatabaseAsync(PortalDbContext context) @@ -144,9 +170,9 @@ public async Task AddUsersAsync_AddsUsersToDatabase() await seeder.AddUsersAsync(new[] { user }); - var stored = await context.Users.FirstOrDefaultAsync(u => u.Email == email); + var stored = await LoadUserRawRowAsync(context, email); Assert.NotNull(stored); - Assert.Equal(email, stored!.Email); + Assert.Equal(NormalizeEmailStrict(email), StoredEmailPlaintext(stored!)); } [Fact] @@ -188,7 +214,10 @@ public async Task AddUsersAsync_WhenDuplicateKey_HandlesGracefully() await seeder.AddUsersAsync(new[] { user }); - var users = await newContext.Users.Where(u => u.Email == email).ToListAsync(); + var fingerprint = EmailFingerprint(email); + var users = await newContext.Users + .Where(u => u.EmailHash == fingerprint || u.EmailHash == null && u.Email == NormalizeEmailStrict(email)) + .ToListAsync(); Assert.Single(users); } @@ -244,9 +273,12 @@ public async Task RemoveUsersByEmailAsync_RemovesSpecifiedUsers() await seeder.RemoveUsersByEmailAsync(new[] { toRemove }); await seeder.SaveChangesAsync(); - var remaining = await context.Users.Select(u => u.Email).ToListAsync(); - Assert.DoesNotContain(toRemove, remaining); - Assert.Contains(toKeep, remaining); + var remainingEmails = await context.Users.Select(u => u.Email!).ToListAsync(); + var decrypted = remainingEmails + .Select(e => TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(e)!) + .ToList(); + Assert.DoesNotContain(NormalizeEmailStrict(toRemove), decrypted); + Assert.Contains(NormalizeEmailStrict(toKeep), decrypted); } [Fact] @@ -273,8 +305,11 @@ public async Task RemoveUsersByEmailAsync_NormalizesEmails() await seeder.RemoveUsersByEmailAsync(new[] { email.ToUpperInvariant() }); await seeder.SaveChangesAsync(); - var count = await context.Users.CountAsync(u => u.Email == email); - Assert.Equal(0, count); + var fingerprint = EmailFingerprint(email); + var rowStillThere = await context.Users.FirstOrDefaultAsync( + u => u.EmailHash == fingerprint || + u.EmailHash == null && u.Email == NormalizeEmailStrict(email)); + Assert.Null(rowStillThere); } [Fact] @@ -320,7 +355,7 @@ public async Task SaveChangesAsync_PersistsPendingChanges() await seeder.SaveChangesAsync(); - var stored = await context.Users.FirstOrDefaultAsync(u => u.Email == email); + var stored = await LoadUserRawRowAsync(context, email); Assert.NotNull(stored); } @@ -392,9 +427,11 @@ public void AddUsers_AddsUsersToDatabase() seeder.AddUsers(new[] { user }); - var stored = context.Users.FirstOrDefault(u => u.Email == email); + var stored = context.Users.SingleOrDefault( + u => u.EmailHash == EmailFingerprint(email) || + u.EmailHash == null && u.Email == NormalizeEmailStrict(email)); Assert.NotNull(stored); - Assert.Equal(email, stored!.Email); + Assert.Equal(NormalizeEmailStrict(email), StoredEmailPlaintext(stored!)); } [Fact] @@ -429,7 +466,10 @@ public void SaveChanges_PersistsPendingChanges() seeder.SaveChanges(); - var stored = context.Users.FirstOrDefault(u => u.Email == email); + var stored = context.Users.SingleOrDefault( + u => + u.EmailHash == EmailFingerprint(email) || + u.EmailHash == null && u.Email == NormalizeEmailStrict(email)); Assert.NotNull(stored); } @@ -438,6 +478,9 @@ public void Constructor_WhenDbContextNull_ThrowsArgumentNullException() { var identifierHasher = Substitute.For(); Assert.Throws(() => - new DataSeeder(null!, identifierHasher)); + new DataSeeder( + null!, identifierHasher, + TestPortalCryptography.PiiSymmetricEncryption, + TestPortalCryptography.EmailLookupHasher)); } } diff --git a/test/SEBT.Portal.Tests/Unit/Services/DatabaseSeederTests.cs b/test/SEBT.Portal.Tests/Unit/Services/DatabaseSeederTests.cs index 8d008c8b5..89858ec39 100644 --- a/test/SEBT.Portal.Tests/Unit/Services/DatabaseSeederTests.cs +++ b/test/SEBT.Portal.Tests/Unit/Services/DatabaseSeederTests.cs @@ -10,6 +10,7 @@ using SEBT.Portal.Infrastructure.Seeding.Services; using SEBT.Portal.Infrastructure.Services; using SEBT.Portal.Tests.Unit.Repositories; +using SEBT.Portal.Tests.Unit.TestSupport; using UserEntityFactory = SEBT.Portal.Infrastructure.Helpers.UserFactory; namespace SEBT.Portal.Tests.Unit.Services; @@ -37,7 +38,11 @@ private PortalDbContext CreateContext() private DatabaseSeeder CreateSeeder(PortalDbContext context, SeedingSettings? settings = null) { - var dataSeeder = new DataSeeder(context, TestHasher); + var dataSeeder = new DataSeeder( + context, + TestHasher, + TestPortalCryptography.PiiSymmetricEncryption, + TestPortalCryptography.EmailLookupHasher); var timeProvider = new FakeTimeProvider(FixedSeedTime); return new DatabaseSeeder(dataSeeder, settings, timeProvider: timeProvider); } @@ -138,7 +143,8 @@ public async Task SeedUsersAsync_ShouldNormalizeEmails() Assert.All(users, user => { Assert.NotNull(user.Email); - Assert.Equal(user.Email, user.Email!.ToLowerInvariant()); + var plain = TestPortalCryptography.StoredEmailPlaintext(user.Email!); + Assert.Equal(plain, plain.ToLowerInvariant()); }); } @@ -190,7 +196,7 @@ public async Task SeedTestUsersAsync_WhenDatabaseIsEmpty_ShouldCreateAllTestUser var users = await context.Users.ToListAsync(); Assert.Equal(3, users.Count); - var emails = users.Select(u => u.Email).ToHashSet(); + var emails = users.Select(u => TestPortalCryptography.StoredEmailPlaintext(u.Email!)).ToHashSet(); Assert.Contains("co-loaded@example.com", emails); Assert.Contains("non-co-loaded@example.com", emails); Assert.Contains("not-started@example.com", emails); @@ -209,7 +215,7 @@ public async Task SeedTestUsersAsync_ShouldCreateUsersWithCorrectProperties() // Assert - Check co-loaded user var coLoadedUser = await context.Users - .FirstOrDefaultAsync(u => u.Email == "co-loaded@example.com"); + .FirstOrDefaultAsync(u => u.EmailHash == TestPortalCryptography.FingerprintEmail("co-loaded@example.com") || u.EmailHash == null && u.Email == TestPortalCryptography.NormalizeEmailStrict("co-loaded@example.com")); Assert.NotNull(coLoadedUser); Assert.True(coLoadedUser!.IsCoLoaded); Assert.Equal((int)IdProofingStatus.Completed, coLoadedUser.IdProofingStatus); @@ -219,7 +225,7 @@ public async Task SeedTestUsersAsync_ShouldCreateUsersWithCorrectProperties() // Check non-co-loaded user var nonCoLoadedUser = await context.Users - .FirstOrDefaultAsync(u => u.Email == "non-co-loaded@example.com"); + .FirstOrDefaultAsync(u => u.EmailHash == TestPortalCryptography.FingerprintEmail("non-co-loaded@example.com") || u.EmailHash == null && u.Email == TestPortalCryptography.NormalizeEmailStrict("non-co-loaded@example.com")); Assert.NotNull(nonCoLoadedUser); Assert.False(nonCoLoadedUser!.IsCoLoaded); Assert.Equal((int)IdProofingStatus.InProgress, nonCoLoadedUser.IdProofingStatus); @@ -227,7 +233,7 @@ public async Task SeedTestUsersAsync_ShouldCreateUsersWithCorrectProperties() // Check not-started user var notStartedUser = await context.Users - .FirstOrDefaultAsync(u => u.Email == "not-started@example.com"); + .FirstOrDefaultAsync(u => u.EmailHash == TestPortalCryptography.FingerprintEmail("not-started@example.com") || u.EmailHash == null && u.Email == TestPortalCryptography.NormalizeEmailStrict("not-started@example.com")); Assert.NotNull(notStartedUser); Assert.False(notStartedUser!.IsCoLoaded); Assert.Equal((int)IdProofingStatus.NotStarted, notStartedUser.IdProofingStatus); @@ -261,7 +267,7 @@ public async Task SeedTestUsersAsync_WhenUsersAlreadyExist_ShouldSkipExistingUse // Verify the existing user wasn't modified var coLoadedUser = await context.Users - .FirstOrDefaultAsync(u => u.Email == "co-loaded@example.com"); + .FirstOrDefaultAsync(u => u.EmailHash == TestPortalCryptography.FingerprintEmail("co-loaded@example.com") || u.EmailHash == null && u.Email == TestPortalCryptography.NormalizeEmailStrict("co-loaded@example.com")); Assert.NotNull(coLoadedUser); Assert.False(coLoadedUser!.IsCoLoaded); // Should remain as originally set } @@ -333,23 +339,23 @@ public async Task SeedTestUsers_WhenDatabaseIsEmpty_ShouldCreateAllTestUsers() var users = await context.Users.ToListAsync(); Assert.Equal(3, users.Count); - var emails = users.Select(u => u.Email).ToHashSet(); + var emails = users.Select(u => TestPortalCryptography.StoredEmailPlaintext(u.Email!)).ToHashSet(); Assert.Contains("co-loaded@example.com", emails); Assert.Contains("non-co-loaded@example.com", emails); Assert.Contains("not-started@example.com", emails); // Verify Phone/SnapId/TanfId stored as plaintext; SSN stored as hash - var coLoaded = users.First(u => u.Email == "co-loaded@example.com"); - Assert.Equal("8185558439", coLoaded.Phone); - Assert.Equal("SNAP-CO-001", coLoaded.SnapId); - Assert.Equal("TANF-CO-001", coLoaded.TanfId); + var coLoaded = users.First(u => u.EmailHash == TestPortalCryptography.FingerprintEmail("co-loaded@example.com") || u.EmailHash == null && u.Email == TestPortalCryptography.NormalizeEmailStrict("co-loaded@example.com")); + Assert.Equal("8185558439", TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(coLoaded.Phone!)); + Assert.Equal("SNAP-CO-001", TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(coLoaded.SnapId!)); + Assert.Equal("TANF-CO-001", TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(coLoaded.TanfId!)); Assert.NotNull(coLoaded.Ssn); Assert.Equal(64, coLoaded.Ssn!.Length); Assert.NotEqual("123456789", coLoaded.Ssn); - var nonCoLoaded = users.First(u => u.Email == "non-co-loaded@example.com"); - Assert.Equal("5555551234", nonCoLoaded.Phone); - Assert.Equal("SNAP-NCO-001", nonCoLoaded.SnapId); + var nonCoLoaded = users.First(u => u.EmailHash == TestPortalCryptography.FingerprintEmail("non-co-loaded@example.com") || u.EmailHash == null && u.Email == TestPortalCryptography.NormalizeEmailStrict("non-co-loaded@example.com")); + Assert.Equal("5555551234", TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(nonCoLoaded.Phone!)); + Assert.Equal("SNAP-NCO-001", TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(nonCoLoaded.SnapId!)); } [Fact] @@ -560,7 +566,7 @@ public async Task ClearSeededDataAsync_ShouldOnlyDeleteKnownScenarioEmails() // Assert - Only non-scenario users should remain var users = await context.Users.ToListAsync(); Assert.Equal(2, users.Count); - var emails = users.Select(u => u.Email).ToHashSet(); + var emails = users.Select(u => TestPortalCryptography.StoredEmailPlaintext(u.Email!)).ToHashSet(); Assert.Contains("user1@production.com", emails); Assert.Contains("random@example.com", emails); Assert.DoesNotContain("co-loaded@example.com", emails); @@ -586,7 +592,8 @@ public async Task SeedUsersAsync_ShouldCreateUsersWithValidData() { Assert.NotNull(user.Email); Assert.NotEmpty(user.Email!); - Assert.Contains("@", user.Email!); + var plainEmail = TestPortalCryptography.StoredEmailPlaintext(user.Email!); + Assert.Contains("@", plainEmail); Assert.InRange(user.IalLevel, 0, 3); // Valid UserIalLevel range Assert.NotEqual(default(DateTime), user.CreatedAt); Assert.NotEqual(default(DateTime), user.UpdatedAt); @@ -609,7 +616,8 @@ public async Task SeedTestUsersAsync_ShouldNormalizeEmailsToLowercase() Assert.All(users, user => { Assert.NotNull(user.Email); - Assert.Equal(user.Email, user.Email!.ToLowerInvariant()); + var plain = TestPortalCryptography.StoredEmailPlaintext(user.Email!); + Assert.Equal(plain, plain.ToLowerInvariant()); }); } @@ -629,7 +637,8 @@ public async Task SeedTestUsers_ShouldNormalizeEmailsToLowercase() Assert.All(users, user => { Assert.NotNull(user.Email); - Assert.Equal(user.Email, user.Email!.ToLowerInvariant()); + var plain = TestPortalCryptography.StoredEmailPlaintext(user.Email!); + Assert.Equal(plain, plain.ToLowerInvariant()); }); } @@ -649,14 +658,14 @@ public async Task SeedTestUsersAsync_WithCustomEmailPattern_ShouldCreateUsersWit var users = await context.Users.ToListAsync(); Assert.Equal(3, users.Count); - var emails = users.Select(u => u.Email).ToHashSet(); + var emails = users.Select(u => TestPortalCryptography.StoredEmailPlaintext(u.Email!)).ToHashSet(); Assert.Contains("sebt.dc+co-loaded@codeforamerica.org", emails); Assert.Contains("sebt.dc+non-co-loaded@codeforamerica.org", emails); Assert.Contains("sebt.dc+not-started@codeforamerica.org", emails); // Verify co-loaded user still has correct properties var coLoadedUser = await context.Users - .FirstOrDefaultAsync(u => u.Email == "sebt.dc+co-loaded@codeforamerica.org"); + .FirstOrDefaultAsync(u => u.EmailHash == TestPortalCryptography.FingerprintEmail("sebt.dc+co-loaded@codeforamerica.org") || u.EmailHash == null && u.Email == TestPortalCryptography.NormalizeEmailStrict("sebt.dc+co-loaded@codeforamerica.org")); Assert.NotNull(coLoadedUser); Assert.True(coLoadedUser!.IsCoLoaded); Assert.Equal((int)IdProofingStatus.Completed, coLoadedUser.IdProofingStatus); @@ -679,7 +688,7 @@ public async Task SeedTestUsersAsync_WithCustomEmailPattern_MockHouseholdData_Sh var users = await context.Users.ToListAsync(); Assert.Equal(19, users.Count); - var emails = users.Select(u => u.Email).ToHashSet(); + var emails = users.Select(u => TestPortalCryptography.StoredEmailPlaintext(u.Email!)).ToHashSet(); Assert.Contains("sebt.co+co-loaded@codeforamerica.org", emails); Assert.Contains("sebt.co+verified@codeforamerica.org", emails); Assert.Contains("sebt.co+singlechild@codeforamerica.org", emails); @@ -702,7 +711,7 @@ public async Task SeedTestUsersAsync_WithMockHouseholdData_IdProofInProgressUser await seeder.SeedTestUsersAsync(useMockHouseholdData: true); var user = await context.Users - .SingleOrDefaultAsync(u => u.Email == "id-proof-in-progress@example.com"); + .SingleOrDefaultAsync(u => u.EmailHash == TestPortalCryptography.FingerprintEmail("id-proof-in-progress@example.com") || u.EmailHash == null && u.Email == TestPortalCryptography.NormalizeEmailStrict("id-proof-in-progress@example.com")); Assert.NotNull(user); Assert.False(user!.IsCoLoaded); Assert.Equal((int)IdProofingStatus.InProgress, user.IdProofingStatus); @@ -724,16 +733,16 @@ public async Task SeedTestUsersAsync_WithMockHouseholdData_AndStateDc_ShouldSeed var users = await context.Users.ToListAsync(); Assert.Equal(SeedScenarios.UserScenarios.Count, users.Count); var pending = await context.Users - .SingleOrDefaultAsync(u => u.Email == "co-loaded-pending-id-proofing@example.com"); + .SingleOrDefaultAsync(u => u.EmailHash == TestPortalCryptography.FingerprintEmail("co-loaded-pending-id-proofing@example.com") || u.EmailHash == null && u.Email == TestPortalCryptography.NormalizeEmailStrict("co-loaded-pending-id-proofing@example.com")); Assert.NotNull(pending); Assert.True(pending!.IsCoLoaded); Assert.Equal((int)IdProofingStatus.NotStarted, pending.IdProofingStatus); Assert.Equal((int)UserIalLevel.None, pending.IalLevel); Assert.Null(pending.IdProofingCompletedAt); Assert.Null(pending.IdProofingExpiresAt); - Assert.Equal("8185558438", pending.Phone); - Assert.Equal("SNAP-CO-001", pending.SnapId); - Assert.Equal("TANF-CO-001", pending.TanfId); + Assert.Equal("8185558438", TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(pending.Phone!)); + Assert.Equal("SNAP-CO-001", TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(pending.SnapId!)); + Assert.Equal("TANF-CO-001", TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(pending.TanfId!)); } [Fact] @@ -751,7 +760,7 @@ public async Task SeedTestUsersAsync_WithMockHouseholdData_AndStateDc_NoIal0Or1U .Where(u => (u.IalLevel == (int)UserIalLevel.None || u.IalLevel == (int)UserIalLevel.IAL1) && u.IdProofingCompletedAt != null) - .Select(u => u.Email) + .Select(u => TestPortalCryptography.StoredEmailPlaintext(u.Email!)) .ToList(); Assert.Empty(invalid); } diff --git a/test/SEBT.Portal.Tests/Unit/Services/PiiPlaintextEncryptionBackfillTests.cs b/test/SEBT.Portal.Tests/Unit/Services/PiiPlaintextEncryptionBackfillTests.cs new file mode 100644 index 000000000..3327f2485 --- /dev/null +++ b/test/SEBT.Portal.Tests/Unit/Services/PiiPlaintextEncryptionBackfillTests.cs @@ -0,0 +1,192 @@ +using Microsoft.EntityFrameworkCore; +using Microsoft.Extensions.Logging.Abstractions; +using SEBT.Portal.Core.Exceptions; +using SEBT.Portal.Core.Services; +using SEBT.Portal.Core.Utilities; +using SEBT.Portal.Infrastructure.Data.Entities; +using SEBT.Portal.Infrastructure.Helpers; +using SEBT.Portal.Infrastructure.Services; +using SEBT.Portal.Tests.Unit.Repositories; +using SEBT.Portal.Tests.Unit.TestSupport; +using Xunit; + +namespace SEBT.Portal.Tests.Unit.Services; + +/// +[Collection("SqlServer")] +[Trait("Category", "SqlServer")] +public class PiiPlaintextEncryptionBackfillTests : IClassFixture +{ + private readonly SqlServerTestFixture _fixture; + + public PiiPlaintextEncryptionBackfillTests(SqlServerTestFixture fixture) + { + _fixture = fixture; + } + + [Fact] + public async Task ApplyAsync_invokedTwice_leaves_stable_envelopes_and_hashes() + { + using var db = _fixture.CreateContext(); + var email = $"backfill-twice-{Guid.NewGuid()}@example.com"; + + var user = UserFactory.CreateUserEntity(e => + { + e.Email = email; + e.EmailHash = null; + e.Phone = "+15551234001"; + e.SnapId = "snap-plain"; + e.TanfId = "tanf-plain"; + e.DateOfBirth = "1988-06-01"; + }); + db.Users.Add(user); + await db.SaveChangesAsync(); + + var challenge = new DocVerificationChallengeEntity + { + PublicId = Guid.CreateVersion7(), + UserId = user.Id, + ProofingDateOfBirth = "1990-01-15", + ProofingIdType = "itin", + ProofingIdValue = "111223333" + }; + db.DocVerificationChallenges.Add(challenge); + await db.SaveChangesAsync(); + + async Task ApplyBackfillOnFreshContextAsync() + { + using var fresh = _fixture.CreateContext(); + var backfill = new PiiPlaintextEncryptionBackfill( + fresh, + TestPortalCryptography.PiiSymmetricEncryption, + TestPortalCryptography.EmailLookupHasher, + NullLogger.Instance); + await backfill.ApplyAsync(); + } + + await ApplyBackfillOnFreshContextAsync(); + + db.ChangeTracker.Clear(); + var storedUserAfter1 = db.Users.Single(u => u.Id == user.Id); + var fingerprint = TestPortalCryptography.FingerprintEmail(email); + + Assert.Equal(fingerprint, storedUserAfter1.EmailHash); + Assert.StartsWith(PiiAesGcmSymmetricEncryption.EnvelopePrefix, storedUserAfter1.Email!, StringComparison.Ordinal); + Assert.StartsWith(PiiAesGcmSymmetricEncryption.EnvelopePrefix, storedUserAfter1.Phone!, StringComparison.Ordinal); + + Assert.Equal( + "1988-06-01", + TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(storedUserAfter1.DateOfBirth)); + + var storedChallengeAfter1 = db.DocVerificationChallenges.Single(c => c.Id == challenge.Id); + Assert.StartsWith( + PiiAesGcmSymmetricEncryption.EnvelopePrefix, + storedChallengeAfter1.ProofingDateOfBirth!, + StringComparison.Ordinal); + Assert.Equal( + "1990-01-15", + TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy( + storedChallengeAfter1.ProofingDateOfBirth)); + + // Second run: idempotent (no duplicate processing / exceptions). + await ApplyBackfillOnFreshContextAsync(); + + db.ChangeTracker.Clear(); + var storedUserAfter2 = db.Users.Single(u => u.Id == user.Id); + Assert.Equal(storedUserAfter1.Email, storedUserAfter2.Email); + Assert.Equal(storedUserAfter1.EmailHash, storedUserAfter2.EmailHash); + + var storedChallengeAfter2 = db.DocVerificationChallenges.Single(c => c.Id == challenge.Id); + Assert.Equal(storedChallengeAfter1.ProofingDateOfBirth, storedChallengeAfter2.ProofingDateOfBirth); + + Assert.Equal( + "snap-plain", + TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(storedUserAfter2.SnapId)); + Assert.Equal( + EmailNormalizer.Normalize(email), + TestPortalCryptography.PiiSymmetricEncryption.DecryptOrPassThroughLegacy(storedUserAfter2.Email)); + } + + [Fact] + public async Task ApplyAsync_whenDecryptFails_doesNotPartiallyCommitBatch() + { + using var seedDb = _fixture.CreateContext(); + var firstEmail = $"backfill-fail-first-{Guid.NewGuid()}@example.com"; + var secondEmail = $"backfill-fail-second-{Guid.NewGuid()}@example.com"; + + seedDb.Users.Add(UserFactory.CreateUserEntity(e => + { + e.Email = firstEmail; + e.EmailHash = null; + e.Phone = "+15555550101"; + })); + seedDb.Users.Add(UserFactory.CreateUserEntity(e => + { + e.Email = secondEmail; + e.EmailHash = null; + e.Phone = "+15555550102"; + })); + await seedDb.SaveChangesAsync(); + + // Throw on second row in the same batch to verify SaveChanges is not partially applied. + var throwingCrypto = new ThrowOnNthDecryptCrypto( + inner: TestPortalCryptography.PiiSymmetricEncryption, + throwOnCall: 2); + + using var runDb = _fixture.CreateContext(); + var backfill = new PiiPlaintextEncryptionBackfill( + runDb, + throwingCrypto, + TestPortalCryptography.EmailLookupHasher, + NullLogger.Instance); + + await Assert.ThrowsAsync(() => backfill.ApplyAsync()); + + using var verifyDb = _fixture.CreateContext(); + var users = await verifyDb.Users + .Where(u => u.Email == firstEmail || u.Email == secondEmail) + .OrderBy(u => u.Email) + .ToListAsync(); + + Assert.Equal(2, users.Count); + Assert.All(users, u => + { + Assert.DoesNotContain(PiiAesGcmSymmetricEncryption.EnvelopePrefix, u.Email ?? string.Empty, StringComparison.Ordinal); + Assert.Null(u.EmailHash); + Assert.DoesNotContain(PiiAesGcmSymmetricEncryption.EnvelopePrefix, u.Phone ?? string.Empty, StringComparison.Ordinal); + }); + } + + private sealed class ThrowOnNthDecryptCrypto : IPiiSymmetricEncryption + { + private readonly IPiiSymmetricEncryption _inner; + private readonly int _throwOnCall; + private int _callCount; + + public ThrowOnNthDecryptCrypto(IPiiSymmetricEncryption inner, int throwOnCall) + { + _inner = inner; + _throwOnCall = throwOnCall; + } + + public bool IsEnvelope(string? storedValue) => _inner.IsEnvelope(storedValue); + + public string? Encrypt(string? plaintext) => _inner.Encrypt(plaintext); + + public string Decrypt(string storedValue) => _inner.Decrypt(storedValue); + + public string? DecryptOrPassThroughLegacy(string? storedValue) + { + _callCount++; + if (_callCount == _throwOnCall) + { + throw new PiiDecryptException("Synthetic decrypt failure for batch rollback test."); + } + + return _inner.DecryptOrPassThroughLegacy(storedValue); + } + + public string ReSealWithActiveEncryptor(string envelopeCiphertext) => + _inner.ReSealWithActiveEncryptor(envelopeCiphertext); + } +} diff --git a/test/SEBT.Portal.Tests/Unit/Startup/PiiEncryptionGuardTests.cs b/test/SEBT.Portal.Tests/Unit/Startup/PiiEncryptionGuardTests.cs new file mode 100644 index 000000000..d2687b913 --- /dev/null +++ b/test/SEBT.Portal.Tests/Unit/Startup/PiiEncryptionGuardTests.cs @@ -0,0 +1,141 @@ +using SEBT.Portal.Api.Startup; +using SEBT.Portal.Core.AppSettings; +using Xunit; + +namespace SEBT.Portal.Tests.Unit.Startup; + +public class PiiEncryptionGuardTests +{ + [Fact] + public void ValidateForProduction_WhenNull_Throws() + { + var ex = Assert.Throws(() => + PiiEncryptionGuard.ValidateForProduction(null)); + Assert.Contains("PiiEncryption", ex.Message, StringComparison.Ordinal); + } + + [Fact] + public void ValidateForProduction_WhenEmptyActiveKeyId_Throws() + { + var ex = Assert.Throws(() => + PiiEncryptionGuard.ValidateForProduction( + new PiiEncryptionSettings + { + ActiveKeyId = " ", + Keys = [new PiiEncryptionKeySetting { KeyId = "k1", KeyMaterialBase64 = "YjJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmI=" }] + })); + Assert.Contains("ActiveKeyId", ex.Message, StringComparison.Ordinal); + } + + [Fact] + public void ValidateForProduction_WhenNoKeys_Throws() + { + var ex = Assert.Throws(() => + PiiEncryptionGuard.ValidateForProduction( + new PiiEncryptionSettings { ActiveKeyId = "prod-key", Keys = [] })); + Assert.Contains("Keys", ex.Message, StringComparison.Ordinal); + } + + [Fact] + public void ValidateForProduction_WhenDevelopmentActiveKeyId_Throws() + { + var ex = Assert.Throws(() => + PiiEncryptionGuard.ValidateForProduction( + new PiiEncryptionSettings + { + ActiveKeyId = PiiEncryptionGuard.ForbiddenDevelopmentActiveKeyId, + Keys = + [ + new PiiEncryptionKeySetting + { + KeyId = "k1", + KeyMaterialBase64 = "YjJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmI=" + } + ] + })); + Assert.Contains("local-dev", ex.Message, StringComparison.OrdinalIgnoreCase); + } + + [Fact] + public void ValidateForProduction_WhenNullKeyEntry_Throws() + { + var ex = Assert.Throws(() => + PiiEncryptionGuard.ValidateForProduction( + new PiiEncryptionSettings + { + ActiveKeyId = "prod-key", + Keys = [null!, new PiiEncryptionKeySetting { KeyId = "prod-key", KeyMaterialBase64 = "YjJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmI=" }] + })); + Assert.Contains("null", ex.Message, StringComparison.OrdinalIgnoreCase); + } + + [Fact] + public void ValidateForProduction_WhenEmptyKeyId_Throws() + { + var ex = Assert.Throws(() => + PiiEncryptionGuard.ValidateForProduction( + new PiiEncryptionSettings + { + ActiveKeyId = "prod-key", + Keys = + [ + new PiiEncryptionKeySetting { KeyId = " ", KeyMaterialBase64 = "YjJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmI=" } + ] + })); + Assert.Contains("KeyId", ex.Message, StringComparison.Ordinal); + } + + [Fact] + public void ValidateForProduction_WhenEmptyKeyMaterial_Throws() + { + var ex = Assert.Throws(() => + PiiEncryptionGuard.ValidateForProduction( + new PiiEncryptionSettings + { + ActiveKeyId = "prod-key", + Keys = + [ + new PiiEncryptionKeySetting { KeyId = "prod-key", KeyMaterialBase64 = " " } + ] + })); + Assert.Contains("KeyMaterialBase64", ex.Message, StringComparison.Ordinal); + } + + [Fact] + public void ValidateForProduction_WhenPlaceholderKeyMaterial_Throws() + { + var ex = Assert.Throws(() => + PiiEncryptionGuard.ValidateForProduction( + new PiiEncryptionSettings + { + ActiveKeyId = "prod-key", + Keys = + [ + new PiiEncryptionKeySetting + { + KeyId = "k1", + KeyMaterialBase64 = PiiEncryptionGuard.ForbiddenPlaceholderKeyMaterialBase64 + } + ] + })); + Assert.Contains("placeholder", ex.Message, StringComparison.OrdinalIgnoreCase); + } + + [Fact] + public void ValidateForProduction_WhenValid_DoesNotThrow() + { + PiiEncryptionGuard.ValidateForProduction( + new PiiEncryptionSettings + { + ActiveKeyId = "prod-key", + Keys = + [ + new PiiEncryptionKeySetting + { + KeyId = "prod-key", + KeyMaterialBase64 = "YjJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmI=" + } + ] + }); + } +} diff --git a/test/SEBT.Portal.Tests/Unit/TestSupport/TestPortalCryptography.cs b/test/SEBT.Portal.Tests/Unit/TestSupport/TestPortalCryptography.cs new file mode 100644 index 000000000..bbeee708f --- /dev/null +++ b/test/SEBT.Portal.Tests/Unit/TestSupport/TestPortalCryptography.cs @@ -0,0 +1,51 @@ +using Microsoft.Extensions.Options; +using SEBT.Portal.Core.AppSettings; +using SEBT.Portal.Core.Services; +using SEBT.Portal.Core.Utilities; +using SEBT.Portal.Infrastructure.Services; + +namespace SEBT.Portal.Tests.Unit.TestSupport; + +/// +/// Shared reversible PII + email-hash services for repositories and seed integration tests (deterministic AES-256-GCM keys). +/// +public static class TestPortalCryptography +{ + public static readonly string TestIdentifierHasherSecretKey = "TestKeyMustBeAtLeast32CharactersLong!!"; + + public static readonly IOptions PiiOptions = Options.Create( + new PiiEncryptionSettings + { + ActiveKeyId = "test-primary", + Keys = + [ + new PiiEncryptionKeySetting + { + KeyId = "test-primary", + KeyMaterialBase64 = + // 32 repetitions of ASCII 'b' — decoded length must be exactly 256 bits for AES-256-GCM key. + "YmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmI=" + } + ] + }); + + public static readonly IOptions IdentifierOptions = + Options.Create(new IdentifierHasherSettings { SecretKey = TestIdentifierHasherSecretKey }); + + public static readonly IPiiSymmetricEncryption PiiSymmetricEncryption = + new PiiAesGcmSymmetricEncryption(PiiOptions); + + public static readonly IEmailLookupHasher EmailLookupHasher = + new EmailLookupHasher(IdentifierOptions); + + public static string NormalizeEmailStrict(string plaintext) => + EmailNormalizer.Normalize(plaintext); + + public static string FingerprintEmail(string anyCasePlaintextEmail) => + EmailLookupHasher.HashNormalized(NormalizeEmailStrict(anyCasePlaintextEmail))!; + + public static string StoredEmailPlaintext(string? ciphertext) => + string.IsNullOrEmpty(ciphertext) + ? "" + : PiiSymmetricEncryption.DecryptOrPassThroughLegacy(ciphertext)!; +} diff --git a/test/SEBT.Portal.Tests/Unit/UseCases/Auth/ValidateOtpCommandHandlerTests.cs b/test/SEBT.Portal.Tests/Unit/UseCases/Auth/ValidateOtpCommandHandlerTests.cs index 32ee21218..7169f967a 100644 --- a/test/SEBT.Portal.Tests/Unit/UseCases/Auth/ValidateOtpCommandHandlerTests.cs +++ b/test/SEBT.Portal.Tests/Unit/UseCases/Auth/ValidateOtpCommandHandlerTests.cs @@ -453,8 +453,7 @@ public async Task Handle_ShouldPassUserWithIalLevel_ToJwtTokenService() Email = command.Email, IalLevel = UserIalLevel.IAL1, IdProofingSessionId = "session-123", - IdProofingCompletedAt = null, - IdProofingExpiresAt = DateTime.UtcNow.AddYears(1) + IdProofingCompletedAt = null }; otpRepository.GetOtpCodeByEmailAsync(Arg.Is(email => email == command.Email))