How to use data redaction in .net core web api

Table of contents

Intro

Applying masking or obscuring your senistive data in logs, error messages or other sources keeps us in compliance with privacy rules and protecting sensitive data.

We will be seeing in step by step on how to achieve this .net core web api

Step 1 - Packages required

dotnet add package Microsoft.Extensions.Compliance.Redaction
dotnet add package Microsoft.Extensions.Telemetry
dotnet add package Microsoft.Extensions.Telemetry.Abstractions

Step 2 - Defining your data classification

We need to classify our data to apply the appropriate attributes for the data.

Below the class specify a classification

public static class MyTaxonomyClassifications
{
    public static string Name => "MyTaxonomy";
    public static DataClassification Private => new(Name, nameof(Private));
}

After data classification, next step is to create an attribute to use it our dto classes

public sealed class PrivateDataAttribute : DataClassificationAttribute
{
    public PrivateDataAttribute() : base(MyTaxonomyClassifications.Private) { }
}

Step 3 - Applying in User Data

We have an user class that has email id in order to protect the senitive information, we apply the attribute PrivateDataAttribute

public record User(string Name, int Age, [PrivateData] string Email);

Step 4 - Defining the Redactor

By default, we have the ErasingRedactor that replaces any input with an empty string and the HmacRedactor uses HMACSHA256 to encode data being redacted.

For this, we use a custom star redactor which replaces the string with stars, it should inherit the Redactor

public sealed class StarRedactor : Redactor
{
    private const string Stars = "****";

    public override int GetRedactedLength(ReadOnlySpan<char> input) => Stars.Length;

    public override int Redact(ReadOnlySpan<char> source, Span<char> destination)
    {
        Stars.CopyTo(destination);

        return Stars.Length;
    }
}

Step 5 - Registring the Redactor configuration

Register our custom redactor in the Service collection

builder.Services.AddRedaction(options =>
{
    // Configure redaction options here if needed
    options.SetRedactor<StarRedactor>(MyTaxonomyClassifications.Private);
});

It is important to specify in the Logging to enable redaction to mask sensitive information

builder.Services.AddLogging(builder =>
{
    // Enable redaction.
   builder.EnableRedaction();
});

We are adding Json console inorder to view the state of our object is santized by our custom redactor, as the default console logger would not show the state. In Azure Application insights you would able to see the masking done in the custom properties.

builder.Logging.AddJsonConsole(options =>
{
    options.JsonWriterOptions = new System.Text.Json.JsonWriterOptions { Indented = true };
});

Step 6 - Defining the Logger extension methods

By default, the logger doesn’t validate the attributes in order to enforce it, we should express the LogProperties in the respective object.

Here we define an custom extension for the user object

public static partial class LogExtensions
{
    [LoggerMessage(
        EventId = 5001,
        Level = LogLevel.Information,
        Message = "Processing request for user")]
    public static partial void LogUserRegistrationAttempt(
        this ILogger logger,
        [LogProperties] User user);
}

Step 7 - Final Step

Everything wired up now, inorder to test the santization of sensitive data, created the below post endpoint which would receive the User data and log it.

app.MapPost("/user", (User user, ILogger<User> logger) =>
{
    // Process the user data here
    logger.LogUserRegistrationAttempt(user);
    return Results.Ok(user);
}).WithName("CreateUser").WithOpenApi();

Supplied an sample request

{
  "name": "John",
  "age": 24,
  "email": "[email protected]"
}

In the console, we able to see the state of the data is santized for the sensitive information.

{
  "EventId": 5001,
  "LogLevel": "Information",
  "Category": "User",
  "Message": "Processing registration request for user",
  "State": {
    "Message": "Microsoft.Extensions.Logging.ExtendedLogger\u002BModernTagJoiner",
    "user.Email": "****", -> `Data is masked`
    "user.Age": 24,
    "user.Name": "John",
    "{OriginalFormat}": "Processing registration request for user"
  }
}

Final thoughts

This example is generated in-order to give a basic understanding of the redaction in .net core web api, for more in-depth understanding, pls refer MS Docs