What is hybrid cache and why should you use it

Table of contents

What is cache

A cache is a temporary storage location that holds frequently accessed data so it can be served quickly to future requests.

In an application, the cache stored in the hardware memory (RAM) and when the requests comes in, it checks the cache first and returns the data if it’s present.

Problems in cache

• Stale data - Data needs to be refreshed immediately when it’s not in sync
• Cache stampede - Heavily used data expires and surge of requests comes-in accessing the key causes a stampede
• Cold start - At initial state the cache is empty and it’s need to pull the data from the original source.
• Consistency across distrubuted cache - In a multi node /server environment where a single node / server the data gets refreshed but it’s not reflected in other servers / node.

How Hybrid cache solves the problems

Hybrid cache (typically combining a local in-memory cache with a distributed cache like Redis often called L1/L2 caching) solves these problems

1. Stale data

Problem: Instance A updates a product’s name in the DB, but Instance B’s local cache still has the old product name and there is no way for A to tell B to refresh.

Hybrid cache fix: When Instance A updates the product name, it writes through to the distributed cache (Redis) and invalidates/updates that key. Instance B’s local cache entry has a short TTL (say, 5 seconds), so it expires quickly and re-fetches from Redis and getting the fresh value instead of waiting on a stale local copy for minutes.

2. Cache stampede

Problem: A popular product page’s cache entry expires. 10,000 concurrent requests all miss the cache at once and hammer the database simultaneously.

Hybrid cache fix: Local caches on each instance expire at slightly different times, so not all instances miss at once. And even when a local cache misses, it checks the distributed cache first, which likely still has the data (or one instance repopulates it) so the database only gets hit once, not 10,000 times.

3. Cold start

Problem: You deploy a new instance (or restart one after a crash). Its local cache is empty, so the first wave of requests all go straight to the database, causing a latency spike.

Hybrid cache fix: The new instance’s local cache is empty, but it checks the distributed cache (Redis) first which already has data warmed up by other running instances. So requests are served from Redis (fast) instead of the database (slow), while the local cache gradually warms up in the background.

4. Consistency across nodes

Problem: Instance A caches user -> {name: "Vignesh"} . User updates their name to “Vig”. Instance B still serves “Vignesh” from its own local cache indefinitely, with no shared source of truth.

Hybrid cache fix: The distributed cache (Redis) acts as the shared source of truth between nodes. When the update happens, Redis is updated/invalidated. Each instance’s local cache has a short TTL, so within a few seconds, all instances re-sync from Redis and converge on “Vig” — instead of staying permanently out of sync.

The leverage of in-memory with a distributed cache solves the common cache problem but with a minimal seconds delay which is negligible.

How to implement Hybrid cache

• Hybrid cache package

dotnet add package Microsoft.Extensions.Caching.Hybrid --version 9.3.0

• Register the service

// Add services to the container.
var builder = WebApplication.CreateBuilder(args);

//Add hybrid cache
builder.Services.AddHybridCache();

• Get and save cache items

The GetOrCreateAsync method is the most simplest and recommended way for the most scenarios, we can simply inject the HybridCache and use the method to cache the item retrieved from a data source.

public class SomeService(HybridCache cache)
{
    private HybridCache _cache = cache;

    public async Task<string> GetSomeInfoAsync(string name, int id, CancellationToken token = default)
    {
        return await _cache.GetOrCreateAsync(
            $"{name}-{id}", // Unique key to the cache entry
            async cancel => await GetDataFromTheSourceAsync(name, id, cancel),
            cancellationToken: token
        );
    }

    public async Task<string> GetDataFromTheSourceAsync(string name, int id, CancellationToken token)
    {
        // Placeholder for retrieving data from a real backing store (database, HTTP API, etc.).
        // A real implementation would await that call; this stub returns synchronously,
        // so it intentionally has no await (and produces compiler warning CS1998).
        string someInfo = $"someinfo-{name}-{id}";
        return someInfo;
    }
}

• Remove Cache entries

We can leverage the method RemoveAsync with the key to remove the cached entry from the cache.

• Remove Cache entries by tags

We can use tags to invalidate all the cache entries that are related to a tag this eliminates need the need of removing the entries key by key.

        var tags = new List<string> { "tag1", "tag2", "tag3" };
        var entryOptions = new HybridCacheEntryOptions
        {
            Expiration = TimeSpan.FromMinutes(1),
            LocalCacheExpiration = TimeSpan.FromMinutes(1)
        };
        return await _cache.GetOrCreateAsync(
            $"{name}-{id}", // Unique key to the cache entry
            async cancel => await GetDataFromTheSourceAsync(name, id, cancel),
            entryOptions,
            tags, //bind the entries with tag
            cancellationToken: token
        );

        RemoveByTagAsync("*") //remove all tags
        RemoveByTagAsync("tag1") //remove all the cache that holds the `tag1` tag

• Integration with Redis

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = 
        builder.Configuration.GetConnectionString("RedisConnectionString");
});

The secondary storage is not limited Redis you can use Postgres, SqlServer as well.

Now, we have got the idea of how to implement the HybridCache, next we can see the limitations.

Limitations

The following properties of HybridCacheOptions let you configure limits that apply to all cache entries:

MaximumPayloadBytes - Maximum size of a cache entry. Default value is 1 MB. Attempts to store values over this size are logged, and the value isn’t stored in cache. MaximumKeyLength - Maximum length of a cache key. Default value is 1024 characters. Attempts to store values over this size are logged, and the value isn’t stored in cache.

Final thoughts

The article highlighted the benefits of HybridCache and the easier way to implement in the repository. It is wise to make decision upon yourself to use it based on the needs of your architecture.