How to use Redis to build recently viewed items functionality

Its a very common feature, specially in eCommerce site to display a list of recently viewed items. The idea is the application keep track of a certain number of items user viewed and display them. This helps user to quickly view any items that he/she previously viewed without searching the store again.

Lets say we have to store maximum 5 recent items. So anytime user view details of an item then we need to save that item against user id and when we display the list of recently viewed items, that item should be at top. If a user view an item that he/she viewed before than instead of adding that item we need move the item to top. After adding an item we also need to check how many items added. If the maximum items per user reached then we need to remove the item that has been viewed by user least recently. You most probably don’t want to store those recent items forever. At the same time you don’t want  to expire them at any specific future time. The lifespan of users recent items needs to slide as the user browse.

This could be a perfect use case for Redis. The redis datatype that best fits for this requirement is SortedSet. Because SortedSet store items in order based on score and make sure there’s not duplicate item. At the same time sorted set allow you to remove items by index range. All these features matches with the requirements we have.

In this sample I am using StackExchange.Redis nuget package. We need a class that can make sure only one instance of ConnectionMultiplexer created as creation of this class is costly and should be shared accross your application.

// Make sure you initialize this class in your IOC as singleton.
public class ConnectionFactory : IConnectionFactory
{
    private readonly string _connectionString;
    private ConnectionMultiplexer _instance;
    private static volatile object _syncLock = new object();

    public ConnectionFactory(string connectionString)
    {
        _connectionString = connectionString;
    }

    public ConnectionMultiplexer Create()
    {
        if (_instance != null && _instance.IsConnected) return _instance;

        lock (_syncLock)
        {
            if (_instance != null && _instance.IsConnected) return _instance;

            if (_instance != null)
            {
                _instance.Dispose();
            }

            _instance = ConnectionMultiplexer.Connect(_connectionString);
        }

        return _instance;
    }

    public void Dispose()
    {
        if (_instance != null)
        {
            _instance.Dispose();
        }
    }
}

Here is the sample repository that shows how to use SortedSet to implement RecentItems feature:

public class RecentBooksRepository : IRecentBooksRepository
{
    private static readonly DateTime BaseDate = new DateTime(2014,10,1,0,0,0,DateTimeKind.Utc); 
    private const string RecentItemsSetKey = "books:ri:{0}";
    private readonly IConnectionFactory _factory;
    private readonly ISettings _settings;

    public RecentBooksRepository(IConnectionFactory factory, ISettings settings)
    {
        _factory = factory;
        _settings = settings;
    }

    public Task AddAsync(Guid userId, long bookId)
    {
        var setKey = SetKey(userId);

        var transaction = Db().CreateTransaction();

        var score = BaseDate.Ticks - DateTime.UtcNow.Ticks;
        
        // add item with score
        transaction.SortedSetAddAsync(setKey, bookId, score);

        // This will remove any items from set after max items
        transaction.SortedSetRemoveRangeByRankAsync(setKey, _settings.MaxItems, _settings.MaxItems + 1);

        // Apply or update lifespan of this sorted set (Sliding Expiry)
        transaction.KeyExpireAsync(setKey, TimeSpan.FromDays(_settings.LifeSpanInDays));
        
        return transaction.ExecuteAsync();
    }

    public async Task<IEnumerable<long>> GetAsync(Guid userId)
    {
        var setKey = SetKey(userId);

        var transaction = Db().CreateTransaction();

        var getSetItemsTask = transaction.SortedSetRangeByRankAsync(
            setKey, 
            0, 
            _settings.MaxItems - 1, 
            Order.Ascending, 
            CommandFlags.PreferSlave);

        // Apply or update lifespan of this sorted set (Sliding Expiry)
        transaction.KeyExpireAsync(setKey, TimeSpan.FromDays(_settings.LifeSpanInDays));

        await transaction.ExecuteAsync();
        
        return getSetItemsTask.Result.Select(x => ((string)x).ToLong() ?? 0);
    }

    private string SetKey(Guid userId)
    {
        return RecentItemsSetKey.FormatWith(userId);
    }

    private IDatabase Db()
    {
        return _factory.Create().GetDatabase(_settings.Db);
    }
}

public static class StringExtensions
{
    public static string FormatWith(this string source, params object[] args)
    {
        return string.Format(source, args);
    }
}

public static class NumericExtensions
{
    public static long? ToLong(this string source)
    {
        long result;
        return long.TryParse(source, out result) ? result : (long?)null;
    }
}

Hope this will give you clear idea how you can use redis to achieve this functionality.

You most probably noticed that I am not storing books data in sorted set just ids of books. That means after getting most recent item ids I need to load book data for all those ids. The main reason behind this is that book data like price might change and then its hard to sync all users recent items which contains that book id. That’s why its better to store a small set of books data in redis by book id separately instead of in the sortedset. That also allow other functionality share same book data in redis if required. When a price of book change, its easy to update that specific book details in redis. This actually required another blog post and out of scope of this post.

That’s all for today. Happy coding 🙂

Advertisements

3 thoughts on “How to use Redis to build recently viewed items functionality

  1. Nicely done – thanks for sharing this howto! One question though – is there a reason you had decided to use the Sorted Set data structure instead of a List? Although both can be used for this purpose, the list is slightly more efficient in terms its memory consumption and the computational complexity needed for pushing/trimming it (vs. adding and removing by score from a Sorted Set).

    • Hi Itamar, thanks for you comments. I thought about List but one of the requirement is the item needs to be unique. SortedSet provide this by default. In list you have to maintain this by yourself as below, that’s why I go for SortedSet.


      var transaction = Db().CreateTransaction();

      // make sure we don't have any existing item in list
      transaction.ListRemoveAsync(listKey, bookId);

      // add item top of list
      transaction.ListLeftPushAsync(listKey, bookId);

      // keep the size of list within maximum range
      transaction.ListTrimAsync(listKey, 0, _settings.MaxItems - 1);

      // Apply or update lifespan of this list (Sliding Expiry)
      transaction.KeyExpireAsync(listKey, TimeSpan.FromDays(_settings.LifeSpanInDays));

      return transaction.ExecuteAsync();

      I didn’t compare the performance of both approach. Removing item from list by matching could be less per-formative too. If list contains only 5/10 items then may be nothing to worry.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s