Recently I worked on configuring a caching policy, but only wanted to cache the returned models from an API call in certain cases depending on the values of the model received and that's what this blog post is about.

The challenge

If you just want to selectively cache for a specific response type, e.g. HttpResponseMessage, then that is actually not too difficult and you can read about how to do that in the following blog post, where the policy gets configured to only cache 200 OK responses.

It's basically just a ITtlStrategy that in certain situations returns a "time-to-live" of 0, meaning the item expires from the cache immediately.

Caching at the HttpResponseMessage level might not be the right thing though, and there are some thing to consider in that regard, like re-reading response streams, deserializing the response again etc. Instead I wanted to selectively cache the deserialized result model. Doing this for different models seemed too cumbersome, as I would have to create specific policies for each return type (because of generics, i.e. ResultTtl<T>).

The solution

The solution I came up with was to create a custom ITtlStrategy that would look for a specific interface and choose depending on that.

Here's the interface ICacheable:

public interface ICacheable
{
    /// <summary>
    /// Should return true if the object shouldn't be cached in its current state.
    /// </summary>
    bool DisallowCaching();
}

Any model can implement that interface and return a fitting value depending on it's current state.

For this interface to have any effect we need a ITtlStrategy, and here it is:

/// <summary>
/// Dynamic TTL strategy that doesn't cache the item if it implements
/// <see cref="ICacheable"/> interface and doesn't allow caching in its current state.
/// </summary>
public class CacheableFilterTtl : ITtlStrategy
{
    private static readonly ZeroTtl = new Ttl(TimeSpan.Zero);

    private readonly ITtlStrategy _innerStrategy;

    public CacheableFilterTtl(ITtlStrategy innerStrategy)
    {
        _innerStrategy = innerStrategy ?? throw new ArgumentNullException(nameof(innerStrategy));
    }

    public CacheableFilterTtl(TimeSpan ttl)
    {
        _innerStrategy = new RelativeTtl(ttl);
    }

    /// <inheritdoc />
    public Ttl GetTtl(Context context, object result)
    {

        if (result is ICacheable cacheable && cacheable.DisallowCaching())
        {
            return ZeroTtl;
        }

        return _innerStrategy.GetTtl(context, result);
    }
}

It's rather simple, really. It can either be created as wrapping another ITtlStrategy as a fallback strategy, or with a simple RelativeTtl based on a TimeSpan.

When figuring out the Ttl to return it looks at the result and if it implements ICacheable and does not allow caching then it returns a Ttl of 0. If the result is not of type ICacheable then it just uses the fallback strategy.

Creating the cache policy using the custom ITtlStrategy is very simple:

IAsyncCacheProvider cacheProvider = null; // Get from Service Provider
Policy.CacheAsync(cacheProvider, new CacheableFilterTtl(TimeSpan.FromHours(1));

This will create a cache policy with a relative time-to-live of 1 hour as the default, unless the result implements ICacheable and does not allow caching.

• Part 1: Getting started
• Part 2: Naming, structure, and readability (FluentAssertions)
• Part 3: Mocking and writing testable code (NSubstitute)
• Part 4: Keeping it DRY and maintainable (AutoFixture)
• Part 5: Test Driven Development (TDD) with Live Unit Testing (LUT)

Naming tests

Test method names should generally be detailed, descriptive and readable.

We don't really care about normal naming conventions here, because we want to be able to tell what a test is supposed to test for at a glance. This also helps when a test inevitably fails; you don't have to dig into the code to figure out what scenario is actually failing.

The test class itself should normally just match the class your are testing postfixed with Tests, e.g. BasketTests.

For the naming of the test methods I usually go by a pattern like this {method-name}_{scenario}_{expected-outcome}. Some examples:

1. AddItem_NullItem_ThrowsArgumentNullException
2. AddItem_NewItem_AddsNewItem
3. AddItem_ExistingItem_DoesNotAddNewItem
4. AddItem_ExistingItem_ShouldIncreaseQuantityOfExistingItem

AddItem is the method being tested.

NullItem, NewItem, ExistingItem indicates a sort of input for the test, or as a way to sort of grouping the tests by a specific scenario.

The last part is self-explanatory and just rather precisely describes what we expect to happen, given the scenario.

Compare that to test names like these:

• AddNullItemTest
• AddNewItemTest
• AddExistingItemTest

You can't tell much just from the name of these; you will have to look at the code to see what is actually being tested and what is causing it to fail. What method is actually being tested? What was the expected outcome?

This also illustrates a common thing - maybe the tests are testing too much? Keeping the tests simple and focused on a single thing instead of a bit of everything will help make the tests more readably and easier to debug.

What parts of your code to test

A common issue I hear or notice is that many people are unsure of what to actually test.

As I see it there at least two ways to approach this:

1. From the outside. Looking at the method signature and the inputs, what do you expect the outcome should be?
2. From the inside. Looking at the code where does it branch into different scenarios and edge-cases (if-conditions, exceptions etc.)

Starting with the 1. case - looking at it from the outside - is sort of close to Test Driven Development (TTD) in my eyes, where you can have some requirements that should be matched by the code which you want to verify. It could be acceptance criteria of a feature request or the expected outcome detailed in a bug report.

An example could be the basket example from Part 1 of this series. A requirement for when adding items is that it should not create a new (duplicate) item in the basket if the item already exist, instead it should add the quantity to the existing item. Here we are just looking at the input and expected outcome.

The 2. approach is to look at it from the inside; looking at the code itself and if there are any obvious logic branches or handling of edge cases. You can then add tests to make sure cover all the different scenarios and thereby increase your code coverage (although just because the code is covered doesn't mean that all cases are covered).

I find that approach number 1 is best at finding requirements that were missed during development, while number 2 is good for legacy code before adding new features or doing refactoring, because it helps ensure the existing functionality doesn't break.

Structuring tests

When writing test they should really - like most stuff - be structured for readability and maintainability. As much as possible you want to easily be able to tell at a glance what a test does.

I always follow the "Arrange, Act and Assert" pattern (AAA). Shortly explained that means:

1. Arrange: Setup and prepare everything for your test
2. Act: Execute the code to be tested
3. Assert: Verify that the results are as expected

It's not strictly necessary but I almost always add a comment above each section unless it's 2-liner or something very very simple. It helps me to visually separate the sections and allows me to have empty lines between logically grouped arrange-steps.

[Fact]
public void AddItem_ExistingItem_DoesNotAddNewItem()
{
    // Arrange
    var itemNo = "qwerty";
    var basket = new Basket
    {
        Items = new List<BasketItem>
        {
            new BasketItem
            {
                ItemNo = itemNo,
                Quantity = 1
            }
        }
    };
    Assert.Equal(1, basket.Items.Count);

    // Act
    basket.AddItem(itemNo, 1);

    // Assert
    Assert.Equal(1, basket.Items.Count);
}

Example of the Arrange-Act-Assert pattern with comments for each section to visually separate them.

Readability versus repetition (DRY)

Usually I prefer to have a bit of repetition in favor of sacrificing readability. It can often be tempting - and I see this often - to extract the arrange-section into a separate method and just call that as the setup for a lot of different test, e.g. a CreateBasket() setup method.

That can be okay in some scenarios, but what usually ends up happening is that the setup-method is then made to work with all your different tests and has some setup that is not necessary for every test and might even mask some issues that would have been noticed if the setup was done more specifically for each test. It also make it harder to see at a glance what the setup for the test actually is without having to jump to that method.

Sometimes a compromise can be made if you use the same setup for a group of tests and another setup for another group of tests. Then you could have setup-methods like CreateEmptyBasket(), CreateBasketWith3Items() etc., making the method describe the setup being done.

Test a single thing per test

The last thing I want to mention regarding the structure of your tests is try to focus on testing a single thing per test. If you test too much in a single test and the test fails, then it will not be immediately apparent what is actually failing.

If you have a failed test named something like TestThatBasketWorks() which tests both adding and deleting items, then you won't know if it is adding or deleting items that failed the test and you have to spent time figuring that out first.

If you instead have 2 tests - AddItem_AddsANewItem() and DeleteItem_RemovesItem() - and one of them fails, then you know right away which part of the basket has broken.

Increasing readability of assertions and error messages

I always use the FluentAssertionspackage when writing unit tests.

<PackageReference Include="FluentAssertions" Version="5.10.3" />
<PackageReference Include="FluentAssertions.Analyzers" Version="0.11.4" />

It's a very nice set of extension methods that helps you write more natural assertions and also more descriptive error messages in case a test fails. You simply add a call to the extension method Should() after the object/value that you want to do one ore more assertions on and then you can chain other calls like Be("something") or BeTrue() or HaveCount(2) for verifying the amount of items in a collection.

Go check out their "Getting stared" section in the documentation for more information.

Assert.NotNull(basket.Items);
Assert.Empty(basket.Items);

// With FluentAssertions we can write the above like this:
basket.Items.Should().NotBeNull();
basket.Items.Should().BeEmpty();

// You can even chain assertions like this:
basket.Items.Should().NotBeNull().And.BeEmpty();

Using the normal way with the Assert class the assertions reads a bit as if spoken by Yoda: "assert that not null, the basket items are".

The assertions made with FluentAssertionsreads more naturally like "basket items should not be null" and "basket items should not be empty".

Regarding the more descriptive error messages, let's look at this example:

Assert.Equal(1, basket.Items.Count);
// Error message:
//   Assert.Equal() Failure
//   Expected: 1
//   Actual: 0

basket.Items.Should().HaveCount(1);
// Error message with FluentAssertions:
//   Expected collection to contain 1 item(s) but found 0.

The error message with FluentAssertionscontains more information about the assertion being made. In this case that we were expecting a collection to have a specific count. Without FluentAssertionswe just now that some comparison of integer values were not equal as expected.

You can also add a "because" text to the assertion, specifying why the assertion is needed.

basket.Items.Should().HaveCount(3, "because we added 3 different items");
// Error message:
//   Expected collection to contain 3 item(s) because we added 3 different items, but found 1.

Next up in Part 3

That's all for this time. In the next part I'll talk about mocking dependencies with NSubstitute and about how to make you code more easily testable.

• Part 1: Getting started
• Part 2: Naming, structure, and readability (FluentAssertions)
• Part 3: Mocking and writing testable code (NSubstitute)
• Part 4: Keeping it DRY and maintainable (AutoFixture)
• Part 5: Test Driven Development (TDD) with Live Unit Testing (LUT)

But most people I know really don't write a lot of unit tests or are not very good at it - which might be the reason they don't do it to begin with.

So I've been wanting to do a series of blog post about the whole process of writing unit tests in .NET (C#), how to get going and how to make it easier and faster to write good, maintainable unit tests.

• Part 1: Getting started
• Part 2: Naming, structure and readability (FluentAssertions)
• Part 3: Mocking and writing testable code (NSubstitute)
• Part 4: Keeping it DRY and maintainable (AutoFixture)
• Part 5: Test Driven Development (TDD) with Live Unit Testing (LUT)

Choosing a test framework

Before getting started you will need to choose a test framework. The two most popular ones are xUnitand NUnit.

I personally much prefer xUnit and feel it's more intuitive and requires less scaffolding. xUnit is also used internally by Microsoft and for the unit tests of the .NET code base itself.

Creating a test project

For this blog series I've created a simple solution named Blog.Commerce to use as an example and build upon during the series.

Normally you would create a test project per library/web project you want to add unit tests for.

In Visual Studio you can simply add a new xUnit test project like shown below or you can create it with dotnet new xunit. There's no set rule on what to name your test project or where to place it. Personally I usually name it {OtherProjectName}.Tests, which in this case would then be Blog.Commerce.Tests, and place it next to the existing project. Some people prefer to keep it separate in different folders. The default basic project file for a test project should look close to this:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <!-- You can change this to 'netcoreapp3.1' if working with .NET Standard or .NET Core -->
    <TargetFramework>net5.0</TargetFramework>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.8.3" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.3">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector" Version="3.0.2">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

</Project>

Tip: You can change the <TargetFramework> to <TargetFrameworks> and add multiple targets to run your tests for multiple frameworks, for example <TargetFrameworks>netcoreapp3.1.;net5.0</TargetFrameworks>

There you go. That wasn't so complicated - let's get started with writing some tests.

Setting the scene

For this example I've created a simple Basket with a list of BasketItems.

using System.Collections.Generic;
using System.Linq;

namespace Blog.Commerce
{
    public class Basket
    {
        public IList<BasketItem> Items { get; set; } = new List<BasketItem>();

        public void AddItem(string itemNo, int quantity)
        {
            if (TryGetItem(itemNo, out var basketItem))
            {
                basketItem.Quantity += quantity;
            }
            else
            {
                Items.Add(new BasketItem
                {
                    ItemNo = itemNo,
                    Quantity = quantity
                });
            }
        }

        private bool TryGetItem(string itemNo, out BasketItem basketItem)
        {
            basketItem = Items.SingleOrDefault(x => x.ItemNo == itemNo);
            return basketItem != null;
        }
    }
}

namespace Blog.Commerce
{
    public class BasketItem
    {
        public string ItemNo { get; set; }

        public int Quantity { get; set; }
    }
}

I have some simple expectations (or requirements) for this basket that I would like to verify with some simple unit tests:

1. A new basket has an non-null, empty list of items
2. Adding a new item will add a new item to the list of items
3. Adding an item that already exist will add to the existing quantity and not add a new item to the list

Creating a test

I want to create a test for my Basket.cs class, so I make a new class (following the same folder structure of the class I want to test) and naming it {Class}Tests.cs which in this cases becomes BasketTests.cs.

For testing my 1. expectation above I create a new method called Ctor_ItemsIsEmpty (with Ctormeaning Constructor) and add the [Fact] attribute above it (this marks it as a test that should be executed by the xUnit test runner).

In the method I then create a new basket and makes some assertions to verify the outcome I expect of the test.

using System.Collections.Generic;
using Xunit;

namespace Blog.Commerce.Tests
{
    public class BasketTests
    {
        [Fact]
        public void Ctor_ItemsIsEmpty()
        {
            var basket = new Basket();

            Assert.NotNull(basket.Items);
            Assert.Empty(basket.Items);
        }
    }
}

Above you see the finished code for our first test. We simply create a new basket and then assert that the items is neither null nor empty. When the test is run those assertions will throw an exception and make the test fail if they are not correct.

How to name, organizing and structe your tests will be expanded on in Part 2 of this series.

Below is how my solution looks at this point.

Running your tests

In Visual Studio you can run your tests by going to the Test menu and selecting Run All Tests which will then open the Test Explorer window and run your tests. Or you can run your tests with dotnet test in your solution directory (or by specifying a .sln or .csproj file directly) which should give you this output: If for example we didn't initialize the list of items to an empty list, then the test would fail and it would look something like this:

Running the same test with different input

Another common case is to reuse the same test with different input, saving you time and lines of code.

To do that we add some parameters to the test method and instead of the [Fact] attribute we use a [Theory] attribute.

The simplest way to supply input data is by using multiple [InlineData] attributes for each set of input parameters you want to run the test with. There are a lot of other ways to supply data to a test like the [ClassData] and [MemberData] attribute - you can even create your own custom implementation.

So let's add an example of a test for (part of) our 3. case mentioned earlier - adding an item that already exists should add to the quantity of the existing item.

[Theory]
[InlineData(1, 1, 2)]
[InlineData(1, 3, 4)]
public void AddItem_ExistingItem_ShouldIncreaseQuantityOfExistingItem(
    int originalQuantity, int addQuantity, int expectedQuantity)
{
    // Arrange
    var itemNo = "qwerty";

    var basket = new Basket
    {
        Items = new List<BasketItem>
        {
            new BasketItem
            {
                ItemNo = itemNo,
                Quantity = originalQuantity
            }
        }
    };

    // Act
    basket.AddItem(itemNo, addQuantity);

    // Assert
    Assert.Equal(expectedQuantity, basket.Items[0].Quantity);
}

Here we have 3 input parameters: originalQuantity, addQuantity and expectedQuantity. We have added the [InlineData] twice, each with a value for every method parameter. The test will now be executed twice and with the specified inputs respectively.

Adding a few mere tests to cover my expectations mentioned earlier and I end up with the following final test class for now.

using System.Collections.Generic;
using Xunit;

namespace Blog.Commerce.Tests
{
    public class BasketTests
    {
        [Fact]
        public void Ctor_ItemsIsEmpty()
        {
            var basket = new Basket();

            Assert.NotNull(basket.Items);
            Assert.Empty(basket.Items);
        }

        [Fact]
        public void AddItem_NewItem_AddsNewItem()
        {
            // Arrange
            var basket = new Basket();
            Assert.Equal(0, basket.Items.Count);

            // Act
            basket.AddItem("qwerty", 1);

            // Assert
            Assert.Equal(1, basket.Items.Count);
        }

        [Fact]
        public void AddItem_ExistingItem_DoesNotAddNewItem()
        {
            // Arrange
            var itemNo = "qwerty";
            var basket = new Basket
            {
                Items = new List<BasketItem>
                {
                    new BasketItem
                    {
                        ItemNo = itemNo,
                        Quantity = 1
                    }
                }
            };
            Assert.Equal(1, basket.Items.Count);

            // Act
            basket.AddItem(itemNo, 1);

            // Assert
            Assert.Equal(1, basket.Items.Count);
        }

        [Theory]
        [InlineData(1, 1, 2)]
        [InlineData(1, 3, 4)]
        public void AddItem_ExistingItem_ShouldIncreaseQuantityOfExistingItem(
            int originalQuantity, int addQuantity, int expectedQuantity)
        {
            // Arrange
            var itemNo = "qwerty";

            var basket = new Basket
            {
                Items = new List<BasketItem>
                {
                    new BasketItem
                    {
                        ItemNo = itemNo,
                        Quantity = originalQuantity
                    }
                }
            };

            // Act
            basket.AddItem(itemNo, addQuantity);

            // Assert
            Assert.Equal(expectedQuantity, basket.Items[0].Quantity);
        }
    }
}

Coming up in Part 2

In the next part I'll go a bit more in-depth with how I prefer to name and structure my tests, what to test and how to write more readable assertions with more descriptive error messages in case things blow up ;)

• Part 1: Getting started
• Part 2: Naming, structure and readability (FluentAssertions)
• Part 3: Mocking and writing testable code (NSubstitute)
• Part 4: Keeping it DRY and maintainable (AutoFixture)
• Part 5: Test Driven Development (TDD) with Live Unit Testing (LUT)

Some of the issues I've had along the way:

• Not working at all...

• "Try it out" not working because:

  • HTTP/HTTPS not being picked up
  • Document being cached by different host or scheme meaning the opposite would not work

  It wasn't that straight forward but through a lot of googling and reading GitHub issues I got a working solution put together.

In my specific case I was running ASP.NET Core 3.1 in a linux docker container in Azure, which some of the solution might be colored by.

If I remember correctly the most important parts are the X-Forwarded-Host and X-Forwarded-PathBase HTTP headers, so it's important that these are forwarded from your proxy to your Swagger service.

public class Startup
{
    // ...

    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        // ...

        app.UseXForwardedHeaders();
        app.UseSwaggerWithReverseProxySupport();
    }
}

static class StartupExtensions
{
    public static IApplicationBuilder UseXForwardedHeaders(this IApplicationBuilder app)
    {
        var options = new ForwardedHeadersOptions
        {
            ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
        };

        // Only loopback proxies are allowed by default.
        // Clear that restriction because forwarders are being enabled by explicit configuration.
        // https://stackoverflow.com/a/56469499/5358985
        options.KnownNetworks.Clear();
        options.KnownProxies.Clear();

        app.UseForwardedHeaders(options);

        app.Use((context, next) =>
        {
            if (context.Request.Headers.TryGetValue("X-Forwarded-PathBase", out var pathBases))
            {
                context.Request.PathBase = pathBases.First();
            }
            return next();
        });

        return app;
    }

    public static IApplicationBuilder UseSwaggerWithReverseProxySupport(this IApplicationBuilder app)
    {
        app.UseOpenApi(config =>
        {
            // Without this the document will be cached with wrong URLs 
            // and not work if later accessed from another host/path/scheme
            config.CreateDocumentCacheKey = request => request.Headers["X-Forwarded-Host"].FirstOrDefault() +
                                                        request.Headers["X-Forwarded-PathBase"].FirstOrDefault() +
                                                        request.IsHttps;

            // Change document host and base path from headers (if set)
            config.PostProcess = (document, request) =>
            {
                document.BasePath = request.Headers["X-Forwarded-PathBase"].FirstOrDefault();
                document.Host = request.Headers["X-Forwarded-Host"].FirstOrDefault();
            };
        });

        app.UseSwaggerUi3(settings =>
        {
            settings.TransformToExternalPath = (route, request) =>
            {
                var pathBase = request.Headers["X-Forwarded-PathBase"].FirstOrDefault();
                return !string.IsNullOrEmpty(pathBase)
                    ? $"{pathBase}{route}"
                    : route;
            };
        });

        return app;
    }
}

Normally, for synchronous calls, you can just add a .Throws<TException>() after the method you want to mock throwing an exception (or .ThrowsForAnyArgs<TException>() if you don't care about the input arguments). If you do this on a method returning a Task then that call would instantly throw, which is - most often - not what happens; it's the work being done by the Task throwing an exception, not the action of creating the Task.

As you can read in the Stack Overflow post the way to simulate a Task throwing an exception is to have the method return an already faulted Task, which can be done (sort of easily) with the static method Task.FromException<TReturn>(Exception).

interface IService
{
    Task<MyModel> GetAsync();
}

var service = Substitute.For<IService>();
service.GetAsync().Returns(Task.FromException<MyModel>(new ArgumentException("bla bla"));

But that's a bit too verbose for my taste, compared to this synchronous version:

var service = Substitute.For<IService>();
service.Get().Throws<ArgumentException>();

So I ended up creating a few extension methods enabling a more simple and readable syntax, so I could instead write it like this:

var service = Substitute.For<IService>();

// Any Exception
service.GetAsync().TaskThrowsException("optional message");

// Specific Exception v1
service.GetAsync().TaskThrows<MyModel, ArgumentException>("optional message");
// Specific Exception v2
service.GetAsync().TaskThrows(typeof(ArgumentException), "optional message");

// Specific Exception instance
service.GetAsync().TaskThrows(new ArgumentException());

That's a lot cleaner and more readable in my eyes :)

The TaskThrows<MyModel, ArgumentException>() might seem a little weird, but for the generic Task<T> we sadly can't have the generic arguments inferred as we can for the others :(

Here are all the extension methods for both Task and Task<T>:

public static class NSubstituteExtensions
{
    public static ConfiguredCall TaskThrowsException<T>(this Task<T> task, string message = null)
    {
        return TaskThrows<T, Exception>(task, message);
    }

    public static ConfiguredCall TaskThrows<T, TException>(this Task<T> task, string message = null)
        where TException : Exception
    {
        return TaskThrows(task, typeof(TException));
    }

    public static ConfiguredCall TaskThrows<T>(this Task<T> task, Type exceptionType, string message = null)
    {
        if (!typeof(Exception).IsAssignableFrom(exceptionType))
            throw new ArgumentException($"Type has to be a subclass of System.Exception", nameof(exceptionType));

        var exception = Activator.CreateInstance(exceptionType, message) as Exception;
        return TaskThrows(task, exception);
    }

    public static ConfiguredCall TaskThrows<T>(this Task<T> task, Exception exception)
    {
        return task.Returns(Task.FromException<T>(exception));
    }

    public static ConfiguredCall TaskThrows<TException>(this Task task, string message = null)
        where TException : Exception
    {
        return TaskThrows(task, typeof(TException));
    }

    public static ConfiguredCall TaskThrows(this Task task, Type exceptionType, string message = null)
    {
        if (!typeof(Exception).IsAssignableFrom(exceptionType))
            throw new ArgumentException($"Type has to be a subclass of System.Exception", nameof(exceptionType));

        var exception = Activator.CreateInstance(exceptionType, message) as Exception;
        return TaskThrows(task, exception);
    }

    public static ConfiguredCall TaskThrows(this Task task, Exception exception)
    {
        return task.Returns(Task.FromException(exception));
    }
}

However, you might experience weird behavior where it only works sometimes. This post is about explaining why and how to work around it.

The issue

Let us start with an example of changing the publish view (the view used after publishing a page) by using an interface IAllPropertiesPublishView.

/// <summary>
/// Interface for changing PublishView to AllPropertiesView.
/// </summary>
public interface IAllPropertiesPublishView { }

[UIDescriptorRegistration]
public class AllPropertiesPublishViewDescriptor : UIDescriptor<IAllPropertiesPublishView>
{
    public AllPropertiesPublishViewDescriptor()
    {
        PublishView = CmsViewNames.AllPropertiesView;
    }
}

We can add this interface to all the page types where we want to change the publish view to the All Properties view.

Now, if we want this to apply to all page types extending PageData instead of having to remember to add the interface, then we can change this to inherit UIDescriptor<PageData> instead. That should work, right? It will. Sometimes. It depends on the order of initialization/assembly scanning.

If you enable debug logging you should see a message in the log like Discarding ui descriptor registration for type {0} provided by {1} since a desciptor for the type is already registered.

EPiServer already has an internal UI Descriptor, PageUIDescriptor, which has some default configuration for PageData:

[UIDescriptorRegistration]
public class PageUIDescriptor : UIDescriptor<PageData>, IEditorDropBehavior
{
    public PageUIDescriptor()
      : base("epi-iconObjectPage")
    {
        IsPrimaryType = true;
        CommandIconClass = "epi-iconPage";
        AllowSelectItSelf = true;
        ContainerTypes = new [] { typeof (PageData) };
        EditorDropBehaviour = EditorDropBehavior.CreateLink;
    }

    public bool AllowSelectItSelf { get; private set; }

    public EditorDropBehavior EditorDropBehaviour { get; set; }
}

When the site starts it is uncertain if the default UI Descriptor or our custom one will be loaded first. Only the first one will be registered and initialized; others for the same specific type will be skipped and a debug message will be written to the logs.

A solution

One way around this issue is to create an initialization module and modify the already registered and initialized UI Descriptors.

[InitializableModule]
[ModuleDependency(typeof(DataInitialization))]
public class PageDataDefaultViewsInitialization : IInitializableModule
{
    public override void Initialize(InitializationEngine context)
    {
        // Get the UIDescriptorRegistry instance
        var registry = context.Locate.Advanced.GetInstance<UIDescriptorRegistry>();

        // Get the UIDescriptor for PageData
        var descriptor = registry.UIDescriptors
            .FirstOrDefault(x => x.ForType == typeof(PageData));

        // Abort if it didn't exist
        if (descriptor == null)
            return;

        // Modify the descriptor as you please
        descriptor.DefaultView = CmsViewNames.AllPropertiesView;
        descriptor.PublishView = CmsViewNames.AllPropertiesView;
    }

    public void Uninitialize(InitializationEngine context)
    {

    }
}

The code is rather simple. We are just getting the UIDescriptorRegistry instance and then getting the first UIDescriptor for PageData (if there is any). If it exists we then modify it however we please, as we would have done in our custom UIDescriptor.

Accessing the UIDescriptorRegistry.UIDescriptors property is what triggers the loading/initialization of all the UI Descriptors.

This approach can also be used if you need some more generic logic for customizing UI Descriptors, e.g. depending on type name or for 3rd party types that you don't control.

Conclusion

There can only be one UI Descriptor per specific type/interface. If there are multiple descriptors only one will be registered and initialized; the others will be skipped. It looks like it depends on a race condition in the type scanning which descriptor will be loaded first, so it can change between site restarts.

One solution is to instead modify the registered UI Descriptors by finding the ones you care about from the UIDescriptorRegistry.

Bonus info

The priority of UI Descriptors, i.e. which UI Descriptor is chosen for a specific type, is controlled by the (private) UIDescriptorRegistry.TypeComparer:

public override int Compare(UIDescriptor x, UIDescriptor y)
{
    if (x == y)
        return 0;
    Type forType1 = x.ForType;
    Type forType2 = y.ForType;
    return forType1.IsInterface && !forType2.IsInterface || forType1.IsAssignableFrom(forType2) ? 1 : -1;
}

In short, the more specific your UI Descriptor is, the higher priority.

Let's say we the following classes and interfaces:

public class CustomPage : BasePage, ISomeInterface
{
}

public interface ISomeInterface : IBase { }

UI Descriptors for all those would be prioritized in this order:

1. UIDescriptor<CustomPage>
2. UIDescriptor<BasePage>
3. UIDescriptor<ISomeInterface>
4. UIDescriptor<IBase>

So, concrete types are above interfaces. Interfaces inheriting another interface before the inherited interface etc.

Well, at some point I got tired of that and also wanted to expand my Azure-knowledge and wanted to try out Azure Search for something - so I ended up making https://sitecorereleasenotes.com, go check it out.

What is it?

It's a site that allows you to search for words or issue numbers mentioned in the Sitecore XP release notes located on https://dev.sitecore.net/Downloads/Sitecore_Experience_Platform.aspx. That means all release notes from 8.0 and onward.

It will return a list of the 5 most relevant releases for your search query, showing what it matched and with a link to both the release notes and the download page.

It doesn't allow complicated/specialized queries but it works very well for issue numbers and most simple queries.

How does it work?

It is rather simple, actually. It is just an Azure Function that crawls the downloads page daily and indexes all the release notes into Azure Search. The site itself is more or less just a simple interface to query the index and display the results nicely, running as an Azure Web App.

Most of the Azure services (Functions, Search, Web App/App Service) can use the free tier (or a very cheap one). It's also possible to use the free tier for the Web App, but I've gone with the B1 (Basic) for the SSL support, custom domain and the "Always On" options.

The crawling is scheduled to run daily, but it only takes ~15 seconds so it could be run more often - but Sitecore doesn't put out new releases THAT often and I can live with up to a one day delay.

How can I get involved?

The code is available on Github. You are more than welcome to create an issue if you have any feature requests, or even better submit a pull request!

The code is not that pretty as it just barely got past the prototype stage, to a stage where I was comfortable showing it off to the public.

Do you plan on improving it?

Maybe, it depends. I don't have any plans right now as it currently fills most of my own needs. I have plenty of ideas but there is only so much time in a day and it is competing with other more interesting stuff.

If it gains a lot of traction and interest I might get back into it at some point, but otherwise I hope it's useful for some people as it is now.

My requirements/reasons for wanting to run Solr in a Docker container:

• Not having to install Java (nobody wants that these days)
• Easy install of Solr "as a service" (I can't be the only one thinking it's a bit of a hassle to setup on windows, right?)
• Run Solr only when needed
• Run different versions of Solr (or quickly switch between versions)

During my research and experiments I got to thinking about SQL Server as well. It is not too long ago that it started being supported on Linux and I found out that Microsoft have made some SQL Server 2017 docker images - so why not try and get Sitecore running with that as well?

Overview

I will go through the steps for configuring the different containers and getting Sitecore 9 XP0 installed using the Sitecore Installation Framework (SIF) with both SQL Server 2017 and Solr 6.6.2 running in containers in Docker.

Prepare yourself, this is a long one...

• Why I chose Linux containers
• Prerequisites
• 1. Setting up the containers
     • 1.1 Building a custom image for the nginx HTTPS proxy
• 1. Running the containers and importing the created certificate
• 1. Modify SIF config files
• 1. Create and run install script with SIF-less
• 1. Continue with the post-installation steps in Sitecore installation guide
• Closing notes
• Troubleshooting
• Additional resources

I've made the files needed for the Docker setup available on GitHub.

Why I chose Linux containers

Docker (on Windows 10) can run either Linux containers or Windows containers - but not both at the same time. I could have gone with Windows containers, which would have enabled me to also run IIS and potentially the xConnect service in containers as well, but I decided against that for several reasons.

There are more Linux images available and they just have a bigger community compared to Windows images as they have only been reasonably supported by Docker somewhat recently. I also think it's easier to create custom images in Linux using apt-get or apk add (and I know almost nothing about Linux) compared to windows where you usually download a file from a specific URL, unzip etc. all using the command line. Windows is just better suited for a GUI and it's just all a bit cumbersome.

Also, the Sitecore Installation Framework (SIF) tinkers with your local machine with regards to IIS websites, application pools and so on, so I would have had to customize that as well if I wanted to use SIF for the installation.

Prerequisites

• Download and install Sitecore Installation Framework and Sitecore Fundamentals (they are listed on the Sitecore 9 download page)
• Download the Sitecore 9.0 XP Single (XP0) package or which ever package is relevant for you.
• Download and install Docker

You will need to configure Docker to have at least 3.25 GB memory available (required for SQL Server). Right click the tray icon and go to Settings. Go to the Advanced tab and adjust the slider for the available memory.

1. Setting up the containers

For my current setup we need 3 services running in Docker:

• Solr
• SQL Server
• nginx (proxying HTTPS for Solr)

Instead of the nginx proxy you could also create a custom image where you configure SSL for Solr, but I found this solution - using nginx - easier to setup.

To set all this up we need to create a few files. We start by composing our services for the docker-compose command with a docker-compose.yml file.

docker-compose.yml

version: '3'
services:

  sql2017:
    image: microsoft/mssql-server-linux:2017-latest
    ports:
      - "1433:1433"
    volumes:
      - ./Sqldata:/var/opt/mssql
    environment:
      ACCEPT_EULA: "Y"
      SA_PASSWORD: Qwerty123

  solr662:
    image: solr:6.6.2
    ports:
      - "8983" # Only exposes the port to other services
    volumes:
      - ./Solrdata:/solrhome
    environment:
      SOLR_HOME: /solrhome
      INIT_SOLR_HOME: "yes"

  proxy:
    build: 
      context: .
      args: 
        SERVER_NAME: solr
    image: nginx-solr-proxy
    ports:
      - "8983:443"
    volumes:
      - ./Certs:/certs
    environment:
      SERVER_NAME: solr
      PROXY_PASS: http://solr662:8983
      PROXY_REDIRECT: http://solr662/solr/ https://solr:8983/solr/
    depends_on:
      - solr662

What all this does is configure 3 services: sql2017, solr662 and proxy. Each services specifies an image (proxy actually builds its own, more on that in a bit), exposes some ports, mounts some folders so we can persist data and configures some relevant environment variables.

• sql2017
  • Use the official microsoft/mssql-server-linux image, tag 2017-latest
  • Bind port 1433 on host machine to port 1433
  • Mount the folder ./Sqldata to where the databases are stored inside the container
• solr662
  • Use the official solr image, tag 6.6.2
  • Expose port 8983 only to the other services
  • Mount the folder ./Solrdata to the Solr home folder inside the container
• proxy
  • Build a new image (uses the Dockerfile file as explained later) with name nginx-solr-proxy and use this image
  • Bind port 8983 on host machine to port 443
  • Mount the folder ./Certs. This folder will contain the SSL certificate used by the proxy. This will need to be installed into your root certificate store to be trusted (Chrome will start complain because it's not correctly singed by a CA, but SIF will work).

NOTE: This setup requires you to add 127.0.0.1 solr to your hosts file. If you want to use another hostname just change the value of the SERVER_NAME variable both under build: and environment: and change the hostname of the last part of the PROXY_REDIRECT environment variable to match, e.g. PROXY_REDIRECT: http://solr662/solr/ https://YOUR_CUSTOM_HOSTNAME:8983/solr/.

Later on, when the services are up and running, you will have to install the snakeoil.cer certificate located in the mounted ./Certs folder.

1.1 Building a custom image for the nginx HTTPS proxy

For the nginx proxy we need to customize the barebone nginx image to include a newly created SSL certificate and configure nginx to our needs. This can be done in many ways and my way probably isn't the best way (it can definitely be improved), but it works pretty in my opinion. In any case it's a good starting point.

To do this we need three files:

• Dockerfile - specifies how to build the image
• docker-entrypoint.sh - run each time the image is run
• .dockerignore - ignores files not needed by Dockerfile

Dockerfile

FROM nginx:mainline-alpine

ARG SERVER_NAME=solr

ENV SSL_CERTIFICATE=/etc/ssl/certs/snakeoil.cer
ENV SSL_CERTIFICATE_KEY=/etc/ssl/private/snakeoil.key
RUN apk add --no-cache openssl \
    && openssl ecparam -out ${SSL_CERTIFICATE_KEY} -name prime256v1 -genkey \
    && openssl req -new -key ${SSL_CERTIFICATE_KEY} -x509 -sha256 -nodes \
        -days 3650 -subj "/CN=${SERVER_NAME}" -out ${SSL_CERTIFICATE}

RUN rm /etc/nginx/conf.d/*.conf

COPY . /

ENTRYPOINT ["/docker-entrypoint.sh"]

.dockerignore

**/*
!docker-entrypoint.sh

What this does is pull the official nginx image with the tag mainline-alpine (the alpine version has a smaller footprint) and uses this as the baseline for our new image.

Then we are creating the certificate and key with the specified SERVER_NAME (as specified in docker-compose.yml or defaulting to solr) as the subject of the certificate.

Then we clear out the default nginx config files, copy everything in the current directory (that is not ignored by .dockerignore, i.e. only the docker-entrypoint.sh is copied).

At last we specify the script to run on startup, docker-entrypoint.sh.

docker-entrypoint.sh

#!/bin/sh

mkdir /certs -p
cp /etc/ssl/certs/snakeoil.cer /certs/
cp /etc/ssl/private/snakeoil.key /certs/

cat > /etc/nginx/conf.d/proxy.conf << EOT
map \$http_upgrade \$connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    listen 443 ssl default;

    server_name ${SERVER_NAME:-_};
    ssl_certificate /etc/ssl/certs/snakeoil.cer;
    ssl_certificate_key /etc/ssl/private/snakeoil.key;

    location / {
        client_body_buffer_size ${CLIENT_BODY_BUFFER_SIZE:-128k};
        client_max_body_size ${CLIENT_MAX_BODY_SIZE:-16m};
        proxy_set_header Host ${PROXY_HOST:-\$host};
        proxy_set_header X-Forwarded-Proto \$scheme;
        proxy_set_header X-Forwarded-Port \$server_port;
        proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
        proxy_pass ${PROXY_PASS:-http://upstream};
        proxy_redirect ${PROXY_REDIRECT:-default};
        proxy_http_version 1.1;
        proxy_set_header Upgrade \$http_upgrade;
        proxy_set_header Connection \$connection_upgrade;
        proxy_buffering ${PROXY_BUFFERING:-off};
        proxy_connect_timeout ${PROXY_CONNECT_TIMEOUT:-60s};
        proxy_read_timeout ${PROXY_READ_TIMEOUT:-180s};
        proxy_send_timeout ${PROXY_SEND_TIMEOUT:-60s};
    }
}

server {
    listen 80 default;

    server_name ${SERVER_NAME:-_};
    return 301 https://\$server_name\$request_uri;
}
EOT

echo "Starting nginx"
exec nginx -g 'daemon off;'

NOTE: Make sure this file is saved with unix-style line endings LF and not Windows-style CRLF

The entrypoint script just configures nginx and copies the certificate to the certs folder so you can easily access it and import it into your root certificate store.

2. Running the containers and importing the created certificate

To build our custom image and run the services we configured in docker-compose.yml simply run the following command in the same folder as your docker-compose.yml, Dockerfile etc.

docker-compose up -d

If you leave out the -d parameter it will attach to the containers and show log output.

It should result in something like this:

Building proxy
Step 1/8 : FROM nginx:mainline-alpine
 ---> 5c6da346e3d6
Step 2/8 : ARG SERVER_NAME=solr
 ---> Using cache
 ---> 83967fbb9479
Step 3/8 : ENV SSL_CERTIFICATE /etc/ssl/certs/snakeoil.cer
 ---> Using cache
 ---> 82b6dfcb6d13
Step 4/8 : ENV SSL_CERTIFICATE_KEY /etc/ssl/private/snakeoil.key
 ---> Using cache
 ---> 5e89eeaa6573
Step 5/8 : RUN apk add --no-cache openssl     && openssl ecparam -out ${SSL_CERTIFICATE_KEY} -name prime256v1 -genkey     && openssl req -new -key ${SSL_CERTIFICATE_KEY} -x509 -sha256 -nodes         -days 3650 -subj "/CN=${SERVER_NAME}" -out ${SSL_CERTIFICATE}
 ---> Using cache
 ---> b6e33bf3b7e4
Step 6/8 : RUN rm /etc/nginx/conf.d/*.conf
 ---> Using cache
 ---> 8def646dfe62
Step 7/8 : COPY . /
 ---> Using cache
 ---> 2ca0fb012838
Step 8/8 : ENTRYPOINT /docker-entrypoint.sh
 ---> Using cache
 ---> 66b5af8fe6fd
Successfully built 66b5af8fe6fd
Successfully tagged nginx-solr-proxy:latest
Creating docker_solr662_1 ...
Creating docker_sql2017_1 ...
Creating docker_sql2017_1
Creating docker_solr662_1 ... done
Creating docker_proxy_1 ...
Creating docker_sql2017_1 ... done

The first part of the container name will use the name of the current folder, docker in my case.

You should now see three new folders:

• Sqldata
• Solrdata
• Certs

Import the certificate snakeoil.cer from the Certs folder into your root certificates store. Without this the Sitecore Installation Framework (SIF) will not trust the server and be unable to create the Solr cores etc. as part of the installation.

3. Modify SIF config files

As we don't have a Solr service installed we need to modify the SIF config files. We have to remove the StopSolr and StartSolr tasks from the following config files:

• xconnect-solr.json
• sitecore-solr.json

They are logically located in the Tasks section. You can comment them out or delete them completely. Here is an example from the xconnect-solr.json file:

// ...

"Tasks": {
    // Tasks are separate units of work in a configuration.
    // Each task is an action that will be completed when Install-SitecoreConfiguration is called.
    // By default, tasks are applied in the order they are declared.
    // Tasks may reference Parameters, Variables, and config functions. 

    // "StopSolr": {
    //     // Stops the Solr service if it is running.
    //     "Type": "ManageService",
    //     "Params": {
    //         "Name": "[parameter('SolrService')]",
    //         "Status": "Stopped",
    //         "PostDelay": 1000
    //     }
    // },

    // ...
}

// ...

Because we only mounted the Solr home folder instead of the whole Solr installation directory we also have to change the Solr.Server variable to be [variable('Solr.FullRoot')] in the files mentioned above. It should look like below.

// ...
"Solr.Server": "[variable('Solr.FullRoot')]",
// ...

4. Create and run install script with SIF-less

Rob Ahnemann has created this great tool called SIF-less to make it easier to specify the parameters for a SIF install and also to check that everything is setup correctly beforehand.

Check out his initial blog post and the follow up. SIF-less can be downloaded from his Bitbucket repository or from this direct link.

Above are the settings I used. A few notes to be aware of:

- Use 'localhost,1433' as SQL Server (including the ´'´s) otherwise the PowerShell script will not parse it as two separate values and fail
- Point **Solr Folder** to the mounted `Solrdata` folder
- **Solr Service** can be ignored (might fail if empty, I haven't tried)

All tests except Solr Config Sets Exists and Solr Service Exists should be green.

Click the Generate Files button and run the generated .ps1 file (something like SIFless-EZ-1511282466.ps1) to begin the installation.

The installation should hopefully complete without any issues, if not let me know in the comments.

5. Continue with the post-installation steps in Sitecore installation guide

You should finish off your install by following the post-installation steps in the installation guide, starting in chapter 6.

It's mostly the default stuff of rebuilding Search Indexes and the Link Database, deploying Marketing Definitions etc., but step 6.1 requires you to add a recognized user to the xDB Shard Databases.

They have included a nifty script to help you with that. Just change the DatabasePrefix to match the prefix you used for your installation. You also have to turn on SQLCMD Mode from the Query menu to be able to successfully run the script.

:SETVAR DatabasePrefix sc90 
:SETVAR UserName collectionuser 
:SETVAR Password Test12345 
:SETVAR ShardMapManagerDatabaseNameSuffix _Xdb.Collection.ShardMapManager 
:SETVAR Shard0DatabaseNameSuffix _Xdb.Collection.Shard0 
:SETVAR Shard1DatabaseNameSuffix _Xdb.Collection.Shard1 
GO 

IF(SUSER_ID('$(UserName)') IS NULL)
BEGIN
    CREATE LOGIN [$(UserName)] WITH PASSWORD = '$(Password)';
END;
GO 

USE [$(DatabasePrefix)$(ShardMapManagerDatabaseNameSuffix)] 

IF NOT EXISTS (SELECT * FROM sys.database_principals WHERE name = N'$(UserName)')
BEGIN
    CREATE USER [$(UserName)] FOR LOGIN [$(UserName)]
        GRANT SELECT ON SCHEMA :: __ShardManagement TO [$(UserName)]
        GRANT EXECUTE ON SCHEMA :: __ShardManagement TO [$(UserName)]
END;
GO 

USE [$(DatabasePrefix)$(Shard0DatabaseNameSuffix)]  

IF NOT EXISTS (SELECT * FROM sys.database_principals WHERE name = N'$(UserName)')
BEGIN
    CREATE USER [$(UserName)] FOR LOGIN [$(UserName)]
        EXEC [xdb_collection].[GrantLeastPrivilege] @UserName = '$(UserName)'
END;
GO 

USE [$(DatabasePrefix)$(Shard1DatabaseNameSuffix)]  

IF NOT EXISTS (SELECT * FROM sys.database_principals WHERE name = N'$(UserName)')
BEGIN
    CREATE USER [$(UserName)] FOR LOGIN [$(UserName)]
        EXEC [xdb_collection].[GrantLeastPrivilege] @UserName = '$(UserName)'
END; 
GO

Closing notes

This ended up being quite a long blog post. It took me quite some time to put all the pieces together to get everything running, but a lot of that was because I know almost nothing about either Docker nor Linux.

In the end I think the required setup/files are actually pretty simple though, considering what is actually being done. All the files can be found on GitHub.

One thing I specifically would like to improve on in the future is the whole SSL certificate setup, which still requires some manual steps and is not fully trusted in the browser.

I hope you enjoyed it all and got it working! If you have any comments, questions or suggestions for improvements please let me know in the comments section.

Troubleshooting

Getting "exec user process caused "no such file or directory"" when starting nginx container

Make sure the docker-entrypoint.sh file is saved with unix-style line endings LF and not Windows-style CRLF.

https://forums.docker.com/t/standard-init-linux-go-175-exec-user-process-caused-no-such-file/20025/3

Additional resources

• https://hub.docker.com/r/microsoft/mssql-server-linux/

• https://docs.microsoft.com/en-us/sql/linux/quickstart-install-connect-docker

You can control the BSON representation of your property by adding the [BsonRepresentation] attribute or set it on your class map.

[BsonRepresentation(BsonType.String)]
public DateTimeOffset EventStart { get; set; }

As mentioned the DateTimeOffsetSerializer supports three different representation types:

• BsonType.Array (default)
• BsonType.Document
• BsonType.String

Array representation (default)

By default DateTimeOffset is serialized as an array consisting of the ticks and the offset in minutes.

// Serialization code for BsonType.Array
bsonWriter.WriteStartArray();
bsonWriter.WriteInt64(value.Ticks);
bsonWriter.WriteInt32((int)value.Offset.TotalMinutes);
bsonWriter.WriteEndArray();

// Example JSON output
// [NumberLong("636415154304356493"), 120]

This is probably the reason you are here right now. This is different than DateTime which is stored as a BSON date (ISODate("2017-09-20T12:46:53.552Z")) and it makes it harder to query manually and harder to read.

I'm sure there's a good reason why this was made the default, but I'm not exactly sure why.

String representation

The string representation is more recognizable to most people and probably what you are looking for.

// Serialization code for BsonType.String
bsonWriter.WriteString(JsonConvert.ToString(value));

// Example JSON output
// "2017-09-20T14:47:37.8324739+02:00"

Document representation

The last of the three options is the document representation, which is more or less a combination of the array and string representation combined in an object (document).

// Serialization code for BsonType.Document
bsonWriter.WriteStartDocument();
bsonWriter.WriteDateTime("DateTime", BsonUtils.ToMillisecondsSinceEpoch(value.UtcDateTime));
bsonWriter.WriteInt64("Ticks", value.Ticks);
bsonWriter.WriteInt32("Offset", (int)value.Offset.TotalMinutes);
bsonWriter.WriteEndDocument();

// Example JSON output
// {
//     "DateTime": ISODate("2017-09-20T12:56:06.642Z"),
//     "Ticks": NumberLong("636415161666429142"),
//     "Offset": 120
// }

In my situation I wanted to hit a URL 10 minutes after the Octopus deployment was completed and I didn't want the deployment to "hang" for 10 minutes seemingly still deploying.

I tried several different methods, like mentioned in the beginning, but the tasks were never run to completion.

I'm pretty sure I know why this happens, although I haven't actually confirmed it. I'm assuming Octopus fires up a new ScriptCS process with our C# script as the input. When our script reaches the end, i.e. not awaiting any tasks etc., then the process exits and any unfinished tasks goes with it.

Alternative solution using Azure

I ended up with sort of an alternative solution using Azure Functions and Queue Storage which is free (or close to) for most solutions that don't require a lot of throughput.

It's pretty simple and in short it consists of an HTTP endpoint which adds (delayed) a message to the queue, and then a function with a queue trigger which executes the message. In Octopus I then just call the HTTP endpoint to trigger a delayed action and that's that.

I'll leave it at that and leave the details of the Azure stuff to a future blog post.

Some time ago I wanted to query some of this data and make some charts of lap times, gap to leader etc. from the latest F1 races, but I couldn't find any .NET library for it. I could have just made some simply calls to the specific endpoints I needed, but I like to make libraries, soo...

I still haven't gotten around to the chart part, but at least I've finished the library.

The library can be installed from NuGet - search for ErgastApiClient or install it through the Package Manger Console:

PM> Install-Package ErgastApiClient

If you are not a programmer but want to query the data anyway you can use the manual interface.

Using the library

The library is pretty simple to use. Create (and reuse) an ErgastClient and then create one of the many request types:

• RaceResultsRequest
• LapTimesRequest
• PitStopsRequest
• CircuitInfoRequest
• DriverInfoRequest
• DriverStandingsRequest
• ...

You then use the client to execute the request and it will return a matching response type. The client takes care of caching for you, so you don't query the Ergast web service unnecessarily.

Below is a complete example of doing a RaceResultsRequest.

// Relevant imports
using ErgastApi.Client;
using ErgastApi.Ids;
using ErgastApi.Requests;

// The client should be stored and reused during the lifetime of your application
var client = new ErgastClient();

// All request properties are optional (except 'Season' if 'Round' is set)
var request = new RaceResultsRequest
{
    Season = "2017",     // or Seasons.Current for current season
    Round = "11",        // or Rounds.Last or Rounds.Next for last or next round
    DriverId = "vettel", // or Drivers.SebastianVettel

    Limit = 30      // Limit the number of results returned
    Offset = 0      // Result offset (used for paging)
};

// RaceResultsRequest returns a RaceResultsResponse
// Other requests returns other response types
RaceResultsResponse response = await client.GetResponseAsync(request);

// We know it's only a single race, round 11
var race = response.Races.First();

race.Round;                // 11
race.RaceName;             // Hungarian Grand Prix
race.Circuit.CircuitName;  // Hungaroring

// There is only one driver in the results - Vettel
// Otherwise the results would be ordered by finishing position
var vettel = race.Results[0];

vettel.Driver.Code;           // VET
vettel.FastestLap.LapNumber;  // 69
vettel.Position;              // 1 (finishing position)

The RaceResultsResponse contains a list of races matching the query. In this example there is only the one race (Round 11, Hungary, 2017). Each race contains info about the race, the circuit, time of the event etc. and of course the race results.

Each result (in this case there is only Vettel) contains info about the driver, finishing position, starting position (Grid), fastest lap etc.

The above example is the equivalent of this manual query. You can also see the actual JSON data by adding .json to the URL of the manual request.

Source code

The source code is available on GitHub which also has some more examples of how you can use the library/Ergast API.

If you encounter any bugs or have a feature request, please create an issue on GitHub. Pull Requests are also very welcome.

ERROR: Script execution failed. [InvalidProgramException] Common Language Runtime detected an invalid program.
Exit code: 1
The remote script failed with exit code 1

This is most likely because .NET 4.5.2 is not installed on the machine. If you install that the error should go away.

There's a bit more info available on the following links:

• https://github.com/dotnet/corefx/issues/320

• https://github.com/dotnet/corefx/pull/687

<sitecore>
  <contentSearch>
    <indexConfigurations>
      <!-- DEFAULT SEARCH SECURITY OPTION
            This setting is the default search security option that will be used if
            the search security option is not specified during the creation of search context.
            The accepted values are DisableSecurityCheck and EnableSecurityCheck.
      -->
      <defaultSearchSecurityOption>DisableSecurityCheck</defaultSearchSecurityOption>
    </indexConfigurations>
  </contentSearch>
</sitecore>

The check is performed when enumerating the results (after the search has already been done) by fetching each item from Sitecore and then checking the item security and skipping those items that are not accessible to the current user.

That means if there are 10 results in total and you want to get the first 5, but none of those 5 are readable to the current user, then you get an empty list - even though there are actually more results. The TotalSearchResults will also return 10 even though those first 5 are not readable. Not optimal.

My Solution

I've come up with a better solution. It's somewhat limited and specifically tailored to the needs we had at the time as it is not really practical to store all access rights for all users and roles for all items in the index. It's all a compromise between complexity, search/indexing speed and the size of the index itself.

Supported features

• Direct user security
• Direct role security
• Inherited role security

Unsupported features

• Inherited user security

Overview

The solution consists of a few different elements.

1. A computed field that stores the access rights we need
2. An interface that has a property for the computed field
3. [optional] Extension methods to easily filter by security when querying an index
4. [optional] An attribute to automatically filter by security when using the attributed SearchResultItem in queries

Step 1: Computed field

Most of the logic happens in this computed field that we need to create. To be able to filter the results at query time we need to store some information in the index about which users and roles that have access to each item.

Overview of what we are doing in the computed field:

1. Check if the Everyone role has access and if so just store "everyone"
2. Get all roles in the sitecore domain that matches RoleRegex
3. Create a list of all those matching roles that are allowed to read the item
4. Get a list of all user accounts that have access rights specified directly on the item and are allowed to read it
5. Return a combined list of those roles and users that have access to the item

You might need to modify this a bit to match your needs, e.g. if your roles are not in the sitecore domain.

*Note: The more roles that match the regex, the longer time the indexing will take as it has to check security for each role and this is done for each indexed item.*

using System.Collections.Generic;
using System.Linq;
using System.Text.RegularExpressions;
using Sitecore.Configuration;
using Sitecore.ContentSearch;
using Sitecore.ContentSearch.ComputedFields;
using Sitecore.Data.Items;
using Sitecore.Security.Accounts;
using Sitecore.Security.Domains;
using Sitecore.SecurityModel;

namespace Sandbox.Search.ComputedFields
{
    public class ReadableTo : IComputedIndexField
    {
        // TODO: Modify to match your needs!
        // The more matching roles the longer indexing will take.
        private const string RoleDomain = "sitecore";
        private static readonly Regex RoleRegex = new Regex(".*");

        public string FieldName { get; set; }

        public string ReturnType { get; set; }

        public object ComputeFieldValue(IIndexable indexable)
        {
            Item item = indexable as SitecoreIndexableItem;

            if (item == null) return null;

            using (new SecurityStateSwitcher(SecurityState.Enabled))
            {
                if (item.Security.CanRead(Account.FromName("Everyone", AccountType.Role)))
                    return "everyone";
            }

            // Get roles in 'sitecore' domain matching the regex and check security (including inheritance)
            // More roles equals longer indexing time as security is checked for each role
            var roles = Domain.GetDomain(RoleDomain)
                .GetRoles()
                .Where(x => RoleRegex.IsMatch(x.Name));

            var allowedRoles = new List<string>();
            using (new SecurityStateSwitcher(SecurityState.Enabled))
            {
                foreach (var role in roles)
                {
                    if (item.Security.CanRead(role))
                        allowedRoles.Add(role.Name);
                }
            }

            // Get accounts (users/roles) with directly specified security
            var allowedAccounts =
                item.Security.GetAccessRules().Helper
                    .GetAccounts()
                    .Where(x => item.Security.CanRead(x))
                    .Select(x => x.Name);

            return allowedRoles.Concat(allowedAccounts);
        }
    }
}

Index configuration

We then need to add this new computed field to the index configuration for each index we want to use this with, so that we can use it when querying the index.

<!-- Add under /fieldMap/fieldNames in your index configuration -->
<field fieldName="ReadableTo" storageType="NO" indexType="TOKENIZED" vectorType="NO" boost="1f" type="System.String"     settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
  <analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider" />
</field>

<!-- Add under computed fields (/fields[@hint='raw:AddComputedIndexField']) in your index configuration -->
<field fieldName="ReadableTo" type="Sandbox.Search.ComputedFields.ReadableTo, Sandbox.Search" />

Step 2: Interface

Next up we need an interface with a property matching our new computed field. This is not strictly necessary, but it enables us to create an attribute and some extension methods to more easily use the new feature. If you don't need those extra features then you can just add a property named ReadableTo (or whatever you named it in the index configuration) to your SearchResultItem.

using System.Collections.Generic;

namespace Sandbox.Search
{
    public interface IIndexSecuritySupport
    {
        IEnumerable<string> ReadableTo { get; }
    }
}

Step 3: Extension methods (optional)

To make life easier for ourselves we can add two extension methods - one for the current user and one for a specific user.

IQueryable<TItem> FilterReadable<TItem>(this IQueryable<TItem> queryable) where TItem : IIndexSecuritySupport
IQueryable<TItem> FilterReadable<TItem>(this IQueryable<TItem> queryable, User user) where TItem : IIndexSecuritySupport

These will filter our IQueryable<TItem> to only items readable by either the current or the specified user respectively.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
using System.Text.RegularExpressions;
using Sitecore.Configuration;
using Sitecore.ContentSearch.Linq;
using Sitecore.ContentSearch.Linq.Utilities;
using Sitecore.Diagnostics;
using Sitecore.Security.Accounts;

namespace Sandbox.Search
{
    public static class QueryableSecurityExtensions
    {
        // This should match the regex in you computed field
        private static readonly Regex RoleRegex = new Regex($".*");

        private static readonly Type IndexSecurityType = typeof(IIndexSecuritySupport);

        private static readonly MethodInfo EnumerableContainsMethod;
        private static readonly PropertyInfo ReadableToProperty;

        static QueryableSecurityExtensions()
        {
            // Enumerable.Contains<string>(IEnumerable<string>, string)
            EnumerableContainsMethod =
                typeof(Enumerable).GetMethods()
                    .Single(m => m.Name == "Contains" && m.GetParameters().Length == 2)
                    .MakeGenericMethod(typeof(string));

            // IIndexSecuritySupport.ReadableTo
            ReadableToProperty = typeof(IIndexSecuritySupport).GetProperty(nameof(IIndexSecuritySupport.ReadableTo));
        }

        public static IQueryable<TItem> FilterReadable<TItem>(this IQueryable<TItem> queryable) where TItem : IIndexSecuritySupport
        {
            return FilterReadable(queryable, User.Current);
        }

        public static IQueryable<TItem> FilterReadable<TItem>(this IQueryable<TItem> queryable, User user) where TItem : IIndexSecuritySupport
        {
            Assert.ArgumentNotNull(queryable, nameof(queryable));
            Assert.ArgumentNotNull(user, nameof(user));
            Assert.IsTrue(IndexSecurityType.IsAssignableFrom(typeof(TItem)), $"Queryable type must implement the '{IndexSecurityType.Name}' interface.");

            var roles =
                user.Roles.Where(x => RoleRegex.IsMatch(x.Name))
                    .Select(x => x.Name)
                    .ToList();

            roles.Add(user.Name);

            var parameterExpression = Expression.Parameter(typeof(TItem), "param");
            var memberExpression = Expression.MakeMemberAccess(parameterExpression, ReadableToProperty);

            var everyone = GetContainsConstantPredicate<TItem>("everyone", parameterExpression, memberExpression);
            var containsAny = GetContainsAnyPredicate<TItem>(roles, parameterExpression, memberExpression);

            return queryable.Filter(everyone.Or(containsAny));
        }

        private static Expression<Func<TItem, bool>> GetContainsAnyPredicate<TItem>(IEnumerable<string> values, ParameterExpression parameterExpression, MemberExpression memberExpression)
        {
            var containsAnyPredicate = PredicateBuilder.False<TItem>();
            foreach (var value in values)
            {
                var containsExpression =
                    Expression.Lambda<Func<TItem, bool>>(
                        Expression.Call(EnumerableContainsMethod, memberExpression, Expression.Constant(value)),
                        parameterExpression);
                containsAnyPredicate = containsAnyPredicate.Or(containsExpression);
            }
            return containsAnyPredicate;
        }

        private static Expression<Func<TItem, bool>> GetContainsConstantPredicate<TItem>(string value, ParameterExpression parameter, MemberExpression member)
        {
            return
                Expression.Lambda<Func<TItem, bool>>(
                    Expression.Call(EnumerableContainsMethod, member, Expression.Constant(value)), parameter);
        }
    }
}

Without the extension methods we would have to do this everytime:

var user = Sitecore.Context.User;

var names =
    user.Roles.Where(x => RoleRegex.IsMatch(x.Name))
        .Select(x => x.Name)
        .ToList();

names.Add(user.Name);

var containsAnyFilter = PredicateBuilder.False<ArticleSearchResultItem>();
foreach (var name in names)
{
    containsAnyFilter = containsAnyFilter.Or(x => x.ReadableTo.Contains(name));
}

query.Where(x => x.Something == "abcdef")
     .Filter(x => x.ReadableTo.Contains("everyone") || containsAnyFilter)

Now we can just do this:

// Returns only indexed items readable to the current user
query.Where(x => x.Something == "abcdef").FilterReadable();

// Returns only indexed items readable to 'extranet\SomeUser'
var user = User.FromName("extranet\\SomeUser", false);
query.Where(x => x.Something == "abcdef").FilterReadable(user);

Step 4: Predefined query attribute for SearchResultItem (optional)

Sitecore allows us to use the [PredefinedQuery] attribute on classes to apply predefined queries when querying an index using that type. This attribute allows us to add constant filters. An example could be to always filter by a specific template ID:

[PredefinedQuery("TemplateID", ComparisonType.Equal, "{7FAFEDF6-9438-4CAD-9E04-3FCD89206D2F}", typeof(ID)]
public class ArticleSearchResultItem : SearchResultItem
{
    // ...
}

// Now you don't have to add this every time you want to search for articles:
// .Where(x => x.TemplateID == new ID("{7FAFEDF6-9438-4CAD-9E04-3FCD89206D2F}"))

It is also possible to make our own with our own logic by creating a class inheriting the IPredefinedQueryAttribute interface. This allows us to create an attribute that makes sure that only items readable to the current user is returned when querying the index using types marked with that attribute.

using System;
using System.Linq;
using Sitecore.ContentSearch;
using Sitecore.ContentSearch.Diagnostics;
using Sitecore.ContentSearch.Linq.Parsing;

namespace Sandbox.Search
{
    /// <summary>
    /// Automatically applies filtering on security to queries using the attributed class.
    /// </summary>
    [AttributeUsage(AttributeTargets.Class, Inherited = false)]
    public class ApplySecurityAttribute : Attribute, IPredefinedQueryAttribute
    {
        private static readonly Type IndexSecurityType = typeof (IIndexSecuritySupport);

        public IQueryable<TItem> ApplyFilter<TItem>(IQueryable<TItem> queryable, IIndexValueFormatter valueFormatter)
        {
            if (!IndexSecurityType.IsAssignableFrom(typeof (TItem)))
            {
                SearchLog.Log.Warn($"The type '{typeof (TItem).FullName}' does not implement the '{IndexSecurityType.Name}' interface so security filtering is not being applied.");
                return queryable;
            }

            // The casts looks hacky but it works
            return queryable
                .Cast<IIndexSecuritySupport>()
                .FilterReadable() // Our extension method from step 3
                .Cast<TItem>();
        }
    }
}

Then we just apply this to our custom SearchResultItem class:

// Your search result item also needs to inherit the IIndexSecuritySupport interface
[ApplySecurity]
public class ArticleSearchResultItem : SearchResultItem, IIndexSecuritySupport
{
}

Now we don't need to remember to add the FilterReadable() call to queries using the ArticleSearchResultItem type.

// This is now the same as 'query.Where(x => x.Something == "abcdef").FilterReadable()'
query.Where(x => x.Something == "abcdef");

Conclusion

So that's it. It looks a lot more complex than it really is.

There are two reasons why I'm not handling inherited user security. Well, first off - we didn't need it. But besides that, as far as I know it's not possible to check for inherited access rights without checking if each individual user has any inherited rights - and that would have to be done for every user on your site.

Let me know if you have any issues with this or comments in general.

ERROR Sitecore.Social: Application in args should not be null
Exception: Sitecore.Social.NetworkProviders.Exceptions.AuthException
Message: Exception of type 'Sitecore.Social.NetworkProviders.Exceptions.AuthException' was thrown.
Source: Sitecore.Social.NetworkProviders
   at Sitecore.Social.NetworkProviders.Exceptions.Analyzers.ArgsExceptionAnalyzer.Analyze(AuthArgs args)
   at Sitecore.Social.Client.Connector.SocialLogin.ProcessRequest(HttpContext httpContext)

I ran into this exception the other day and was a bit confused at it had been working fine.

It turns out a change had been made to our setup to always include the language as the first part in the URL - and if it was missing it would redirect to the correct URL including the language part.

When you click on one of the Social Connector buttons to log in, the module makes a POST request to /Social/TwitterConnector/Login (or the equivalent for the other connectors). These use standard MVC to generate a form which points to the Login method on the corresponding controller.

@using (Html.BeginForm("Login", null, new { area = "Social" }, FormMethod.Post, new { style = "display:inline" }))
{
  @Html.HiddenFor(model => model.Parameters)

  <input type="image" title='@Model.ToolTip' style="margin:0 1px" src="@Url.Content(Model.Icon)" />
}

The redirect logic that was setup was also redirecting POST requests, but redirected POST requests turn into GET and also no longer include the form parameters.

Conclusion

So what can we learn from all this?

Do not redirect POST requests. Please.

If you want to return View() in your controller then you either need to set layout to null/empty string on the view itself or add it to _ViewStart.cshtml - otherwise you will end up with an infinite loop.

@{
    Layout = "";
}

An alternative is to make sure you don't have a _ViewStart.cshtml file with your main layout specified.

The loop happens because of placeholders in the main layout file. If one of the renderings in a placeholder returns a view with View() - instead of a partial view with PartialView() - then it will try to load the layout file, which will then load the renderings in the placeholders, which will then load the layout file and so on.

As mentioned to avoid this issue you can just return PartialView() instead as that does not try to load the layout again.

In those situation I usually end up searching StackOverflow for the solution and most likely have to test a few of them to find the best and most modern version.

To help my self I've decided to save some of them for ease of access and future use, so here goes:

• Pad left
• Debounce

Pad left

This will add a character to the left of the input value until the total length of the value is the specified width.

This is useful for stuff like right-aligning text and padding numbers with leading zeros.

var padLeft = function (value, width, char) {
    char = (char !== undefined) ? char : " ";
    width = (typeof width === "number") ? width : parseInt(width);
    value = String(value);
    return Array(1 + width - value.length).join(char) + value;
}

This works with both strings and numbers for input. Please note that negative numbers is not handled correctly.

Examples

// Right-aligning text
padLeft("Blue", 10, " ")
padLeft("Orange", 10, " ")

// Result:
//      Blue
//    Orange

// Padding numbers with leading zeros
padLeft(5, 3, 0)
padLeft(42, 3, 0)
padLeft(137, 3, 0)

// Result:
// 005
// 042
// 137

Debounce

Debouncing means waiting a set amount of time before executing a specified function.

This is useful in many different situations, for example an autosuggest feature where you only want to load suggestion after the user is done typing instead of continuously after each keypress.

// 'wait' is time in milliseconds
var debounce = function (func, wait) {
    var timeout;
    return function () {
        var context = this, args = arguments;
        clearTimeout(timeout);
        timeout = setTimeout(function () {
            func.apply(context, args);
        }, wait || 0);
    };
}

Examples

var loadSuggestions = function() {
    // Load suggestions from web service
}

// debounce(..) returns a new function.
// You have to save and reuse the returned function for it to work as expected.
var loadSuggestionsDebounce = debounce(loadSuggestions, 250);

var textfield = document.getElementById("textfield");
textfield.addEventListener("keypress", loadSuggestionsDebounce);

This post is basically just a cheat sheet of the email options that can be configured and where you can configure them.

The official documentation explains the basics like mail server and credentials etc. but it's missing a few bits.

The 'Send Email Message' save action parameters

On the item /sitecore/system/Modules/Web Forms for Marketers/Settings/Actions/Save Actions/Send Email Message in the Parameters field you can specify values (as XML) for the following settings:

<Host>smtp.domain.com</Host>
<Port>587</Port>
<Login>user@domain.com</Login>
<Password>b</Password>
<From>user@domain.com</From>
<EnableSsl>true</EnableSsl>
<IsBodyHtml>true</IsBodyHtml>
<IsIncludeAttachments>false</IsIncludeAttachments>

<!-- Not mentioned in official docs but should work as well -->
<Subject>Default subject</Subject>
<Mail>Default email body</Mail>
<CC>user@domain.com</CC>
<BCC>user@domain.com</BCC>
<LocalFrom></LocalFrom>
<RecipientGateway></RecipientGateway>
<FromPhone></FromPhone>

Settings from config files

It's only a subset of these settings that can be specified in config files. Values specified on the save action item itself will override any values from config files. Settings like EnableSsl, IsBodyHtml and IsIncludeAttachments can sadly only be specified on the save action although it should be somewhat trivial to change this, potentially by hooking into the processMessage pipeline.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:set="http://www.sitecore.net/xmlconfig/set/">
  <sitecore>
    <settings>
      <setting name="MailServer" set:value="smtp.domain.com" />
      <setting name="MailServerPort  " set:value="587" />
      <setting name="MailServerUserName" set:value="user@domain.com" />
      <setting name="MailServerPassword" set:value="b" />

      <!-- Corresponds to <From> - default sender address -->
      <setting name="Analytics.EMailFromAddress" set:value="user@domain.com" />
    </settings>
  </sitecore>
</configuration>

Note when upgrading from Sitecore 6.5

Most of the forms created with the earlier version of WFFM had <Host> and <From> parameters with the default values specified on the form itself in the Save Actions field. These were overriding anything specified in either config files or on the save action item. To fix this we had to manually edit the XML of the Save Actions field on all the forms.

The old forms also had a reference to the old Save to database save action (that no longer exists after 8.0) which we also had to remove.

I created this .aspx page/script to do this for us - use at your own risk!

InvalidOperationException: No route in the route table matches the supplied values.

The issue

This can happen when you have deleted/disabled the Sitecore.Speak.Mvc.config, which Sitecore recommends you do on Content Deliver servers, and use the default MVC helper methods @Html.Action and @Html.RenderAction in your renderings.

For some reason those methods requires a route containing the pattern {controller}/{action} to work. These are part of the default route in a standard MVC project, but usually not in a Sitecore project.

But then why does it work until we delete/disable Sitecore.Speak.Mvc.config?

Because that config files adds a processor to the initialize pipeline.

<initialize>
  <processor type="Sitecore.Mvc.Pipelines.Initialize.InitializeCommandRoute, Sitecore.Speak.Client" patch:before="processor[@type='Sitecore.Mvc.Pipelines.Loader.InitializeRoutes, Sitecore.Mvc']"/>
</initialize>

This processor registers the following routes:

protected virtual void RegisterRoutes(RouteCollection routes, PipelineArgs args)
{
    string commandRoutePrefix = SpeakSettings.Mvc.CommandRoutePrefix;
    routes.MapRoute("Sitecore.Speak.Commands", commandRoutePrefix + "{controller}/{action}");
    routes.MapRoute("Sitecore.Speak", "sitecore/shell/api/sitecore/{controller}/{action}");
}

So when you delete/disable this config file, those routes do no longer get registered.

How to fix it

It's pretty simple to fix. You just have to add a route containing the {controller}/{action} pattern in your own processor in the initialize pipeline.

// Something like this
RouteTable.Routes.MapRoute(
    "Bogus-Required-Route",
    "bogus-required-route/{controller}/{action}",
);

I'm not sure if the route would overwrite Sitecore's route handling, so to be sure I would use a distinct path for the route that you are sure to never match a page URL for your site.

If you want to prevent the action method from being invoked as a result of a user request you can add the [ChildActionOnly] attribute to it.

The Config Enable-Disable spreadsheet

If for some reason, like me, you can never find the download link for the Config Enable-Disable spreadsheet, then you can find them on this page.

Apparently the name of the standard values item, __Standard Values, is hardcoded inside Sitecore in several places - which makes sense as the item isn't otherwise discernible.

When item-level language fallback is enabled and you try to retrieve an item, it checks if item fallback is enabled on the item itself (checkbox field, part of the standard fields).

private bool IsItemFallbackEnabled(Item item)
{
    if (item == null)
        return false;

    if (StandardValuesManager.IsStandardValuesHolder(item))
        return item.Fields[FieldIDs.EnableItemFallback].GetValue(false) == "1";

    using (new LanguageFallbackItemSwitcher(new bool?(false))) 
    {
        return item.Fields[FieldIDs.EnableItemFallback].GetValue(true, true, false) == "1";
    }
}

The call to StandardValuesManager.IsStandardValuesHolder(item) ends up doing this check:

return item.Name.Equals("__Standard Values", StringComparison.OrdinalIgnoreCase);

So if the standard values item is not named exactly that, then it tries to get the field value again... and again... and again. You get the point.

This is probably not an issue you would run into very often, but it's always nice to be prepared. We ran into this issue with a client where we had installed an older version of the 301 Redirect Module (version 1.3 if I remember correctly), which had standard values items with a hyphen instead of a space in the name - __Standard-Values instead of the correct __Standard Values.

You can no longer download version 1.3 and later versions do not have this problem. The module has also recently been updated with a lot of new features like Content Editor notifications and automatic generation of redirects when items are moved.

The inifinite loop issue has been reported to Sitecore Support and they have filed it as a bug with reference number 118166.