LINQ gems: Troubles with Out Parameters

How do you deal with out parameters inside of LINQ queries?

Almost every time I come across the need to invoke a method inside of a LINQ query, which uses an out argument, I need to stop for a moment and think about what the correct solution is. And unfortunately, there does not seem to be a solution, which is the obviously right one.

One of the typical examples of consuming an out variable is when accessing the elements of a Dictionary using the TryGetValue method. In the sample code I'll be using below, we have a dictionary of common English words together with their usage counts (perhaps extracted from a piece of text somewhere). We then declare an array of test words, for which we want to obtain their usage count (and eventually sort by it).

Using a local variable

Using a local variable is probably the first thing that a developer will try:

Dictionary<string, int> wordUsage = new Dictionary<string, int>
{
    { "hello", 100 },
    { "world", 20  },
    { "use", 75 },
    { "case", 10 }
};

var testWords = new[] { "hello", "world", "use", "case" };

int usageVar = 0;
var query = 
  from word in testWords
  let wordFound = wordUsage.TryGetValue(word, out usageVar)
  select (word, usageVar);

foreach (var (word, usage) in query)
{
    Console.WriteLine($"Word: '{word}' is repeated {usage} time(s).");
}

So far, everything works as expected:

Word: 'hello' is repeated 100 time(s).
Word: 'world' is repeated 20 time(s).
Word: 'use' is repeated 75 time(s).
Word: 'case' is repeated 10 time(s).

If we now however add a request to sort the output by the usage count:

var query = 
  from word in testWords
  let wordFound = wordUsage.TryGetValue(word, out usageVar)
  orderby usageVar
  select (word, usageVar);

Things start to break apart:

Word: 'hello' is repeated 10 time(s).
Word: 'world' is repeated 10 time(s).
Word: 'use' is repeated 10 time(s).
Word: 'case' is repeated 10 time(s).

Why? Because OrderBy requires the whole sequence to be processed, before it can perform a sort on the elements of the sequence. What it means is that the let projection will execute on all elements in our sample set and the very last processed element in the set (the word 'hello') will assign a value of '10' to the local variable usageVar. This value is then passed four times into the orderby statement. Which is obviously wrong.

So, using a local variable is clearly not the way to go. What are the alternatives then?

Local function

Local functions are a concept introduced in C# 7.0. They let you declare a method inside the body of another method. What we can therefore do is create a simple helper method, which establishes a local scope for the out variable needed by the TryParse method, thus making the usage of the local variable safe.

public static void OutTestUsingLocalFunction()
{
    Dictionary<string, int> wordUsage = new Dictionary<string, int>
    {
        { "hello", 100 },
        { "world", 20  },
        { "use", 75 },
        { "case", 10 }
    };

    var testWords = new[] { "hello", "world", "use", "case" };

    var query = 
      from word in testWords
      let usage = GetWordUsageCount(wordUsage, word)
      orderby usage
      select (word, usage);

    foreach (var (word, usage) in query)
    {
        Console.WriteLine($"Word: '{word}' is repeated {usage} time(s).");
    }

    int GetWordUsageCount(Dictionary<string, int> wordUsageDic, string word)
        => wordUsageDic.TryGetValue(word, out var val) ? val : 0;
}

Finally, we get the expected (sorted) output!

Word: 'case' is repeated 10 time(s).
Word: 'world' is repeated 20 time(s).
Word: 'use' is repeated 75 time(s).
Word: 'hello' is repeated 100 time(s).

Note that in case you're still on an earlier version of C#, you could use a normal (most likely private) class method instead of a local method. If local functions are available though, I personally prefer them, since they help prevent classes from getting polluted with too many single-purpose helper methods.

Value Tuple

Just recently I came across a neat solution, which involves creating a value tuple as a container for the local out variable. Note though that this code is also C# 7.0+ only, as it requires declaring the out variable in the argument list of a method.

var query =
    from word in testWords
    let usage = (result: wordUsage.TryGetValue(word, out var localIntVar), count: localIntVar)
    orderby usage.count
    select (word, usage.count);

This solution will also scale nicely when the invoked method declares multiple out variables.

As a side note - using a value tuple inside of a query would be even easier when combined with tuple deconstruction. Something like this:

let (result, usage) = (wordUsage.TryGetValue(word, out var localIntVar), localIntVar)

Unfortunately, tuple deconstruction in LINQ queries is not supported at the moment:

error CS1001: Identifier expected
error CS1003: Syntax error, '=' expected

Maybe we'll get that supported in one of the future C# versions (more info in this discussion on GitHub).

Future

If you're interested in some of the latest development around local parameters in LINQ queries, check out the following postings in the csharplang GitHub repository:

Proposal: Support out/pattern variables in LINQ with query-scoping #159
Extend Out Variable support to LINQ #15619

Conclusion

To sum it up, if you have a need to consume an out variable inside of a LINQ query, use one of these options:

• A single-purpose local function (or a private class method)
• A helper value tuple

Do NOT use:

• A local variable (unless you really know what you're doing), as it may produce incorrect results
• An out variable declared inside of the argument list of a method call, as that is currently not supported inside of LINQ queries
• A deconstructed value tuple - tuple deconstruction is currently not supported in LINQ queries

What is your experience with out variables in LINQ queries?

If you find this post interesting, be sure to check out other posts on my blog mydevtricks.com.