.NET Framework Compiler: Under the Hood

In the previous blog post, we learned about the C# compilation process - how C# compiler translates high-level programming language into executable machine code. We also discussed the MSBuild Engine used in the .NET framework.

A quick recap of the last article in this series before we move on:

• After a developer writes the C# code, they compile the code resulting in Common Intermediate Language (CIL) stored within Portable Executable files such as .exe or .dll files for Windows. These files are distributed or deployed to users.
• When a user launches a .NET program, the OS invokes the Common Language Runtime (CLR). The CLR's Just-In-Time compiles the CIL to the native code appropriate for the plarform/OS it is running on.

In this week's article, we will build a file using the MSBuild Engine by creating a project file from scratch in order to better understand this process. We will:

• Create a simple "Hello World" console application in C#
• Build (compile) and execute the program using the barebones C# compiler
• Build the program using MSBuild Engine in order to understand how the .NET framework handles the build process

We will be using the .NET CLI from the command prompt instead of Visual Studio IDE to avoid as many abstraction layers as possible for the purpose of our own understanding of the framework.

Let's get started!

Create a Console Application

1. Open a Windows command prompt and create a folder named CSCompilerDemo using the mkdir command. Create a new file named HelloWorld.cs inside the new folder.
   

   mkdir CSCompilerDemo
   cd CSCompilerDemo
   echo > HelloWorld.cs
   

2. Create a new class named HelloWorld and add a static Main method whose purpose is simply to output "Hello World!" to the console. Your HelloWorld.cs should look like the following:
   

   using System;
   
   class HelloWorld
   {
       static void Main(string[] args)
       {
           Console.WriteLine("Hello World!");
       }
   }
   

Build and Test Application

Let's build and test our application. Before we start using the MSBuild Engine to build our application, let's take look at how we can do that without MSBuild. The csc command can be used in the command prompt to compile a C# file. So let's try that.

// Build C# file and output the executable HelloWorld.exe
csc HelloWorld.cs       
// Run the program
HelloWorld.exe

It works!🎉 So why do we need MSBuild if we can just use the csc command to build our application?

In fact, MSBuild uses csc.exe, which we shall learn in a few minutes. However, csc and MSBuild are completely different applications. MSBuild uses csc.exe as its actual compiler, but it knows where to find assemblies, references, etc. based on your solution and project files. When using csc.exe, you must provide paths to your files, references, etc. making compilation much more difficult to manage as it forces you to understand on a deeper level what you are instructing the compiler to do. So, let's give MSBuild a try. But first, let's clean up our project by deleting the generated HelloWorld.exe and HelloWorld.res files since we no longer need these.

Build Application Using MSBuild Engine

As mentioned above, MSBuild relies on the project file. So let's create a minimal project file named HelloWorld.csproj and put it in the same folder directory as the HelloWorld.cs file. Your project file should look like the following:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <PropertyGroup>
        <AssemblyName>HelloWorld</AssemblyName>
        <OutputPath>Bin\</OutputPath>
    </PropertyGroup>
    <ItemGroup>
        <Compile Include="HelloWorld.cs"/>
    </ItemGroup>
    <Target Name="Build">
        <MakeDir Directories="$(OutputPath)" Condition="!Exists('$(OutputPath)')" />
        <Csc Sources="@(Compile)" OutputAssembly="$(OutputPath)$(AssemblyName).exe" />
    </Target>
 </Project>

Now, let's pause and walk through the changes we've just added to this file.

MSBuild project files are XML files that adhere to the MSBuild XML schema. Below is a quick explaination of what each element in the schema represents:

1. <Project>: The schema link in the xmlns attribute represents the XML namespace.
2. <PropertyGroup>: Contains a set of user-defined <Property> elements. Every Property element used in an MSBuild project must be a child of a PropertyGroup element. Properties are name-value pairs that can be used to configure builds. They are useful for passing values to tasks, evaluating conditions, and storing values that will be referenced throughout the project file.
3. <AssemblyName> and <OutputPath>: Properties defined under Property Group. In this case, AssemblyName is the name of the output file and is referenced in the <Csc> element. Similarly, the <OutputPath> is the file path for the output file referenced in the <Csc> element.
4. <ItemGroup>: Contains a set of user-defined <Item> elements. Similar to <PropertyGroup>, every item used in an MSBuild project must be specified as a child of an <ItemGroup> element.
5. <Compile>: This element is an Item element, which is a child element of the <ItemGroup> element described above. Item element defines inputs into the build system, and they typically represent files specified in the Include attribute. An item is a named reference to one or more files. Items contain metadata such as file names, path, and version numbers. The name of the element is the type of the item. Item types are named lists of items that can be used as parameters for tasks. The tasks use the item values to perform the steps of the build process. In this example, the <Compile> item type represents the source files for the compiler, which is HelloWorld.cs.
6. <Target>: Contains a set of tasks for MSBuild to execute sequentially. The Name attribute is a required attribute representing the name of the target. Each task element is a child element of the <Target> element. The element name of the task is determined by the name of the task being created.
7. <MakeDir>: A task specified for the parent "Build" <Target> as described above. This task is used to create directories. In this example, the build will create a folder whose name is specified by the <OutputPath> element described above. The Condition attribute is used to skip the folder creation if such folder already exists.
8. <Csc>: This task is basically equivalent to calling the csc command from the command-line interface to compile the C# file that we discussed in the earlier section. The Sources attribute specifies the C# source files that the compiler should use, and the OutputAssembly attribute specifies the name of the output file. The @ is used with the Compile in order to pass the values specified in the <Compile> Item under <ItemGroup> into the <Csc> task as a parameter. Also, notice how the $ character is used to reference <OutputPath> and <AssemblyName> properties defined earlier. The property names are also placed between the parentheses.

In summary, the project file above is used to instruct MSBuild to:

1. Create a folder name Bin from the current directory.
2. Compile the HelloWorld.cs source file using csc.
3. Place the output file named HelloWorld.exe into the Bin folder.

Now that we have our project file created, we're finally ready to start building our application using MSBuild.

You can run MSBuild from Visual Studio or from the command window. If you have Visual Studio installed, then you already have MSBuild installed. We're going to use the Developer Command Prompt for Visual Studio, which you can search for in the Windows 10 search box in the taskbar. You can also use the ordinary Windows command prompt, but there are additional environment setup required.

From the command prompt, run the following command:

msbuild helloworld.csproj -t:Build

Assuming the build succeeds, you should now have the HelloWorld.exe file in the Bin folder. You can test the built file by cd into the Bin folder and run the HelloWorld command. You should see Hello World! output to the console. 🎉

That's it for this week. In the next part of this series, we will dive a little deeper to look at the compiled intermediate language file to see how it was translated and what it actually does. See you then! 👋