Updated on 2021-10-28
Make your projects far more powerful with some foundational code templating techniques
This article has no downloads, but several tools are linked to from here.
I've noticed developers tend to have a love/hate relationship with code generation. Everyone has their opinions, but hopefully I can win over some detractors here, while providing some tools to ease the process.
I no longer use Visual Studio integration for most of my tools. The reason being is that it's just not as powerful and flexible as the pre-build step. It also takes longer to set up and debug, and makes your dev environment snowflakey and hard to replicate across workstations (installing half a dozen vsixs is no fun). Worse, you can't easily deploy a project to something like Github and expect other users can just clone the repo and go. That last bit is critical for team development, or even for you when you download the project onto a different machine two years from now and forget what tools you had installed. They are also harder to build than simple command line tools, won't work with VS Code (and JetBrains' offering?), etc. You can't put them in make files, and batch files, and the list goes on.
With pre-build steps, you deploy the tools with your project. I usually put them in the root solution folder for easy pathing in the pre-build steps but you do you. That way, when someone clones your repro, they are ready to go.
While it varies by project type, you can usually get to the pre-build steps under Visual Studio by going to the project's properties page, and selecting Build Events. Under Visual Basic, it's under some "Advanced" setting. I don't use Visual Studio for C++ development, but in VS Code you can edit the relevant .json files, or better yet, add a make file.
Know how to use command lines. You're a developer. The CLI should be comfortable enough by now. If not, you're about to get good at it. Fortunately, setting this up is a one-and-done situation so you don't have to mess with it every time you go to build. Your IDE/dev environment will do it for you.
When you set up a build step, remember to quote filenames, or any arguments with potential spaces in them. You can escape quotes with two quotes.
Visual Studio has helpful macros you can use to locate binaries, the solution folder, the project folder, etc. Use those when you make your build steps so that they are robust. In Visual Studio, just separate multiple command lines with enter.
Here's an example of pre-build events from a real world application build I have for an upcoming update to Reggie.
"$(SolutionDir)csppg.exe" "$(ProjectDir)Export\Common.cs.template" /output
"$(ProjectDir)Export.CommonGenerator.cs" /class CommonGenerator /namespace Reggie /internal
"$(SolutionDir)csppg.exe" "$(ProjectDir)Export\TableMatcher.cs.template" /output
"$(ProjectDir)Export.TableMatcherGenerator.cs" /class TableMatcherGenerator
/namespace Reggie /internal
"$(SolutionDir)csppg.exe" "$(ProjectDir)Export\TableTokenizer.cs.template" /output
"$(ProjectDir)Export.TableTokenizerGenerator.cs" /class TableTokenizerGenerator
/namespace Reggie /internal
"$(SolutionDir)csbrick.exe" "$(SolutionDir)LexContext\LexContext.csproj" /output
"$(ProjectDir)LexContext.brick.cs"
"$(SolutionDir)csbrick.exe" "$(SolutionDir)FastFA\FastFA.csproj"
/output "$(ProjectDir)FastFA.brick.cs"All of my tools use the same basic syntax to make it easier to work them. I'm going to give you some boilerplate code later when we cover quickly making your own tools.
These commands will execute any time the project is built. If you don't want them to run every time, for example, if your process takes a long time to execute, you can add an /ifstale switch to your executable that skips the process unless the input file is newer than the output file.
In the .NET realm, there are a couple of different ways to do code generation. You can use the CodeDOM or you can use some sort of text templating system, depending on how you want to go about it.
System.Object in order to get the Visual Basic code provider to render them properly when the object also inherits from interfaces.yield keyword. Even some common operators are missing.**CodePrimitiveExpression and CodeIterationStatement and involves a lot of nesting of objects so building them out in code requires a lot of code.**** Deslang is a powerful tool that employs Slang - a CodeDOM compliant subset of C# which parses into a CodeDOM tree and creates graphs in code you can include in your code generation projects. Using Deslang, you can write code in a C# subset and render it to Visual Basic, for example. You can also do complex operations like visiting each element of a CodeDOM tree in order to do analysis and transformation. Furthermore, you can do things like reflection, including type and method resolution, plus constant expression evaluation (folding) over an in-memory code model. Using Deslang as a pre-build step, you can generate really complicated code very easily and in a language independent manner, and then take the resulting generated code, include it in your code generator project, and do transformations and rendering to the language of your choice with the resulting tree. See the articles linked to prior for more. Even though it is somewhat experimental, and you may have to tinker with it to get it to parse sometimes, once you've got it going it makes blasting out wicked code pretty easy. I highly recommend using Deslang if you choose to go the CodeDOM route. It is orders of magnitude more powerful and efficient than using the CodeDOM by hand.
Deslang to do analysis and transformation
Using csppg, you can create text based templates using ASP-like or T4-like syntax. The templates themselves are generated as C# files. You can then include the C# files in your code generator project to run them any time you need output from them.
You can use text templates to render to Slang, which can then be parsed into code to build the CodeDOM tree the code represents, using Deslang. The tree can then be analyzed and reflected upon before being rendered to any desired language. The only real disadvantage to this is that it's more complicated than the other approaches, and incorporates some of the disadvantages of using the CodeDOM..
I write a lot of code generation tools. You might say I'm a code generation aficionada/o. My father was a toolmaker so in a way I think I've got some of the same drive he did. With it comes some hard won experience.
I used to use the CodeDOM approach because of the advantages to it, but these days I'm about simplifying. I also am more interested in targeting things like C++ and even SQL/DDL.
I would still revert to the CodeDOM model, particularly if I want to use Slang at runtime like Parsley does, but now I tend to want to avoid it mostly due to the added complexity of it, even using tools like Deslang.
Many times, you won't find a tool that does precisely what you need. When that happens, you'll need to make one, which the aforementioned tools can help with. After you've chosen which direction to go with your generator (either CodeDOM or text templates or both) you need to throw together a file to process command line arguments:
using System;
using System.IO;
using System.Reflection;
namespace Tool
{
static class Program
{
static readonly string CodeBase = _GetCodeBase();
static readonly string Filename = Path.GetFileName(CodeBase);
static readonly string Name = _GetName();
static readonly Version Version = _GetVersion();
static readonly string Description = _GetDescription();
static int Main(string[] args) => Run(args, Console.In, Console.Out, Console.Error);
/// <summary>
/// Runs the process
/// </summary>
/// <param name="args">The arguments</param>
/// <param name="stdin">The input stream</param>
/// <param name="stdout">The output stream</param>
/// <param name="stderr">The error stream</param>
/// <returns>The error code</returns>
public static int Run(string[] args, TextReader stdin,
TextWriter stdout, TextWriter stderr)
{
string inputfile = null;
string outputfile = null;
bool ifstale = false;
// our working variables
var result = 0;
TextReader input = null;
TextWriter output = null;
try
{
if (0 == args.Length)
{
result = -1;
_PrintUsage(stderr);
}
else if (args[0].StartsWith("/"))
{
throw new ArgumentException("Missing input file.");
}
else
{
// process the command line args
inputfile = args[0];
for (var i = 1; i < args.Length; ++i)
{
switch (args[i].ToLowerInvariant())
{
case "/output":
if (args.Length - 1 == i) // check if we're at the end
throw new ArgumentException(string.Format
("The parameter \"{0}\" is missing an argument",
args[i].Substring(1)));
++i; // advance
outputfile = args[i];
break;
case "/ifstale":
ifstale = true;
break;
default:
throw new ArgumentException(string.Format
("Unknown switch {0}", args[i]));
}
}
if (string.IsNullOrWhiteSpace(inputfile))
throw new ArgumentException("inputfile");
var cwd = Environment.CurrentDirectory;
if (!ifstale || _IsStale(inputfile, outputfile))
{
if (null != outputfile)
{
stderr.WriteLine("{0} is building file: {1}", Name, outputfile);
cwd = Path.GetDirectoryName(outputfile);
output = new StreamWriter(outputfile);
}
else
{
stderr.WriteLine("{0} is building preprocessor.", Name);
output = stdout;
}
input = new StreamReader(inputfile);
//
// TODO: Do work here
//
}
else
{
stderr.WriteLine("{0} skipped building of {1}
because it was not stale.", Name, outputfile);
}
}
}
// we don't like to catch in debug mode
#if !DEBUG
catch (Exception ex)
{
result = _ReportError(ex, stderr);
}
#endif
finally
{
// close the input file if necessary
if (null != input)
input.Close();
// close the output file if necessary
if (null != outputfile && null != output)
output.Close();
}
return result;
}
static void _PrintUsage(TextWriter w)
{
w.Write("Usage: " + Filename + " ");
w.WriteLine("<inputfile> [/output <outputfile>] [/ifstale]");
w.WriteLine();
w.Write(Name);
w.Write(" ");
w.Write(Version.ToString());
if (!string.IsNullOrWhiteSpace(Description))
{
w.Write(" - ");
w.WriteLine(Description);
}
else
{
w.WriteLine(" - <No description>");
}
w.WriteLine();
w.WriteLine(" <inputfile> The input template");
w.WriteLine(" <outputfile> The generated file - defaults to STDOUT");
w.WriteLine(" <ifstale> Only generate if the input is newer than the output");
w.WriteLine();
}
static bool _IsStale(string inputfile, string outputfile)
{
if (string.IsNullOrEmpty(outputfile) || string.IsNullOrEmpty(inputfile))
return true;
var result = true;
// File.Exists doesn't always work right
try
{
if (File.GetLastWriteTimeUtc(outputfile) >= File.GetLastWriteTimeUtc(inputfile))
result = false;
}
catch { }
return result;
}
static string _GetCodeBase()
{
try { return Assembly.GetExecutingAssembly().GetModules()[0].FullyQualifiedName; }
catch { return Path.Combine(Environment.CurrentDirectory,
typeof(Program).Namespace + ".exe"); }
}
static string _GetName()
{
try
{
foreach (var attr in Assembly.GetExecutingAssembly().CustomAttributes)
{
if (typeof(AssemblyTitleAttribute) == attr.AttributeType)
{
return attr.ConstructorArguments[0].Value as string;
}
}
}
catch { }
return Path.GetFileNameWithoutExtension(Filename);
}
static Version _GetVersion()
{
return Assembly.GetExecutingAssembly().GetName().Version;
}
static string _GetDescription()
{
string result = null;
foreach (Attribute ca in Assembly.GetExecutingAssembly().GetCustomAttributes())
{
var ada = ca as AssemblyDescriptionAttribute;
if (null != ada && !string.IsNullOrWhiteSpace(ada.Description))
{
result = ada.Description;
break;
}
}
return result;
}
// do our error handling here (release builds)
static int _ReportError(Exception ex, TextWriter stderr)
{
_PrintUsage(stderr);
stderr.WriteLine("Error: {0}", ex.Message);
return -1;
}
}
}You can paste the above into Program.cs in your command line project. It is boilerplate for a robust CLI application that's easy to add new parsed arguments to, has a using screen, can detect if the output is stale, and is made such that it can be referenced as a library from another tool, like a Visual Studio Custom File Generator.
You simply put your main code where the TODO: Do work here comment is.
One thing you'll want to avoid when you build a tool like this is using external libraries unless absolutely necessary. The reason being is, as a pre-build step, requiring extra dependencies for the tool complicates its use. I know it's common practice with .NET to have a half a dozen .dlls for even the smallest project, but here you'll want to depart from that because a bunch of .dlls in your build tree is just a mess.
You can use csbrick to crunch all of the source files for a dependent project into a single file "code brick" that can be included into your project in lieu of using a .dll. Just remove the dependency from your references and add a pre-build step that runs csbrick with all your source files from the project. Then add the generated output to your project.
Hopefully, this article helps you improve and streamline your development.