Skip to content

T4 articles

Multiple outputs from T4 made easy – revisited  

My multiple outputs from t4 made easy post contained a class making it easy to produce multiple files from Visual Studio’s text templating engine (T4).

While useful it had a few issues:

  • Getting start/end blocks mixed up resulted in unpredictable behaviour
  • Files were rewritten even when content did not change
  • Did not play well with source control
  • Files not always deleted in VS
  • Failed in Visual Studio’s project-less Web Sites

This helper class forms the basis of multiple file output for Entity Framework templates in .NET 4.0 and the LINQ to SQL templates on CodePlex so we (Jeff Reed, Andrew Peters and myself) made the following changes.

Improvements

Simpler block handling

The header, footer and file blocks can now be completed with EndBlock (EndHeader and EndFooter are gone), although it will automatically end the previous block when it hits a new one or the final Process method.

Skip unchanged files

Files are now only written to disk if the contents are different with the exception of the original T4 output file (we can’t stop that, sorry).

There is additional overhead reading and comparing files we believe unmodified files keeping their dates and source control status are worth it.

Automatic checkout

When the template detects it is running in Visual Studio and that the file it needs to write to is currently in source control but not checked out it will check the file out for you.

Predictable clean-up

All files that were not part of the generation process but are nested under the project item will now be deleted when running inside Visual Studio.

Outside of Visual Studio files are no longer deleted – this was destructive and it couldn’t know which files it generated on a previous run to clean-up correctly anyway.

Website projects fall back to single file generation

Visual Studio has both web sites and web applications with the former being project-less leading to very messy multi-file generation so it forces single file generation.

Internal improvements

Source is now simpler to read and understand with less public visibility and faster and more robust VS interop by batching the files & deletes to a single invoke at the end to avoid conflicts with other add-ins that might be triggered by the changes.

Usage

Initialization

You’ll need to get the code into your template – either copy the code in or reference it with an include directive. Then declare an instance of the Manager class passing in some environmental options such as the desired default output path. (For Visual Studio 2010 remove the #v3.5 portion from the language attribute)

<#@ template language="C#v3.5" hostspecific="True"
#><#@include file="Manager.ttinclude"
#><# var manager = Manager.Create(Host, GenerationEnvironment); #>

File blocks

Then add one line before and one line after each block which could be split out into it’s own file passing in what the filename would be if split. The EndBlock is optional if you want it to carry through to the next one :)

<# manager.StartNewFile("Employee.generated.cs"); #>
public class Employee { … }
<# manager.EndBlock(); #>

Headers & footers

Many templates need to share a common header/footer for such things as comments or using/import statements or turning on/off warnings. Simply use StartHeader and StartFooter and the blocks will be emitted to the start and end of all split files as well as being left in the original output file.

<# manager.StartHeader(); #>
// Code generated by a template
using System;

<# manager.EndBlock(); #>

Process

At the end of the template call Process to handle splitting the files (true) or not (false). Anything not included in a specific StartNewFile block will remain in the original output file.

<# manager.Process(true); #>

Revised Manager class

Latest source available at GitHub
<#@ assembly name="System.Core"
#><#@ assembly name="System.Data.Linq"
#><#@ assembly name="EnvDTE"
#><#@ assembly name="System.Xml"
#><#@ assembly name="System.Xml.Linq"
#><#@ import namespace="System"
#><#@ import namespace="System.CodeDom"
#><#@ import namespace="System.CodeDom.Compiler"
#><#@ import namespace="System.Collections.Generic"
#><#@ import namespace="System.Data.Linq"
#><#@ import namespace="System.Data.Linq.Mapping"
#><#@ import namespace="System.IO"
#><#@ import namespace="System.Linq"
#><#@ import namespace="System.Reflection"
#><#@ import namespace="System.Text"
#><#@ import namespace="System.Xml.Linq"
#><#@ import namespace="Microsoft.VisualStudio.TextTemplating"
#><#+

// Manager class records the various blocks so it can split them up
class Manager {
    private class Block {
        public String Name;
        public int Start, Length;
    }

    private Block currentBlock;
    private List<Block> files = new List<Block>();
    private Block footer = new Block();
    private Block header = new Block();
    private ITextTemplatingEngineHost host;
    private StringBuilder template;
    protected List<String> generatedFileNames = new List<String>();

    public static Manager Create(ITextTemplatingEngineHost host, StringBuilder template) {
        return (host is IServiceProvider) ? new VSManager(host, template) : new Manager(host, template);
    }

    public void StartNewFile(String name) {
        if (name == null)
            throw new ArgumentNullException("name");
        CurrentBlock = new Block { Name = name };
    }

    public void StartFooter() {
        CurrentBlock = footer;
    }

    public void StartHeader() {
        CurrentBlock = header;
    }

    public void EndBlock() {
        if (CurrentBlock == null)
            return;
        CurrentBlock.Length = template.Length - CurrentBlock.Start;
        if (CurrentBlock != header && CurrentBlock != footer)
            files.Add(CurrentBlock);
        currentBlock = null;
    }

    public virtual void Process(bool split) {
        if (split) {
            EndBlock();
            String headerText = template.ToString(header.Start, header.Length);
            String footerText = template.ToString(footer.Start, footer.Length);
            String outputPath = Path.GetDirectoryName(host.TemplateFile);
            files.Reverse();
            foreach(Block block in files) {
                String fileName = Path.Combine(outputPath, block.Name);
                String content = headerText + template.ToString(block.Start, block.Length) + footerText;
                generatedFileNames.Add(fileName);
                CreateFile(fileName, content);
                template.Remove(block.Start, block.Length);
            }
        }
    }

    protected virtual void CreateFile(String fileName, String content) {
        if (IsFileContentDifferent(fileName, content))
            File.WriteAllText(fileName, content);
    }

    public virtual String GetCustomToolNamespace(String fileName) {
        return null;
    }

    public virtual String DefaultProjectNamespace {
        get { return null; }
    }

    protected bool IsFileContentDifferent(String fileName, String newContent) {
        return !(File.Exists(fileName) && File.ReadAllText(fileName) == newContent);
    }

    private Manager(ITextTemplatingEngineHost host, StringBuilder template) {
        this.host = host;
        this.template = template;
    }

    private Block CurrentBlock {
        get { return currentBlock; }
        set {
            if (CurrentBlock != null)
                EndBlock();
            if (value != null)
                value.Start = template.Length;
            currentBlock = value;
        }
    }

    private class VSManager: Manager {
        private EnvDTE.ProjectItem templateProjectItem;
        private EnvDTE.DTE dte;
        private Action<String> checkOutAction;
        private Action<IEnumerable<String>> projectSyncAction;

        public override String DefaultProjectNamespace {
            get {
                return templateProjectItem.ContainingProject.Properties.Item("DefaultNamespace").Value.ToString();
            }
        }

        public override String GetCustomToolNamespace(string fileName) {
            return dte.Solution.FindProjectItem(fileName).Properties.Item("CustomToolNamespace").Value.ToString();
        }

        public override void Process(bool split) {
            if (templateProjectItem.ProjectItems == null)
                return;
            base.Process(split);
            projectSyncAction.EndInvoke(projectSyncAction.BeginInvoke(generatedFileNames, null, null));
        }

        protected override void CreateFile(String fileName, String content) {
            if (IsFileContentDifferent(fileName, content)) {
                CheckoutFileIfRequired(fileName);
                File.WriteAllText(fileName, content);
            }
        }

        internal VSManager(ITextTemplatingEngineHost host, StringBuilder template)
            : base(host, template) {
            var hostServiceProvider = (IServiceProvider) host;
            if (hostServiceProvider == null)
                throw new ArgumentNullException("Could not obtain IServiceProvider");
            dte = (EnvDTE.DTE) hostServiceProvider.GetService(typeof(EnvDTE.DTE));
            if (dte == null)
                throw new ArgumentNullException("Could not obtain DTE from host");
            templateProjectItem = dte.Solution.FindProjectItem(host.TemplateFile);
            checkOutAction = (String fileName) => dte.SourceControl.CheckOutItem(fileName);
            projectSyncAction = (IEnumerable<String> keepFileNames) => ProjectSync(templateProjectItem, keepFileNames);
        }

        private static void ProjectSync(EnvDTE.ProjectItem templateProjectItem, IEnumerable<String> keepFileNames) {
            var keepFileNameSet = new HashSet<String>(keepFileNames);
            var projectFiles = new Dictionary<String, EnvDTE.ProjectItem>();
            var originalFilePrefix = Path.GetFileNameWithoutExtension(templateProjectItem.get_FileNames(0)) + ".";
            foreach(EnvDTE.ProjectItem projectItem in templateProjectItem.ProjectItems)
                projectFiles.Add(projectItem.get_FileNames(0), projectItem);

            // Remove unused items from the project
            foreach(var pair in projectFiles)
                if (!keepFileNames.Contains(pair.Key) && !(Path.GetFileNameWithoutExtension(pair.Key) + ".").StartsWith(originalFilePrefix))
                    pair.Value.Delete();

            // Add missing files to the project
            foreach(String fileName in keepFileNameSet)
                if (!projectFiles.ContainsKey(fileName))
                    templateProjectItem.ProjectItems.AddFromFile(fileName);
        }

        private void CheckoutFileIfRequired(String fileName) {
            var sc = dte.SourceControl;
            if (sc != null && sc.IsItemUnderSCC(fileName) && !sc.IsItemCheckedOut(fileName))
                checkOutAction.EndInvoke(checkOutAction.BeginInvoke(fileName, null, null));
        }
    }
} #>

[)amien

Multiple outputs from T4 made easy  

An improved version is now available.

One of the things I wanted my LINQ to SQL T4 templates to do was be able to split the output into a file-per-entity. Existing solutions used either a separate set of templates with duplicate code or intrusive handling code throughout the template. Here’s my helper class to abstract the problem away from what is already complicated enough template code.

Using the Manager class

Setup

You’ll need to get the code into your template – either copy the code in or reference it with an include directive. Then declare an instance of the Manager class passing in some environmental options such as the desired default output path.

<#@ template language="C#v3.5" hostspecific="True"
#><#@ include file="Manager.ttinclude"
#><# var manager = new Manager(Host, GenerationEnvironment, true) { OutputPath = Path.GetDirectoryName(Host.TemplateFile) }; #>

Define a block

Then add one line before and one line after each block which could be split out into it’s own file passing in what the filename would be if split.

<# manager.StartBlock("Employee.generated.cs"); #>
public class Employee { … }
<# manager.EndBlock(); #>

Headers and footers

Many templates need to share a common header/footer for such things as comments or using/import statements or turning on/off warnings. Simply use StartHeader/EndHeader and StartFooter/EndFooter. The resulting blocks will be emitted into all split files and left in the original output too.

<# manager.StartHeader(); #>
// Code generated template
using System;

<# manager.EndHeader(); #>

Process

At the end of the template call Process to handle splitting the files (true) or not (false). Anything not included in a specific start/end block will remain in the original output file.

<# manager.Process(true); #>

When processing each block name in the Output path will either be overwritten or deleted to enable proper clean-up. It will also add and remove the files from Visual Studio so make sure your generated names aren’t going to collide with hand-written ones!

Manager classes

Here is the Manger class itself as well as the small ManagementStrategy classes that determines what to do with the files within Visual Studio (add/remove project items) and outside of Visual Studio (create/delete files).

Download Manager.ttinclude (4KB)

<#@ assembly name="System.Core"
#><#@ assembly name="EnvDTE"
#><#@ import namespace="System.Collections.Generic"
#><#@ import namespace="System.IO"
#><#@ import namespace="System.Text"
#><#@ import namespace="Microsoft.VisualStudio.TextTemplating"
#><#+

// T4 Template Block manager for handling multiple file outputs more easily.
// Copyright (c) Microsoft Corporation.  All rights reserved.
// This source code is made available under the terms of the Microsoft Public License (MS-PL)

// Manager class records the various blocks so it can split them up
class Manager
{
	private struct Block {
		public String Name;
		public int Start, Length;
	}

	private List<Block> blocks = new List<Block>();
	private Block currentBlock;
	private Block footerBlock = new Block();
	private Block headerBlock = new Block();
	private ITextTemplatingEngineHost host;
	private ManagementStrategy strategy;
	private StringBuilder template;
	public String OutputPath { get; set; }

	public Manager(ITextTemplatingEngineHost host, StringBuilder template, bool commonHeader) {
		this.host = host;
		this.template = template;
		OutputPath = String.Empty;
		strategy = ManagementStrategy.Create(host);
	}

	public void StartBlock(String name) {
		currentBlock = new Block { Name = name, Start = template.Length };
	}

	public void StartFooter() {
		footerBlock.Start = template.Length;
	}

	public void EndFooter() {
		footerBlock.Length = template.Length - footerBlock.Start;
	}

	public void StartHeader() {
		headerBlock.Start = template.Length;
	}

	public void EndHeader() {
		headerBlock.Length = template.Length - headerBlock.Start;
	}	

	public void EndBlock() {
		currentBlock.Length = template.Length - currentBlock.Start;
		blocks.Add(currentBlock);
	}

	public void Process(bool split) {
		String header = template.ToString(headerBlock.Start, headerBlock.Length);
		String footer = template.ToString(footerBlock.Start, footerBlock.Length);
		blocks.Reverse();
		foreach(Block block in blocks) {
			String fileName = Path.Combine(OutputPath, block.Name);
			if (split) {
				String content = header + template.ToString(block.Start, block.Length) + footer;
				strategy.CreateFile(fileName, content);
				template.Remove(block.Start, block.Length);
			} else {
				strategy.DeleteFile(fileName);
			}
		}
	}
}

class ManagementStrategy
{
	internal static ManagementStrategy Create(ITextTemplatingEngineHost host) {
		return (host is IServiceProvider) ? new VSManagementStrategy(host) : new ManagementStrategy(host);
	}

	internal ManagementStrategy(ITextTemplatingEngineHost host) { }

	internal virtual void CreateFile(String fileName, String content) {
		File.WriteAllText(fileName, content);
	}

	internal virtual void DeleteFile(String fileName) {
		if (File.Exists(fileName))
			File.Delete(fileName);
	}
}

class VSManagementStrategy : ManagementStrategy
{
	private EnvDTE.ProjectItem templateProjectItem;

	internal VSManagementStrategy(ITextTemplatingEngineHost host) : base(host) {
		IServiceProvider hostServiceProvider = (IServiceProvider)host;
		if (hostServiceProvider == null)
			throw new ArgumentNullException("Could not obtain hostServiceProvider");

		EnvDTE.DTE dte = (EnvDTE.DTE)hostServiceProvider.GetService(typeof(EnvDTE.DTE));
		if (dte == null)
			throw new ArgumentNullException("Could not obtain DTE from host");

		templateProjectItem = dte.Solution.FindProjectItem(host.TemplateFile);
	}

	internal override void CreateFile(String fileName, String content) {
		base.CreateFile(fileName, content);
		((EventHandler)delegate { templateProjectItem.ProjectItems.AddFromFile(fileName); }).BeginInvoke(null, null, null, null);
	}

	internal override void DeleteFile(String fileName) {
		((EventHandler)delegate { FindAndDeleteFile(fileName); }).BeginInvoke(null, null, null, null);
	}

	private void FindAndDeleteFile(String fileName) {
		foreach(EnvDTE.ProjectItem projectItem in templateProjectItem.ProjectItems) {
			if (projectItem.get_FileNames(0) == fileName) {
				projectItem.Delete();
				return;
			}
		}
	}
}#>

[)amien

LINQ to SQL templates updated, now on CodePlex  

My templates that allow you to customize the LINQ to SQL code-generation process (normally performed by SQLMetal/LINQ to SQL classes designer) have been updated once again.

Updates

  • Now licensed under the Microsoft Public License and hosted at CodePlex
  • User options specified with a var options block at the start of the template
  • Option for each class to be a separate file that is reflected in the VS project (EntityPerFile=true)
  • Detection and support of IsComposable functions
  • General code clean-up and better error handling such as missing DBML file

CodePlex

CodePlex makes it easier for people to be able to see and merge updates in with their own modified versions as well as report issues via the issue tracker etc. There is also an RSS feed that lets you keep track of releases, source updates or whatever else you are interested in.

For now it is a grab-the-source style release but I hope to publish downloadable tested releases wrapped up in a Visual Studio Installer (VSI) package to make getting started easier soon.  Feel free to grab the sources directly via TFS/Subversion to be able to diff them with your own modified versions.

Enjoy!

[)amien

LINQ to SQL template for Visual Studio 2008  

A newer version of this LINQ to SQL template is available.

If you want to customize the LINQ to SQL code generation phase in your project without additional tool dependencies this could be what you’re looking for.

11 Dec 2008 Fixes to association code with one-to-one’s and with no serialization required.
7 Dec 2008 Added IsDelayLoaded, IsUnique. Fixed association code, stored proc update overrides.
17 Nov 2008 Added IsInheritanceDefault read/emit IsDefault on type
26 Oct 2008 Emit column Name attribute if it doesn’t match the member (thanks Steele)
20 Oct 2008 Handle class naming better, fix associations for renamed or out-of-sequence keys and their access modifiers
25 Sep 2008 Figure out missing association keys by using either sides primary key
25 Sep 2008 Empty DBML namespace is now no namespace instead of “MyApplication”
23 Sep 2008 fixed stored procedures with 0 parameters
22 Sep 2008 fixed VB.NET IsForeignKey attribute for associations
18 Sep 2008 now generates stored procedures including insert/update/delete with concurrency checking.

New since ‘reloaded’ version

  • Inheritance – generates subclasses with all properties and code mappings.
  • VB.NET CSharpDataContext.tt is joined by a VB.NET emitting VBNetDataContext.tt.
  • DataContract SP1 – additional mode to emit SP1-compatible DataContract serialization via Roger Jennings.
  • Composite keys – both as the primary key and as a foreign key in an association.
  • Type attributes – the data context and entity types can now be sealed or abstract as well as public, private, protected, internal or protected internal.
  • Associations – prevents foreign key values changing once the object association is made and updates parent side of one-to-many associations.
  • Stored procedures – generates method wrappers and associated methods to facilitate insert/update/delete with concurrency support.

Functionality compared to designer

A primary goal in developing the template was to allow for easy switching between the template and the LINQ to SQL designer so things are very similar.

Missing

  • Comprehensive sanity checking on the DBML.
  • The Custom Tool Namespace and project namespaces are not pulled in when the DBML namespaces not specified.

Fixed

The designer has a few bugs which helpfully this template doesn’t suffer from.

  • Modifying a table via a stored procedure using original values for concurrency will throw ChangeConflictException and not silently fail.
  • Protected internal virtual property doesn’t forget to be virtual.
  • Checks all associations based on a foreign key are not loaded before allowing change and not just the first one.

Improved

  • Fully customizable with full source.
  • Serialization mode to support DataContract improvements in .NET 3.5 SP1
    To use uncomment the line // data.Serialization = SerializationMode.DataContractSP1; in xDataClasses.tt
  • CanBeNull attribute generated for value types (useful when working with metadata).

Source compared to designer

The designer generated code can be difficult to read and isn’t well suited to template generation so the output from this template is different in a number of ways:

Sequence

Everything related to a column mapping – the storage variable, event signatures, attribute and the property itself – is batched together so it can be hit with a single loop making the template shorter and easier to work with.

#regions

Opinion may be divided on the usefulness of #regions in your own code but for code generation of large files I found it invaluable. There are regions for the logical parts of the data context and within each entity such as construction, column mapping, associations and serialization.

Style

The code generated should be a little easier to follow – if/else ordering, no redundant casts or extra brackets etc.

Namespace

I don’t believe adding “this” everywhere or fully qualifying attributes and exceptions makes things easy to read. I realize this might cause some name conflicts for some people but it is easy to change yourself and means the code is shorter and easier to work with for the majority.

Getting started

Although I work on the LINQ to SQL team this template should be treated as a third-party sample and is not supported by Microsoft.

Download via CodePlex

  1. Add the L2ST4.ttinclude and either CSharpDataContext.tt or VBNetDataContext.tt to your project depending on your language type
  2. Rename the xDataContext.tt to match your DBML file but with .tt extension instead of .dbml
  3. Set the existing DBML file’s .designer.cs/vb Build Action property to None to ignore the LINQ to SQL built-in code generation

Note that the template will only regenerate when it has been changed so use Run Custom Tool from the template’s right-mouse button menu in Solution Explorer when you’ve changed the DBML.

Should you wish to switch back to using the designer code then set the DBML file’s .designer.cs/vb Build Action to Compile and either remove the .tt and .ttinclude file for permanent removal or just set the .generated.cs/vb Build Action to None to keep it around. VB.NET users will need to use Show All Files to see the .designer.vb file.

Customization

The template runtime built into Visual Studio 2008 is called T4 and requires no additional tools however if you do a lot of editing you might want to install the Clarius T4 Editor for syntax highlighting and also check out the treasure trove of T4 material that is Oleg Sych’s blog.

The template is simple to follow, it loads the DBML file as an XML document then uses LINQ to XML to instantiate wrapper objects over the elements. This gives you a simple way to change default naming and behavior while making the template simpler to work with.

Let me know how you get on by leaving a comment here.

[)amien