Multiple outputs from T4 made easy – revisited

My multiple outputs from t4 made easy post contained a class making it easy to produce multiple files from Visual Studio’s text templating engine (T4).

While useful it had a few issues:

  • Getting start/end blocks mixed up resulted in unpredictable behavior
  • Files were rewritten even when content did not change
  • Did not play well with source control
  • Files not always deleted in VS
  • Failed in Visual Studio’s project-less Web Sites

This helper class forms the basis of multiple file output for Entity Framework templates in .NET 4.0 and the LINQ to SQL templates on CodePlex so we (Jeff Reed, Andrew Peters and myself) made the following changes.

Improvements

Simpler block handling

The header, footer and file blocks can now be completed with EndBlock (EndHeader and EndFooter are gone), although it will automatically end the previous block when it hits a new one or the final Process method.

Skip unchanged files

Files are now only written to disk if the contents are different with the exception of the original T4 output file (we can’t stop that, sorry).

There is additional overhead reading and comparing files we believe unmodified files keeping their dates and source control status are worth it.

Automatic checkout

When the template detects it is running in Visual Studio and that the file it needs to write to is currently in source control but not checked out it will check the file out for you.

Predictable clean-up

All files that were not part of the generation process but are nested under the project item will now be deleted when running inside Visual Studio.

Outside of Visual Studio files are no longer deleted – this was destructive and it couldn’t know which files it generated on a previous run to clean-up correctly anyway.

Website projects fall back to single file generation

Visual Studio has both web sites and web applications with the former being project-less leading to very messy multi-file generation so it forces single file generation.

Internal improvements

Source is now simpler to read and understand with less public visibility and faster and more robust VS interop by batching the files & deletes to a single invoke at the end to avoid conflicts with other add-ins that might be triggered by the changes.

Usage

Initialization

You’ll need to get the code into your template – either copy the code in or reference it with an include directive. Then declare an instance of the Manager class passing in some environmental options such as the desired default output path. (For Visual Studio 2010 remove the #v3.5 portion from the language attribute)

<#@ template language="C#v3.5" hostspecific="True"
#><#@include file="Manager.ttinclude"
#><# var manager = Manager.Create(Host, GenerationEnvironment); #>

File blocks

Then add one line before and one line after each block which could be split out into it’s own file passing in what the filename would be if split. The EndBlock is optional if you want it to carry through to the next one :)

<# manager.StartNewFile("Employee.generated.cs"); #>
public class Employee {}
<# manager.EndBlock(); #>

Headers & footers

Many templates need to share a common header/footer for such things as comments or using/import statements or turning on/off warnings. Simply use StartHeader and StartFooter and the blocks will be emitted to the start and end of all split files as well as being left in the original output file.

<# manager.StartHeader(); #>
// Code generated by a template
using System;

<# manager.EndBlock(); #>

Process

At the end of the template call Process to handle splitting the files (true) or not (false). Anything not included in a specific StartNewFile block will remain in the original output file.

<# manager.Process(true); #>

Revised Manager class

Latest source available at GitHub
<#@ assembly name="System.Core"
#><#@ assembly name="System.Data.Linq"
#><#@ assembly name="EnvDTE"
#><#@ assembly name="System.Xml"
#><#@ assembly name="System.Xml.Linq"
#><#@ import namespace="System"
#><#@ import namespace="System.CodeDom"
#><#@ import namespace="System.CodeDom.Compiler"
#><#@ import namespace="System.Collections.Generic"
#><#@ import namespace="System.Data.Linq"
#><#@ import namespace="System.Data.Linq.Mapping"
#><#@ import namespace="System.IO"
#><#@ import namespace="System.Linq"
#><#@ import namespace="System.Reflection"
#><#@ import namespace="System.Text"
#><#@ import namespace="System.Xml.Linq"
#><#@ import namespace="Microsoft.VisualStudio.TextTemplating"
#><#+

// Manager class records the various blocks so it can split them up
class Manager {
    private class Block {
        public String Name;
        public int Start, Length;
    }

    private Block currentBlock;
    private List<Block> files = new List<Block>();
    private Block footer = new Block();
    private Block header = new Block();
    private ITextTemplatingEngineHost host;
    private StringBuilder template;
    protected List<String> generatedFileNames = new List<String>();

    public static Manager Create(ITextTemplatingEngineHost host, StringBuilder template) {
        return (host is IServiceProvider) ? new VSManager(host, template) : new Manager(host, template);
    }

    public void StartNewFile(String name) {
        if (name == null)
            throw new ArgumentNullException("name");
        CurrentBlock = new Block { Name = name };
    }

    public void StartFooter() {
        CurrentBlock = footer;
    }

    public void StartHeader() {
        CurrentBlock = header;
    }

    public void EndBlock() {
        if (CurrentBlock == null)
            return;
        CurrentBlock.Length = template.Length - CurrentBlock.Start;
        if (CurrentBlock != header && CurrentBlock != footer)
            files.Add(CurrentBlock);
        currentBlock = null;
    }

    public virtual void Process(bool split) {
        if (split) {
            EndBlock();
            String headerText = template.ToString(header.Start, header.Length);
            String footerText = template.ToString(footer.Start, footer.Length);
            String outputPath = Path.GetDirectoryName(host.TemplateFile);
            files.Reverse();
            foreach(Block block in files) {
                String fileName = Path.Combine(outputPath, block.Name);
                String content = headerText + template.ToString(block.Start, block.Length) + footerText;
                generatedFileNames.Add(fileName);
                CreateFile(fileName, content);
                template.Remove(block.Start, block.Length);
            }
        }
    }

    protected virtual void CreateFile(String fileName, String content) {
        if (IsFileContentDifferent(fileName, content))
            File.WriteAllText(fileName, content);
    }

    public virtual String GetCustomToolNamespace(String fileName) {
        return null;
    }

    public virtual String DefaultProjectNamespace {
        get { return null; }
    }

    protected bool IsFileContentDifferent(String fileName, String newContent) {
        return !(File.Exists(fileName) && File.ReadAllText(fileName) == newContent);
    }

    private Manager(ITextTemplatingEngineHost host, StringBuilder template) {
        this.host = host;
        this.template = template;
    }

    private Block CurrentBlock {
        get { return currentBlock; }
        set {
            if (CurrentBlock != null)
                EndBlock();
            if (value != null)
                value.Start = template.Length;
            currentBlock = value;
        }
    }

    private class VSManager: Manager {
        private EnvDTE.ProjectItem templateProjectItem;
        private EnvDTE.DTE dte;
        private Action<String> checkOutAction;
        private Action<IEnumerable<String>> projectSyncAction;

        public override String DefaultProjectNamespace {
            get {
                return templateProjectItem.ContainingProject.Properties.Item("DefaultNamespace").Value.ToString();
            }
        }

        public override String GetCustomToolNamespace(string fileName) {
            return dte.Solution.FindProjectItem(fileName).Properties.Item("CustomToolNamespace").Value.ToString();
        }

        public override void Process(bool split) {
            if (templateProjectItem.ProjectItems == null)
                return;
            base.Process(split);
            projectSyncAction.EndInvoke(projectSyncAction.BeginInvoke(generatedFileNames, null, null));
        }

        protected override void CreateFile(String fileName, String content) {
            if (IsFileContentDifferent(fileName, content)) {
                CheckoutFileIfRequired(fileName);
                File.WriteAllText(fileName, content);
            }
        }

        internal VSManager(ITextTemplatingEngineHost host, StringBuilder template)
            : base(host, template) {
            var hostServiceProvider = (IServiceProvider) host;
            if (hostServiceProvider == null)
                throw new ArgumentNullException("Could not obtain IServiceProvider");
            dte = (EnvDTE.DTE) hostServiceProvider.GetService(typeof(EnvDTE.DTE));
            if (dte == null)
                throw new ArgumentNullException("Could not obtain DTE from host");
            templateProjectItem = dte.Solution.FindProjectItem(host.TemplateFile);
            checkOutAction = (String fileName) => dte.SourceControl.CheckOutItem(fileName);
            projectSyncAction = (IEnumerable<String> keepFileNames) => ProjectSync(templateProjectItem, keepFileNames);
        }

        private static void ProjectSync(EnvDTE.ProjectItem templateProjectItem, IEnumerable<String> keepFileNames) {
            var keepFileNameSet = new HashSet<String>(keepFileNames);
            var projectFiles = new Dictionary<String, EnvDTE.ProjectItem>();
            var originalFilePrefix = Path.GetFileNameWithoutExtension(templateProjectItem.get_FileNames(0)) + ".";
            foreach(EnvDTE.ProjectItem projectItem in templateProjectItem.ProjectItems)
                projectFiles.Add(projectItem.get_FileNames(0), projectItem);

            // Remove unused items from the project
            foreach(var pair in projectFiles)
                if (!keepFileNames.Contains(pair.Key) && !(Path.GetFileNameWithoutExtension(pair.Key) + ".").StartsWith(originalFilePrefix))
                    pair.Value.Delete();

            // Add missing files to the project
            foreach(String fileName in keepFileNameSet)
                if (!projectFiles.ContainsKey(fileName))
                    templateProjectItem.ProjectItems.AddFromFile(fileName);
        }

        private void CheckoutFileIfRequired(String fileName) {
            var sc = dte.SourceControl;
            if (sc != null && sc.IsItemUnderSCC(fileName) && !sc.IsItemCheckedOut(fileName))
                checkOutAction.EndInvoke(checkOutAction.BeginInvoke(fileName, null, null));
        }
    }
} #>

[)amien

32 responses

  1. Avatar for Johnk

    Have tried this now and it is so much easier than I thought. Thanks

    Johnk 25 November 2009
  2. Avatar for marius

    uff! Differences Between Project and Item Templates...

    public class Class1(){}

    public class Class1(){}

    Somehow an overload or set that do not depend(too much) on ProjectItems

    // :( maybe next version(s)

    I like to load an xml file with XElement then iterate the nodes and attributes and generate a slim model for whatever... but, i like a xsd to check the xml file at the moment somebody writes some funny business. However i also like to asure that the schema is placed somewhere expected but the xx.cs files all together in the xx.tt tree...

    hmm!.... and by the way!... if i choose split, and i do, why is a xx.cs from xx.tt still there and how can i prevent it. Even if the xx.cs is an empty file

    Those are just some loosy brainwaves...

    I DO LIKE THE MANAGER CLASS

    ! THANK YOU !

    marius 27 December 2009
  3. Avatar for Tim Fischer (tangible)

    Hi Damien,

    great thank you very much for this update. I do use your class a lot and it works like a breeze!

    Tim

    Tim Fischer (tangible) 3 January 2010
  4. Avatar for Marius van Belkum

    Hi There,

    This works great!

    As per other Marius' question. How do i prevent the xx.cs file to be generated?

    Marius van Belkum 22 February 2010
  5. Avatar for Damien Guard

    You can't - VS's SingleFileGenerator that T4 is based on insists on creating the original file.

    The best you could hope to do would be to turn this into a .log or something by changing the file extension and outputting some unimportant information.

    Damien Guard 22 February 2010
  6. Avatar for Sky

    Hey D, Thanks for the code. Your listing is missing an end tag. Might confuse/frustrate some....

    Sky 27 February 2010
  7. Avatar for Luis Rocha

    Hi Damien, thanks a lot for providing this template!!

    I've been using your template since the first version, and now I started using it for the Chinook Database to generate multiple sql and batch files. I noticed that the Manager class generates all files with UTF-8 encoding only. In my case, I need to created sql files with Unicode encoding, and batch files with ASCII encoding.

    I changed your original template and added an Encoding attribute to the Block class, and overloaded the StartNewFile method:

    public void StartNewFile(String name) {
        StartNewFile(name, Encoding.UTF8);
    }
    
    public void StartNewFile(String name, Encoding encoding) {
        if (name == null)
            throw new ArgumentNullException("name");
        CurrentBlock = new Block { Name = name, Encoding = encoding };
    

    Then, I used the block encoding in all calls to File.WriteAllText and to File.ReadAllText (to regenerate a file if its encoding has changed). This way we can control the encoding of the generated files. The modified version I am using is located here, just in case you want to integrate it.

    Thanks again for this template!

    Cheers, Luis

    Luis Rocha 16 November 2010
  8. Avatar for Tim

    Damien, Thanks for the help! Big time saver, but I have a really gnawing issue with it. How come when I add to the header AFTER adding a normal block, the header information appears inline and not with the rest of the header? It would make better sense (to me), to output all header information inside the header, and not piecewise, throughout the file (this is only an issue when outputting a single file). Is this by design? From my app's POV its a bug. Thoughts?

    Tim 7 December 2010
  9. Avatar for Damien Guard

    Are you building the header in a single go after a normal block or attempting to build the header in several different blocks?

    Damien Guard 7 December 2010
  10. Avatar for Tim

    ...Attempting to build the header in several different blocks...

    Tim 7 December 2010
  11. Avatar for Damien Guard

    Yes, this template does not supports building one block in multiple places and I think it would add complexity to it that most people don't need.

    Damien Guard 9 December 2010
  12. Avatar for Tim

    Cool. Thanks for the feedback.. My code just has an extra section that preprocesses the header now. A little less efficient -- but its an offline process anyway :) The mileage we get from the outputted code is immense. Thanks!

    Tim 14 December 2010
  13. Avatar for Paul

    Hi Damien, you seem to know your T4 stuff - very handy include.

    A quick question; I'm generating a WCF service (the .svc and .cs file) via T4, everything works fine but for some reason Visual Studio separates the .svc and the .cs file - is there a way to make the .cs file be a shadow of the .svc as it usually does if you create the service manually?

    Regards, Paul.

    Paul 8 September 2011
  14. Avatar for Damien Guard

    There's nothing the template can do automatically as far as I know but you can manually edit the .csproj file and make one DependentUpon the other which will visually stack them in the Project Explorer.

    Damien Guard 8 September 2011
  15. Avatar for Bill Huth

    Just an FYI,

    If the T4Toolbox is included ( ), the Manager produces no files. I think it is in the dispose method of the surrounding class...but I could not track it down.

    Bill

    Bill Huth 3 November 2011
  16. Avatar for Nestor

    How can I delete the original generation? Say that I keep all my output within StartNewFile and EndBlock and I don't want the default file generated? only my blocks. For intance, example.tt would generate example_model.cs and example_viewmodel.cs, but NOT example.cs

    Thanks a lot!

    Nestor 6 July 2012
  17. Avatar for Damien Guard

    There's no way to suppress the default output or reliably delete it (it is often locked).

    I find the best option is to just have it list all the outputs it created and set the output extension to .log

    Damien Guard 6 July 2012
  18. Avatar for Júlio Nobre

    I have just used your approach in order to achieve full class conditional generation. Your class Manager is a really neat and usefull piece of code.

    Thanks a lot for sharing it!

    Júlio Nobre 11 August 2012
  19. Avatar for Lars Ole

    Thanks for sharing this really neat piece of code. With t4toolbox seemingly unsupported, this is a nice way to get no-hassle multifile t4 support in VS2012.

    Lars Ole 16 November 2012
  20. Avatar for Lars Ole

    .. And also: I can't believe three years on from the posting of this article, it remains relevant ..

    Lars Ole 17 November 2012
  21. Avatar for Robert

    Hi Damien, Thx for this nice and clean solution (IMHO this should be part of the VS t4 out of the box experience). I used it in VS 2012. After enabling build support (with this guideline http://www.olegsych.com/2010/04/understanding-t4-msbuild-integration/ ) I found that the line in the factory method:

    return (host is IServiceProvider) ? new VSManager(host, template) : new Manager(host, template);
    

    does not detect the host being MSBuild. It creates a VSManager and there it gets a NPE when trying to obtain the DTE. So I changed this code to:

    if ((host != null) &amp;&amp; (host is IServiceProvider)) {
        var dte = ((IServiceProvider)host).GetService(typeof(EnvDTE.DTE));
        if (dte != null) return new VSManager(host, template);
    }
    return new FileManager(host, template);
    

    and everything works as expected.

    Thx again and regards Robert

    Robert 17 December 2012
  22. Avatar for John

    Having problem with multi-file updates... for some reason the source control IsItemUnderSCC returns false when should be true. I'm using the Perforce V11 PlugIn for source control.

    John 15 March 2013
  23. Avatar for Damien Guard

    Hmm, isItemUnderSCC is a function of the DTE object (Visual Studio) that goes down into the source provider. It sounds like the Perforce plug-in isn't implementing it. You might be able to work around it by having it always try and check the file out.

    Damien Guard 15 March 2013
  24. Avatar for leconehu
    var manager = Manager.Create(Host, GenerationEnvironment);
    

    There is no name of the current context "Host" why?

    leconehu 8 July 2013
  25. Avatar for Richard Collette

    I believe a bug exists in IsFileContentDifferent. Shouldn't the code be?

    protected bool IsFileContentDifferent(string fileName, string newContent) {
        return !File.Exists(fileName) || File.ReadAllText(fileName) != newContent;
    }
    

    It would be a nice feature if the code would remove files that are no longer generated by the template.

    Richard Collette 29 July 2013
  26. Avatar for Damien Guard

    You have !a || !b which is logically the same as what is there of !(a && b)

    To be able to delete files that are no longer there it would need to record a list of them...

    Damien Guard 29 July 2013
  27. Avatar for Stephane Issartel

    I read many solutions/documentation to generate multiple outputs from a single T4 template (including in-depth Oleg Sych's articles). Regarding multiple outputs file generation I truly love your solution.

    Easy to use with neat/lightweighted code in the background (very easy to understand and extend).

    Thanks a lot.

    Stephane Issartel 6 December 2013
  28. Avatar for Kaul

    Damien, this is a super useful piece of code. One problem though: because of its reliance on EnvDTE, it only runs on machines that have visual studio installed. On Machines without visual studio, it is possible to deploy all the T4 dlls: //msdn.microsoft.com/en-us/library/ee847423.aspx#buildserver but not the EnvDTE dll.

    Now that T4 supports build time code generation, would it be possible to use Microsoft.VisualStudio.TextTemplating.Sdk.Host to get rid of the EnvDTE dependency?

    Kaul 2 December 2014
  29. Avatar for Phani

    Thank you very much!! Works like a charm in VS2012 at work.

    But not in VS2015 community edition. I can live with this!

    Phani 31 March 2016
  30. Avatar for Malachite

    I Tried many solutions and this one is the best of all!

    Dont forget that the files it generates are not included in the project automagically, they remain invisible.

    Malachite 31 May 2016
  31. Avatar for Thao

    I set hostspecific="true" and it let me pass that error.

    Thao 7 October 2016
  32. Avatar for tgiphil

    I can't get this to work with .NET standard projects. Anyone have any luck?

    tgiphil 16 June 2018