Posts Tagged HStringTemplate

ANN: HStringTemplate 0.3.1

This release of HStringTemplate (up now at Hackage) fixes a number of bugs pointed out to me by its small but growing user base (thanks, cinema, elliottt!) ranging from the minor (a particular two-level iteration pattern wasn’t working properly) to the truly irritating (poor handling of file groups). It’s still unfortunately skimpy on the docs, outside of the haddocks and the main StringTemplate grammar documentation at http://www.stringtemplate.org (although the examples from my new project [coming soon!] should also prove helpful). However, it does have a set of very nice and handy new features for development.

* renderf, a function similar in spirit to printf, that takes an arbitrary number of heterogeneous (String, value) tuples as arguments. This should cut down considerably on long setAttribute chains. Additionally, with custom instances (not, I’ll grant, trivial to write) it can be used to declaratively chain together strings of attribute retrieval functions in arbitrary monads, as in the above code example from hvac.

* dumpAttribs, a function/template that prints out the tree of the entire attribute environment a template is operating in — extremely handy for development.

* nullGroup, also for use in development, a simple way to display more information about templates that can’t be found. Error messages in usafeVolatileDirectoryGroup have also been significantly improved.

* getStringTemplate’, a version of getStringTemplate guaranteed not to be inlined. While the optimizer will still sometimes rearrange code such that a volatile group is not updated properly, this at least helps remedy the situation (I think).

* Some minor changes: For grammar reasons, dots have been removed from template names — however, underscores and slashes are now available. Additionally, there’s a much improved logic for which aspects of a local environment are overridden and preserved when a template is called from another.

Comments (2)

On Monoids

So HStringTemplate was built around a simple notion: Everything takes an environment, which is a data structure holding attributes and some other good stuff, and eventually returns a string. This was abstracted later on, but we’ll come back to that. Plan text? That returns a string. A reference to an attribute? It returns a string. A nested template? It returns a string. So when we parse/interpret any template, we end up with a list of functions of type Env -> String. Simple enough to get going with.

So how do we execute this template? We need a function of type [Env -> String] -> Env -> String. If we weren’t thinking functionally, we’d have to write a loop. Create an empty string, and for each function in the list, apply it to our environment, append it to the string, and repeat:

function render(template, env) {
	str = "";
	foreach (func in template) {
		str += func(env);
	}
	return str;
}

In functional terms we can write (ignoring stack and efficiency issues) a fold:
render tmp env = foldr (flip (++) . ($ env)) [] tmp

If we think about it a bit, this is actually concat $ map ($env) tmp, or simpler still concatMap ($env) tmp

Step back from the function signature again, and think of it one type at a time and we get an additional abstraction. Instead of ([Env -> String], Env) -> String. we can view it as [Env -> String] -> (Env -> String). But our code doesn’t really represent this, as it explicitly asks for the env parameter. But then again, we could rewrite it as render tmp = \env -> concatMap ($env) tmp, and then pointfreeing the lambda expression we get (tmp >>=) . flip ($) or (concatMap tmp) . flip ($) depending if we want the in this case useless obfuscation of using the list monad.

Good enough, right? Well, imagine that we’ve got our perfect-abstractionometer turned really high for some reason today. There’s a flip and a ($). It looks a bit ugly. It looks a bit obscure. We’ve got compression sure, but not necessarily any more clarity than when we started with. Well, instead of starting from the imperative mindset of how we want this function to work, let’s look at the type signature again and start thinking about what could produce what we’re describing. First, let’s desugar String back to its underlying type, [Char]. That gives us [Env -> [Char]] -> Env -> [Char]. Now since we’re thinking of Env and Char here as abstract atomic types, we can ask, what gives us [a -> [b]] -> a -> [b]? Well, this might be a job for the unwrapped reader monad, since the sequence operation (Monad m => [m a] -> m [a]) resembles what we want. Substitute in (e->) for the m, and we get [e -> a] -> e -> [a], which in our case becomes [Env -> [Char]] -> Env -> [[Char]]. Almost there! Throw in a concat and we get: (concat .) . sequence of type [a -> [b]] -> a -> [b]. And now, not only is our code short, pointfree, and elegant, but it expresses what we wanted all along more clearly than any previous construct.

And yet.. and yet… some nagging bit of our brain says that this isn’t a compound operation, but an elementary one, for a proper algebra. The only question is what that algebra is. Enter monoids. The Data.Monoid library isn’t extensively documented, but the types almost explain themselves. A monoid is about the simplest algebraic structure that one can imagine. In category theory, a monoid can be viewed as a category with one object.

Ok yeah, but what is it? A monoid is a set of elements, an associative binary operator between them, and an identity element. So, you might have a monoid (Integers,+,0) because + is associative between all integers, and X + 0 = X. Integers also give rise to another monoid: (Integers,*,1). And indeed the Sum and Product monoids are both defined in Data.Monoid. We also get other simple ones for booleans. And, of course, we get, for all a, ([a],++,[]). In fact, the list monoid is also known as the free monoid, essentially because, as best I (sorta) understand it, it comes without any constraints. I.e. if for the integers we have [1,4,2,3] we have “1423” (from the monoid on integers derived by turning them to strings and appending them) and we have 24 from the product monoid and we have 10 from the sum monoid and etc. Thus, the free monoid keeps all information that goes into it, and therefore, so to speak freely arises (without constraints) for any set of objects.

What does all this get us? Well, it gets us generality, especially when we throw higher-order functions back in the mix. In particular, we can now notice that Data.Monoid gives us, for instance the monoid of endomorphisms. So we can conceptually “add” (or rather, mappend), for example, a whole set of functions of type “String -> String” into one big “String -> String” function.

Well, that looks helpful, but it’s not exactly what we want. There is, however, an even cooler Monoid: The reader monoid. Monoid b => Monoid (a -> b). Conceptually, there’s only one way its mappend function can work: feed the value a into two functions, yielding two bs. Now, given that these bs are, by definition, Monoids as well, we mappend their results.

Along with mappend, which is the associative element, we of course got mconcat for free, which is defined by default as foldr mappend mempty, and as such is the obvious generalization of concat. So now we can look at the type of mconcat specialized to the reader monoid: Monoid b => [a -> b] -> a -> b. And that, ladies and gentlemen, is what we’ve been looking for all along. (concat .) . sequence becomes simply mconcat! Without a lick of coding on our part.

Oh yeah, here’s the icing: we get an even more general type signature. The function as we initially defined it worked for Strings, and by extension, other lists. But it didn’t work for values of, for example, type ShowS (i.e. String -> String, which is a common hack to avoid stack issues with concatenation by encoding strings as functions). But look! Even if ShowS isn’t a list, it is, after all, an endomorphism! And we know by our discussion above endomorphisms are monoids too. And you know what else is a monoid? Oh yeah, a bytestring! And soforth.

I eventually took this idea even further in the HStringTemplate library, and also found another nice use of monoids that I’ll blog about later. The lesson I got out of this though: sometimes, it turns out the simplest abstractions are the most powerful.

Comments (1)

ANN: HStringTemplate 0.2

HStringTemplate is a general purpose templating system, geared especially towards HTML and based on Terrence Parr’s Java library.

On Hackage at: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/HStringTemplate-0.2

Development version at: darcs get http://code.haskell.org/HStringTemplate/

Haddocks at: http://code.haskell.org/HStringTemplate/dist/doc/html/HStringTemplate/Text-StringTemplate.html

Additional documentation on the grammar at: http://www.antlr.org/wiki/display/ST/StringTemplate+3.1+Documentation as well as in some posts at https://fmapfixreturn.wordpress.com

Lots of cleanup, lots of additions, hopefully this is an extremely usable release. Still no group or interface files, but I suspect that those are mainly useful for code-generation, which seems like something Haskell programmers would want to use something other than a templating system for in any case.

On to the good stuff:

  • Now on hackage!
  • Generics. Not one but two types. One set of simple bindings for the standard Data class, and one for syb-with-class. (Alex Drummond’s RJson library, which is really cool, was very helpful in figuring out how to do this).
  • A withContext method that turns any set of name-value bindings into the context for a StringTemplate. Along with the syb-with-class bindings, this should make for relatively seamless interoperability with HAppS.
  • Lots of other additional bindings for working with standard time formats, numeric types, etc.
  • Encoders. A standard mechanism for HTML-escaping strings (or javascript-escaping, or urlencoding them, or etc.) that A) ensures it happens uniformly and B) ensures it happens no more than once.
  • Improved pretty printing support, eliminating corner-cases where wrapping did not occur.
  • Creation of directory groups is now done properly in the IO monad, with caching functionality moved to the scarily-named unsafeVolatileDirectoryGroup.
  • 80% Top-Level testing coverage in the base. (And more to come, but I only have so much time).

And of course, patches and feedback always welcome.

Leave a Comment

Turtles all the way.

HStringTemplate had a very irritating hack in it up until about twenty minutes ago. Every time we worked with some of StringTemplate’s fancier constructs, particularly chaining template application (i.e. $foo:bar():baz():etc()$ which applies foo to bar, feeds it to baz, then feeds it in turn to etc) or using a template as an attribute (i.e. ${$foo$}$ which creates an anonymous template, that in turn grabs foo from the outer scope) the value had to be rendered down to a string before passing it back into the next scope. Now, this normally would have been unfortunately ugly, but not that harmful. However, because of the wrapping and indentation support that HStringTemplate provides, it meant that we couldn’t get proper indentation in such a case since we didn’t know where to wrap the expression yet (i.e. how far it would be indented) at the time we evaluated it.

That last ugliness is now gone. The approach is genuinely turtles all the way down, as the saying goes. This not only makes things work right, but it makes the entire data model cleaner and more efficient. And, while there were a number of changes to type signatures to pass the right information around (sometimes I feel type signatures are the Dorian Gray’s portrait of the Haskell world — the more elegant your code gets, the more complicated and unwieldy your signatures), the end result required only three changes to lines of actual code, and actually resulted in something slightly smaller!

I hope to get to this in a later blog post, but the lesson is that static typing and type inference aren’t just about bondage and discipline and safety. They can also give you a sort of incredible expressive power, where the information on what you’re doing is carried around invisibly in the type system as a sort of secret parameter that can give you a great deal of control over the way your code actually behaves. While the Java version of StringTemplate, for example, uses separate renderers that control how output displays, everything in HStringTemplate (lexing, parsing, construction of functions, establishing pretty-printing relationships) can be done in a single pass. And thanks to laziness and higher-order functions, this pass can produce partially applied functions where everything else (such as indentation levels) can be figured out in a second single pass at runtime. (NB: this idiom, completely natural to Haskell, is sometimes known as runtime compilation although it’s a different use of the term than is usually the case.)

So what does that mean to the end user? Well, you can do this, for one:


> putStr $ PP.renderStyle (PP.style {PP.lineLength=80}) $ toPPDoc $
  setAttribute "foo" [[(1::Double)..30],[200..230]] $ newSTMP
  "Some long text: $foo:{Some label: [$it$]}:{#$it$#};separator=', '$ END"

Some long text: #Some label: [1.0, 2.0, 3.0, 4.0, 
                              5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 
                              14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 
                              22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.0, 29.0, 
                              30.0]
                 Some label: [200.0, 201.0, 202.0, 203.0, 204.0, 
                              205.0, 206.0, 207.0, 208.0, 209.0, 210.0, 211.0, 
                              212.0, 213.0, 214.0, 215.0, 216.0, 217.0, 218.0, 
                              219.0, 220.0, 221.0, 222.0, 223.0, 224.0, 225.0, 
                              226.0, 227.0, 228.0, 229.0, 230.0]# END

So what’s happening with this expression? The newSTMP call, as usual, parses a string into a template. The setAttribute call inserts, “foo”, a list of two lists of doubles, into the template. The toPPDoc call evaluates the template in the context of the attributes we’ve set, and returns a Pretty Printer Doc. The render call is to the Text.PrettyPrint.HughesPJ library (which I’ve imported, qualified as PP), and tells it how long we want the lines to be, and finally, the putStr call displays the returned string with newline characters appropriately rendered.

Now onto the mechanics of the StringTemplate itself. First we print some text, and then foo, as a list, is iterated over and repeatedly passed into the next part of the expression. The sublists of foo are each printed, preceded by “Some label: ” and surrounded in braces. The values of each sublist are separated by commas, because we set that option in the top level expression, and that gets propagated downwards as well, and then the whole resulting expression gets wrapped in pound signs by the final template application. And the new cool part is, because everything is “turtles all the way,” those sublists are not rendered and wrapped until the entire shebang is called, meaning that not only do we preserve multiple levels of indentation, but that when those sublists are displayed, everything, including the pound signs that were only inserted *after* we evaluated the sublists of foo, is taken into account such that they indent and wrap properly.

The beauty of programming in a language that supports abstractions as elegantly as Haskell does is that doing it this way not only gives nicer performance, and nicer features, but that it creates cleaner, simpler code. Implementing this feature properly didn’t mean fighting the language, but just giving into to what the types had been whispering to me all along.

Comments (1)

More Simple HStringTemplate Examples

Today I pushed a few changes to the repo so that it builds properly with GHC 6.8. Beyond that, I realized, with a little prompting, that I foolishly hadn’t exported a decent function to query groups of StringTemplates. Additionally, ac on #haskell (the friendliest programming IRC channel this side of the millennium) asked for a simple function that generalized setAttribute to set a whole bunch of attributes at once. The following simple example shows the use of groups, the setManyAttrib function, and how to globally insert options for a single or multiple templates to boot.

> let foo = newSTMP "foo" :: StringTemplate String
> let bar = newSTMP "bar $foo()$" :: StringTemplate String
> toString foo
"foo"
> toString bar
"bar No Template Found for: foo"
> let grp = groupStringTemplates [("foo",foo),("bar",bar)]
> toString <$> getStringTemplate "foo" grp
Just "foo"
> toString <$> getStringTemplate "fbar" grp
Nothing
> toString <$> getStringTemplate "bar" grp
Just "bar foo"
> let baz = optInsertTmpl [("separator","; ")] $ newSTMP "$a$ $b$ $c$" :: StringTemplate String
> toString $ setManyAttrib [("a","first"),("b","second"),("c","third")] $ baz
"first second third"
> toString $ setAttribute "a" ([1..4]::[Int]) $ baz
"1; 2; 3; 4 "
> let grp2 = groupStringTemplates [("foo",foo),("bar",bar),("baz",baz)]
> fmap toString $ getStringTemplate "baz" . optInsertGroup [("null","nothing")] $ grp2
Just "nothing nothing nothing"

I still haven’t even touched all the nice bits of the StringTemplate grammar, particularly its recursive constructs. But that’s enough for now.

Comments (4)

HStringTemplate: An Elegant, Functional, Nifty Templating Engine for Haskell.

HStringTemplate is a port of Terrence Parr’s lovely StringTemplate engine to Haskell.

It is available, cabalized, at:
darcs get http://code.haskell.org/HStringTemplate/

As interest has grown in using Haskell for web applications, there has been an increasing buzz about the need for a good templating engine in Haskell. Why might we need this? After all, Haskell has lovely combinator libraries for generating HTML programmatically, enforcing good form through its type system. But sometimes, we don’t want well-formed HTML. We want the ugly stuff that floats around to deal with eight varieties of browser incompatibilities and the latest silly ajax trick. Or sometimes we’re working with a team of graphic designers, and they want to work in almost-HTML. Or sometimes we just want to be able to change the design of a web application on the fly, completely independent of our program logic, and of, heavens forbid, recompiling and possibly messing with a live application.

So template engines are popular, and indeed, considered a key part of most web frameworks out there. One problem — they’re mainly awful, imperatively-conceived behemoths that capriciously mix program logic with display and, consequently, entail a great deal of overhead. Enter StringTemplate, a nifty and fairly-well developed template format that’s both pure and functional, and therefore pretty much the only one of its kind. Indeed, it also seems to be getting heavy use in code generation because its paradigm maps neatly to traversing parse-trees.

HStringTemplate is not feature-complete, and indeed is only at version 0.1. But it should implement pretty much everything you’ll find here, only nicer, because it’s in Haskell. There are scads of different recursive constructs and ways to handle inclusion and inheritance. Furthermore, HStringTemplate handles conditionals, and also a very Haskellish implementation of custom rendering.

Templates can be constructed that return strings, ShowSs, bytestrings, or even pretty printer Docs that handle wrapping, indentation, and fill elegantly. Even better, these templates are parsed and compiled only once, after which point there isn’t a syntax tree anymore, just a function that operates on the environment of attributes that have been passed to it.

Ok. Enough talk. Let’s look at a sample GHCi session. Note that when defining “hello” we have to give the signature explicitly, as it’s not clear until later what type of StringTemplate we’re expecting.

> let hello = newSTMP "something $foo;separator='; '$ something" :: StringTemplate String
> toString $ setAttribute "foo" "SomethingElse" hello
"something SomethingElse something"
> toString $ setAttribute "foo" ["a","b","c"] hello
"something a; b; c something"
> toString $ setAttribute "foo" ([1..10]::[Double]) hello
"something 1.0; 2.0; 3.0; 4.0; 5.0; 6.0; 7.0; 8.0; 9.0; 10.0 something"

And that should be enough to get you started! Be sure to check the haddocks as well, which cover the API in some more detail.

Where I take it from here depends in part on what sort of response I get, so patches, gripes, API comments and feature requests are all more than welcome.

Meanwhile, I’ve learned quite a bit doing this, and suspect that once the benchmarks are in, this will in fact turn out to be a fairly lean and mean implementation compared to the Java one. As a matter of fact, it currently stands at, excluding tests, 414 lines of code. The Java version, admittedly somewhat more featureful, runs in the vicinity of 12,000! Over the next few weeks, I plan to blog about some of my “aha” moments, mainly involving monoids and their many delicious instances.

Comments (7)