One of my scala patterns: Instead of incrementally changing a lot of code to carry over new data/objects, just carry over decorator closures and apply them at the end. All in scala.
Afraid of over-engineering gets you into thinking as you type and, when you're dealing with a complicated parser, this leads to many methods... too many to refactor effectively.
So, while writing a wiki parser, I found myself having to keep extending and refactoring as new ideas followed the morning's Starbucks coffee, like... obviously wiki pages should have tags and other special markups - duh!
So, my parsing code before only processed the markup into html while carrying over a list of links, nothing fancy, like:
class WikiPage(val content: String, val links:List[String]=Nil)
object WikiParser extends RegexParsers {
case class State(html: String, links: List[String] = Nil)
def apply(input: String) = parseAll(wiki, input) getOrElse (State("[[ERROR PARSING]]"))
def wiki = rep(link | text) ^^ {
case l => State(l.map(_.html).mkString, l.flatMap(_.links))
}
def link: Parser[State] = "[[" ~ """[^]]*""".r ~ "]]" ^^ {
case "[[" ~ name ~ "]]" => State("""%s""".format(name, name), List(name))
}
}
The link method parses a link which is propagated up and composed in the wiki parser from all the links in the page.
As you see, if I am to now add more properties to be parsed, like tags, the changes would cascade in a few places, to propagate the new attribute from the parser up - and trust me, this parser has 800 lines of code... lots of stuff to change.
Then after tags, maybe I will think of other properties or scripts or stuff to add, so... I just refactored it to collect a list of decorating closures of the type [WikiPage => WikiPage] :
// tags are new
class WikiPage(val content: String, val links:List[String] = Nil, val tags: List[String] = Nil)
object WikiParser extends RegexParsers {
case class State(html: String, links: List[String]=Nil, decorators: List[WikiPage => WikiPage] = Nil) // add decorators
def wiki = rep(link | tag | text) ^^ {
case l => State(l.map(_.html).mkString, l.flatMap(_.links), l.flatMap(_.decorators)) // add decorators
}
// like before
def link: Parser[State] = "[[" ~ """[^]]*""".r ~ "]]" ^^ {
case "[[" ~ name ~ "]]" => State("""%s""".format(name, name), List(name))
}
// new element - collects a decorator to add tags to page later
def tag: Parser[State] = "{{" ~ """[^}]*""".r ~ "}}" ^^ { // add tags
case "{{" ~ name ~ "}}" => State("""%s""".format(name, name), Nil, { we: WikiPage =>
new WikiPage(we.content, we.links, "CoolScala" :: we.tags)
} :: Nil)
}
}
val WikiParser.State(html, links, decorators) = WikiParser(dbWe.content)
// new - after parsing, fold to apply the decorators
val parsedWe = decorators.foldRight(dbWe)(_ apply _)
Here, you see me adding the tags to the WikiPage and the new tag parser, which in turn just adds a new decorator to be collected and applied at the end.
If I am to add more stuff in the future, like... I don't know... headings to be collected and indexed maybe, the change will be isolated to only the added parser.
One of the drawbacks of this approach is that lower-level code like a tag parser needs to know the higher-level structures...
P.S. If you were looking for an embedded, markdown-based Wiki Engine written in scala, I've got one, by the way - you're looking at it :)
Have fun!
As this parser evolved with more and more features, I ended up turning it into a full AST (Abstract Syntax Tree) - so basically the parser only builds this intermediary syntax tree structure, which at the end is "folded" to both collect the resulting HTML and any other transformations.
This AST tree is folded into text/html later, see Parsing Wikis With An AST Tree.