Expressions DSL in Scala Subscribe Pub Share

I often seem to have a need for an expression DSL, typically for configuring this or that component, implementing rules here and there etc. So, I've been working on an external DSL, for expressions, in scala, for a while, something that can put up with a few types of expressions, like lambdas, embedded Javascript etc...

Here are some examples of the expressions you can parse with it:

  // arrays, lambdas, map, filter:
  a284 = [1,2] + [3] filter (x=> x > 1) map (x=> x + 1),

  // embedded Javascript expressions
  res40=js:email.replace(/(\w+)@(\w+).com/, "$1") ,
  
  later=js:{var d = new Date(); d.setSeconds(d.getSeconds() + 10); d.toISOString();} ,

  // json/object operations
  cart=cart || {id:customer, items:[] } ,
  cart=cart+{items:{sku : sku, quantity : quantity} } ,
  
  // more embedded Javascript expressions
  simpleJs = js:wix.diesel.env ,
  res78=js:cart.items[0].sku 

  constants = 321, 

  addition = last+first,

  jsonBlock={
    "accountNumber": "1729534",
    "start_num": 123
	},

  qualified = jsonBlock.accountNumber,

  // string interpolation:
  interpolation="is${1+2}",
  url="${HOST}/search?q=dieselapps",

  builtInFunctions1 = sizeOf(x=cart.items),

  builtInFunctions2 = typeOf(x=cart.items),
  
  externalFunctions = pack.obj.inst.func(x=cart.items),
  
  anArray = [1,"a",3],
  
  aRange = anArray[1..]

As you can see, it includes elements found in several languages, expressions I deemed interesting, like ranges etc:

• JSON objects supported directly
• operations on arrays and objects
• interpolates strings by default
• ranges and lambdas
• support embedded expressions in js and other languages - you can extend with more
  • we use the Nashorn in jdk 1.8 to run these
• named parameters for calls, not positional (some shortcuts exist though)
• type-aware expressions

You can see a lot of examples of expressions and the unit tests in expr-story

I am sure that a purist would scoff at many elements of this mix, but I find it intuitive and "good enough"... and I didn't even get into the XPath support!

One other thing to note is that it doesn't really have anything for asynchronous execution like streams or such - this is meant only as an expression language and it is complemented by the Reactive rules workflow DSL in Scala⁺⁺, which is all about rules-driven reactive message decomposition.

Using it

It is open-sourced here: diesel/expr and you can use it today, if your project's configuration requires expressions or if you need expressions in any other way.

It is all written in scala and all you need is yet-another-maven-dependency... here's an example client code:

import razie.diesel.expr._
import razie.diesel.dom.RDOM._

val input = "a + 2"
val parser = new SimpleExprParser()
// create and initialize context with "a"
val ctx: ECtx = new SimpleECtx().withP(P.fromTypedValue("a", 1))

// parse and execute any expression
val result = parser.parseExpr(input).map(_.applyTyped("")(new SimpleECtx()))

You would create your own context, containing all the values and variables that the script should have access to and then let it rip!

A sample project that uses the expressions, is in expr1 - see the build.sbt etc.

Parser details

The parser uses the scala combinator parsers, not the best parser library out there, but pretty good and feature-filled.

Basics

The basics of parsing operators, with associativity and precedence, are well known, here's the usual:

  def opsPLUS: Parser[String] = "+" | "-"
  def opsMULT: Parser[String] = "*" | "/"

  // x + y
  def exprPLUS: Parser[Expr] = exprMULT ~ rep(ows ~> opsPLUS ~ ows ~ exprMULT) ^^ {
    case a ~ l => l.foldLeft(a)((x, y) =>
      y match {
        case op ~ _ ~ p => AExpr2(x, op, p)
      }
    )
  }

  // x * y
  def exprMULT: Parser[Expr] = pterm1 ~ rep(ows ~> opsMULT ~ ows ~ pterm1) ^^ {
    case a ~ l => l.foldLeft(a)((x, y) =>
      y match {
        case op ~ _ ~ p => AExpr2(x, op, p)
      }
    )
  }

We foldleft over the repetition a+b+c to make the operation left-associative and parse PLUS before MULT, to ensure precedence.

To avoid too much copy/paste, we can refactor it a little bit:

  // foldLeft associative expressions
  private def foldAssocAexpr2(
    a:Expr, 
    l:List[String ~ Option[String] ~ Expr], 
    f:(Expr, String, Expr) => Expr) = {
    
    l.foldLeft(a)((x, y) =>
      y match {
        case op ~ _ ~ p => f(x, op, p)
      }
    )
  }

  // x + y
  def exprPLUS: Parser[Expr] = exprMULT ~ rep(ows ~> opsPLUS ~ ows ~ exprMULT) ^^ {
    case a ~ l => foldAssocAexpr2(a, l, AExpr2)
  }

  // x * y
  def exprMULT: Parser[Expr] = pterm1 ~ rep(ows ~> opsMULT ~ ows ~ pterm1) ^^ {
    case a ~ l => foldAssocAexpr2(a, l, AExpr2)
  }

We extend this further, to get conditionals and then map/filter etc:

  //  def expr: Parser[Expr] = ppexpr | cond | pterm1
  def expr:  Parser[Expr] = exprAS | pterm1

  def opsas: Parser[String] = "as"
  def opsmaps: Parser[String] = "map" | "filter"
  def opsOR: Parser[String] = "or" | "xor"
  def opsAND: Parser[String] = "and"
  def opsCMP: Parser[String] = ">" | "<" | ">=" | "<=" | "==" | "!=" | "~="
  def opsPLUS: Parser[String] = "+" | "-" | "||" | "|"
  def opsMULT: Parser[String] = "*" | "/"

  // "1" as number
  def exprAS: Parser[Expr] = exprMAP ~ opt(ows ~> opsas ~ ows ~ pterm1) ^^ {
    case a ~ None => a
    case a ~ Some(op ~ _ ~ p) => AExpr2(a, op, p)
  }

  // x map (x => x+1)
  def exprMAP: Parser[Expr] = exprPLUS ~ rep(ows ~> opsmaps ~ ows ~ exprCMP) ^^ {
    case a ~ l => foldAssocAexpr2(a, l, AExpr2)
  }

  // x > y
  def exprCMP: Parser[Expr] = exprPLUS ~ rep(ows ~> opsCMP ~ ows ~ exprPLUS) ^^ {
    case a ~ l => foldAssocAexpr2(a, l, cmp)
  }

Note the difference between AExpr2, which constructs arithmetic expressions and cmp which constructs boolean expressions.

And then, there are the conditionals, the various pterm1 like lambdas, JSON objects etc - you can see them all in the ExprParser. I have left out parsing or and and - you can try your hand at it, if you feel like it...

Have fun with it!

---

See more in: Cool Scala Subscribe You need to log in to post a comment!

---