DSL to JSON Conversion SDK

[dblclik]

Currently, the only method for converting DSL to JSON is via the Syntax Transformer. While this works nicely for front end environments like the Playground, it requires teams either package the ts methods as CLI or host the transformers as APIs and write glue code.

A colleague and I are considering writing a library to make this simple to do directly in Rust, Go, Python, and JS (with other languages easy to add and would welcome collaboration). Does anyone else feel like this is a tool they'd benefit from? Having a custom low-level parser, AST, and converter could also make it easier to support DSL in the engine if we choose, and enable more tools to be built around the DSL w/o requiring integrating the typescript transformer library.

[maxmindlin]

We took a first pass at this and have something usable to show. This repo currently parses DSL document strings into an AST and then provides a transformer for serializing to JSON. It seems to work for many cases, as can be seen in unit and integration tests. It currently does limited syntax checking - it really only throws parser errors related to an inability to build the AST.

I also threw together a Python binding library that simply wraps the former library. It is installed via Pypi with no install requirements other than Python + pip on Linux, Windows, and MacOS. The neat part of this approach is that binding libraries like this (any language that can communicate via C FFI) can share the same core library while still having a native API.

If anybody has thoughts or suggestions please let me know!

[jon-whit]

@dblclik @maxmindlin this is awesome! Thanks for the support on this. We've been wanting to bring more native DSL to JSON transformation to clients across various languages. In the past we had discussed making a DSL to JSON transformer as a WASM module so that it's usable across a variety of languages (any of which that support the WASM runtime). This would allow us to use the WASM module in the browser for the Playground but also use it in OpenFGA (Go) directly.

Have you guys considered this approach at all? I think this would make it more cross-platform consumable. WDYT?

[jon-whit]

Thank you! It's awesome you're working on this, because this is something that's been on our radar for a hot minute but just haven't had time to get around to it quite yet. Thanks for the support 🚀

[dblclik]

@jon-whit - finally, someone who can relate with me to how awful Cython was lol

To be clear, I'm actually proposing that we offer both. Creating a WASM target and/or integrating the libraries that we're building into a WASM binary should be fairly trivial, but providing C FFI integration with all of the language targets we can would be less trivial (err, not trivial lol) if we started from the WASM side.

[dblclik]

Hey @jon-whit - We've gotten a WASM binary that builds from our Rust library HERE, and I've created a pre-production release with the WASM binary included if you don't want to set up the rust toolchain to build your own (LINK). We've tested this with JS and Go and it seems to work well 🤞 🎉

If you check out the Go example I've created, you can see how I'm currently using the WASM binary w/ wasmer. Let us know what you think!

[jon-whit]

@dblclik awesome! I'll take a look at your work this week and run it by the team and get back with you. Thanks 👍 !

[aaguiarz]

Hi all!

I was thinking about the use case for this.

Let's assume we have a CLI that can accept the model, validate it, transform it to JSON, and potentially post it to OpenFGA.

Why would you need to have the DSL->JSON transformation on each SDK?

[aaguiarz]

The CLI has now full support for transforming models.

There are also modules for Go and Javascript in the Language repository to implement transformations in code. Java support is coming.