TLDR
We are working on a project that requires us to regularly transform data from one shape to another for our integrations. We decided liquid templates in the Transform connector in Logic Apps/integration accounts seemed like a reasonable solution. We quickly found out the developer experience is terrible.
Be careful, Liquid template syntax in Azure Logic Apps Transform connector varies between the tooling in VS Code. While your transform may work in VS Code, you will often find it doesn't work as expected in Azure.
What did we run into?
The extensions available in VS Code are built on a ruby flavor of liquid templates while the Transform connector is using DotLiquid version 2.0.254 (as of March 11, 2019). There are differences in syntax and capabilities.
https://github.com/dotliquid/dotliquid/wiki/DotLiquid-for-Designers
You'll find that there are more features available in the ruby flavor that don't exist in DotLiquid
-
Where did I get that 2.0.254 version? Straight from a response from the Microsoft support team. The problem is that is does not correlate to any release on DotLiquid's Github repo.
- https://github.com/dotliquid/dotliquid/releases
- Update: I reached out to the top contributor of DotLiquid and he informed me they havent used github releases in quite a while. That version correlates to the nuget version.
What did our development/deployment process look like?
- Download VSCode and install Shopify Liquid Preview
- Create a repo
- Branch
- Create a transform.liquid and transform.liquid.json and start writing transforms! A great developer experience so far.
- CTL+SHIFT+P to preview your transform against your sample data in your .liquid.json
- Commit the template and and create a PR
- Once your PR makes it to master, upload to development azure environment, run your logic app and start feeling the pain
- If you run into problems, dig into DotLiquid source code and find the feature doesn't exist or the syntax is different
- Create another "azure" version of your template and try to keep the two in sync
- Cry on the inside
Questions for you
- Is anyone else feeling the pain of the transform connector in Logic Apps? If so, what kind of tools have you discovered to improve developer productivity?
Proposal for Microsoft
- Create a DotLiquid Preview extension that aligns to the version of DotLiquid used in the transform connector
- Deploy a new Map type to integration accounts (i.e. Liquid-Ruby) and change the configuration property
Liquid.UseRubyDateFormat = true; - Open source the Transform connector so we can make modifications and extend the capabilities
Side Notes
This is my first blog post ever. I'm interested to hear your feedback on how I can improve. Any and all feedback is welcome.
Top comments (3)
You can submit and vote for feedback and ideas for Azure Logic Apps at aka.ms/logicapps-wish
Lately Logic App has been the main contributor to DotLiquid on GitHub.
Yes we do take the DotLiquid NuGet build in Logic App. I'm not certain what was happening back when this blog was written, we do now work with DotLiquid NuGet packages. We even fixed the package built in the DotLiquid repo as it broke when old built tech certificate expired in June 2020.
There are multiple deviations and gaps between DotLiquid implementation and Shopify specification for Liquid. We have contributed more tests and GitHub issues to the open source project for that. DotLiquid could benefit of additional contributors to fix the deviations and fill the gaps - and in turn your open source contribution would make it to Azure Logic Apps.
Liquid gets so complicated when you have to debug. It was still easier to debug with Shopify, but not with Azure Liquid.
Please do share your alternate solution to this.
Im not at that company anymore, but they ended up just building their own mapper in c#/azure functions