Welcome to the inaugural Enso devblog. In this series, we will share the latest updates from the development of Enso (formerly Luna), a visual language for data processing.

Going forward, we’ll be sharing these updates at the end of each sprint, so you can expect an update every two weeks. This update is from the sprint ending on the 19th June 2020. If you want to keep up with the development of Enso in real time, you can follow along on GitHub.

A new name for a new codebase.

Luna is now Enso. Following a couple of years of going by Luna, we were facing issues that were making it difficult for us, and people looking for Luna. Luna is a popular term, and in programming-language land, is also very close to the popular language Lua, an endless source of confusion.

Our new name, Enso, is inspired by the Japanese 円相 (en-sō), “a circle that is hand-drawn in one or two uninhibited brushstrokes to express a moment when the mind is free to let the body create”. This concept elegantly captures what we are trying to achieve with our visual language, and is a guiding principle in ensuring that Enso is as intuitive, and natural to use, as possible.

You can find us at our new home, https://enso.org. Whilst we continue rebranding, you may still see references to Luna here and there.

As well as a new name, Enso has a completely new codebase. Luna 1.x, culminating in Luna 1.9, was a fantastic proof that what we set out to do is achievable. We were able to quickly validate the design of our dual representation visual language (where both the textual and visual versions of the workflow are equivalent and simultaneously updated), and confirm that many of the features we wanted to see in such a language, such as immutable data structures and currying, worked in the visual medium.

Thanks to the participation of our community in using and contributing to 1.x, we were able to gather a huge amount of feedback, and points of improvement, including in areas of core language design. For example, the feedback from users of 1.x lead us to recognise the necessity of a feature we now call Uniform Call Syntax, which you can learn more about here.

So why start from scratch, and not continue to iterate on 1.9? The design decisions that allowed us to make those quick iterations proved to be detrimental to future progress. For example, the GUI was slow, based on SVGs, and integrated with the Atom editor. It would be difficult, if not impossible, to get the performance we wanted building on that foundation.

Enso is a complete rewrite, taking that validated design, and now focusing on performance and usability. Firstly, the new JIT compiler is built on GraalVM, a highly-performant Java VM. We are already seeing performance in some cases of 2 orders of magnitude faster than Python. The compilation process itself is also very fast, giving immediate feedback, particularly important in a visual language.

The performance of the GUI is night and day compared to Luna 1.x, rewritten from scratch with Rust, and compiled to WebAssembly. We’ve moved away from SVGs in Atom, to our own WebGL engine, heavily optimised to display 2D graphics, allowing for graphs built of thousands of nodes to run in 60 FPS. This engine will also power our new visualisations, displaying millions of points at 60 FPS.

Thanks to GraalVM, we are also able to include some heavily demanded features. Enso will have a much bigger potential for foreign language interoperability — first with Java, but soon Python and R, bringing your existing data processing toolkit to Enso.

We hope to publish a full article reflecting on our previous tech stack decisions, and why we’ve made the changes we have. But for now, let us show you what we’ve been working on.

Can I use Enso?

Enso is currently pre-1.0, and is not ready for day to day data processing use cases. Most notably, there is currently no standard library. For now, intrepid adventurers can try out the latest builds via the artifacts in the GitHub repositories. You can find the instructions for getting and running Enso here.

Integrated Development Environment

The Enso IDE will include everything an end-user needs to process data with Enso. The IDE will support a user in writing both visual and textual Enso, with the tools and convenience expected from a modern IDE, such as auto-complete and context sensitive search.

Nodes

Users of Luna will notice that we’ve adopted a new node shape in Enso. Previously in Luna, the expression that defined a node would be displayed above the circular node, with the ports arranged along the outside of that circle. Now the node is the expression, and the input ports are the parameters of the expression.

As Enso workflows are built from connecting these nodes, we’ve been putting a lot of work into making those connections as seamless, and pleasant as possible.

Here you can see some of the new node and edge interactions:

Creating edges from input nodes.

Disconnecting edges, with highlighting for the edge to be disconnected.

Support for many (MANY) output ports.

There is also some additional handling to prevent triggering a node interaction as you mouse around your canvas, say from one node to another. You may pass your mouse over another node. On a quick cursor over, the ports do not appear, requiring a longer interaction:

As you add more nodes to your canvas, you’re going to want to move them around and organise them. Snapping makes that a breeze, helping you align graphs in your workflow:

Text editing

Enso is dual representation, meaning the visual representation (nodes) are equivalent with a more traditional, textual programming language. Enso needs a text editor as beautiful and usable as its nodes.

This sprint, the Enso text editor got rich text display capabilities. It now supports colours per glyph, formatting such as bold, italics and strikethroughs, and widgets, such as sliders and cooler choosers.

Coloured text for future syntax highlighting.

The rendering engine has been integrated with the Xi Editor rope-based backend. Thanks to Raph Levien and Trisan Hume from the Xi Editor team for all of their support and help!

Dual representation

Numerous bugs in the module responsible for keeping the textual and visual representations in sync have been fixed, and support for runtime type information has been added. This change is not yet visible, but is a precursor to enabling support for types in the visual representation. Users of Luna may recall that ports and connections between nodes were coloured by the type of data they acted upon, which will now be possible in Enso.

Dual representation correctly discovers nodes and connections, and updates the textual representation in real time as a user edits the node:

Real-time updating between visual and textual representation.

Project management

Support for projects was added to the IDE, and it is now possible to open or create projects when invoking the IDE via the --project argument:

Invoking Enso Studio (the IDE) with the --project argument to open existing projects.

Engine

The Enso Engine includes the compiler, runtime and language server, and will, in the future, include the type checker. It is used as the backend of the Enso IDE, but can also be used in isolation.

Language Server

This sprint saw the start of support for auto-completion in the IDE added to the language server.

Auto-complete suggestions are obtained by querying the Semantic Database. The database is updated by extracting autocomplete entries from the compiled Enso workflow. The IDE holds its own view of the Semantic Database, by retrieving it from the server and subscribing for subsequent updates.

Language improvements

Support for the type signature syntax was added. Whilst the signatures are not yet checked, initial support for suspended arguments has been added.

Support for suspended arguments in type signatures.

Booleans have been added to the language. These so-called “magic booleans” look like a normal datatype, but are actually well-optimised primitives under the hood.

Check out these examples:

• https://github.com/luna/enso/blob/main/engine/runtime/src/test/scala/org/enso/std/test/BooleanTest.scala
• https://github.com/luna/enso/blob/main/engine/runtime/src/test/scala/org/enso/std/test/NumberTest.scala

Runtime

Support for running “slow-path” Enso. As a functional language, Enso attempts to perform a variety of optimisations for function calls to ensure they are efficient, and easily inlined. However, there are cases, such as interpreter entry points and very polymorphic call sites, where caching and inlining would not be possible, taking too much time and memory. In this case, Enso takes the “slow-path”, which would be faster in those cases than attempting caching and inlining. You can learn more about “Megamorphic Call Sites” in the runtime docs.

More information

That’s all for this developer update. We’ll be back with more after the next sprint. You can continue to follow along with Enso development on GitHub, by joining our Discord server, or subscribing to updates on our developer mailing list.