loading...

Ruby 2.7: The Pipeline Operator

baweaver profile image Brandon Weaver ・6 min read

Ruby 2.7 has added the pipeline operator ( |> ), but not in the way many Rubyists had expected.

The merge can be found here.

Update:

Matz has expressed that he is looking for opinions on this feature on Twitter

Please keep feedback constructive and civil (MINASWAN), and focus on his mention of the LISP-1/LISP-2 namespacing issues as they are a real issue for implementing real Elixir-like pipes.

Discussion is happening on the Ruby Bugtracker

What it does

The pipeline operator is effectively an alias for dot-chaining, and from the source:

# This code equals to the next code.

foo()
  |> bar 1, 2
  |> display

foo()
  .bar(1, 2)
  .display

...and from the test code:

def test_pipeline_operator
  assert_valid_syntax('x |> y')
  x = nil
  assert_equal("121", eval('x = 12 |> pow(2) |> to_s(11)'))
  assert_equal(12, x)
end

This, as it is written, is the current implementation of the pipeline operator as has been discussed in the Ruby bug tracker.

Chris Salzberg mentioned in the issue that the reason might have been its lower precedence than the dot operator:

The operator has lower precedence than ., so you can do this:

a .. b |> each do
end

With ., because of its higher precedence, you'd have to do use braces:

(a..b).each do
end

...in which he closes:

The bigger point IMHO though is that major controversial decisions are
made based on this kind of very brief, mostly closed discussion.
I don't think that's a good thing for our community.

I am inclined to agree with him and several others discussing this change, and will lay out my reasoning for this below, including points in favor and counterpoints to the contrary.

Points in Favor

Of the points in favor, I have seen a few:

Ruby already has multiple syntaxes for blocks

One of the points raised is that Ruby already has multiple syntaxes for creating blocks, such as do ... end versus { ... }.

Pipelining is seen as a way to multi-line methods and make it clearer that that is being done:

# Original
foo.bar.baz

# Current
foo
  .bar
  .baz

# Alternate Current:
foo.
  bar.
  baz

# Pipelined:
foo
|> bar
|> baz

Counterpoint

My objection to this would be that Ruby blocks are already confusing for newer developers, and are frequently a subject of contention when learning Ruby.

Adding a new language construct that does not have a clear differentiating factor will only exacerbate this and make the language harder to learn.

Precedence

Another point raised was that the pipeline operator has a lower precedence than the ., allowing for paren-free programming as mentioned above:

# Current
(a..b).each do
end

# Pipelined
a .. b |> each do
end

Counterpoint

We also have precedence arguments over the english operators and and or, which are generally agreed upon to not be in common usage in the language because of the confusion they might cause.

As with the above issue of multiple syntaxes for the same task with different precedences, this will also confuse newer programmers.

Main Objections

With recent controversial features like Pattern Matching and Numbered Parameters there have been debates about the exact syntax, but the discussion had several come to the support of the feature as it added something distinctly new to the language.

In this case I do not believe this is so. The new pipeline operator feels like an alias of ., a syntactic sugar when it could have been substantially more for the language.

The main points in favor relate to a difference in precedence evaluation, an issue that has caused great confusion in newer Ruby programmers in the past, and still continues to this day.

Introducing a new symbol in a language should add a new and more expressive way to do things in that language. I do not believe this achieves that goal.

What Could It Have Been?

The main points to the contrary are that the pipeline in other languages is a powerful and expressive feature for code. I would like to show you some of those implementations and let you judge for yourself their merits.

In Javascript

TC39 has discussed a pipeline operator which is currently under careful evaluation. As Javascript is a very similar language to Ruby, many ideas have been shared between the two languages, and many more are very possible.

In their example:

const double = (n) => n * 2;
const increment = (n) => n + 1;

// without pipeline operator
double(increment(double(double(5)))); // 42

// with pipeline operator
5 |> double |> double |> increment |> double; // 42

The pipeline operator is used to provide the output of the left side to the input of the right side. It explicitly requires that this be the sole argument to the function, of which can be circumvented with currying techniques in both languages to partially apply functions.

Javascript Pipeline Applied to Ruby

This is very similar to the Ruby convention of then:

double = -> n { n * 2 }
increment = -> n { n + 1 }

double[increment[double[double[5]]]]
# => 42

5.then(&double).then(&double).then(&increment).then(&double)

If the pipeline operator were to be an alias of then that can reasonably infer to_proc / & and method calls, it would look quite the same as Javascript:

5 |> double |> double |> increment |> double
# => 42

This achieves a few major items:

  1. It removes the need for explicit block-tagging with &
  2. It simplifies the then variant of the code
  3. It (ideally) can intelligently deal with both methods and procs

By dealing with methods and procs, I mean that this would not change the output of the above code:

def double(n)
  n * 2
end

increment = -> n { n + 1 }

5 |> double |> double |> increment |> double
# => 42

This, I believe, is a more true-to-ruby implementation that is expressive and elegant, allowing for simpler syntax to carry a more complicated idea seamlessly.

It uses the idea of duck-typing with operators to say that these two should behave the same to achieve a syntax which is very powerful.

In OCaml and F

The example from OCaml and F# look very similar, so we'll focus on the F# variant which also highlights the difference between this and composition (which I won't cover here, but worth a read):

let (|>) x f = f x

It works by piping the last parameter into the function:

let doSomething x y z = x+y+z
doSomething 1 2 3       // all parameters after function
3 |> doSomething 1 2    // last parameter piped in

I am not immediately familiar with F#, but this appears to be a currying implementation which achieves some of what was mentioned in the Javascript example

F# Pipeline Applied to Ruby

This introduces an interesting idea of currying, which is applying arguments to a function and waiting for a final argument to call through to get a value. We already have this in Ruby with curry:

adds = -> a, b { a + b }.curry

adds[2, 3]
# => 5

adds[2]
# => #<Proc:0x00007fe8ab6996a0 (lambda)>
adds[2][3]
# => 5

While I don't think auto-currying would match Ruby well, it would make an interesting addition to the pipeline operator:

adds = -> a, b { a + b }.curry

def double(n)
  n * 2
end

increment = -> n { n + 1 }

5 |> adds[2] |> double |> increment
# => 15

In Elixir

Elixir works much the same way as the Javascript implementation, except in that it can also "soft-curry" functions that are waiting for an additional input if they're in a pipeline.

Again, I'm not familiar with Elixir to a deep level, and would welcome corrections on this:

"Elixir rocks" |> String.upcase() |> String.split()
["ELIXIR", "ROCKS"]

"elixir" |> String.ends_with?("ixir")
true

Interestingly it contends with one of the issues Ruby would have with some of this, being ambiguous syntax around parentheses:

iex> "elixir" |> String.ends_with? "ixir"
warning: parentheses are required when piping into a function call.
For example:

  foo 1 |> bar 2 |> baz 3

is ambiguous and should be written as

  foo(1) |> bar(2) |> baz(3)

true

Combining Worlds

Now here's a shocking revelation: I like the idea of using it as an alias for ., but....

...it should be a combination of both that and the above ideas from other languages:

5
|> double
|> increment
|> to_s(2)
|> reverse
|> to_i

This gives us a substantial increase in expressive power while unifying the current implementation with established ideas from other languages. I would be very excited if such an implementation were to come into use as it would combine the best of the functional world with Ruby's natural Object Orientation to achieve something entirely new.

To me, that's what Ruby is, achieving something new with ideas from around the world and from different languages. We bring together the novel and exciting and make it our own, and I believe with the pipeline operator we have an amazing chance to do this!

I just do not believe that the current implementation fully realizes this potential, and that makes me sad.

Posted on by:

baweaver profile

Brandon Weaver

@baweaver

Ruby, Javascript, Lemurs, Puns, and Art. Aspie, He / Him. Currently Ruby Infrastructure and Frameworks @Square. Opinions are my own.

Discussion

markdown guide
 

Overall a pretty informative post. However I would like to add that:

5.then(&double).then(&double).then(&increment).then(&double)

Could also be written as:

5.then(&double >> double >> increment >> double)
 

I have never seen this kind of ruby before. That's very interesting.

 

The reason you might not have seen it. Is because the composition operators (>> and <<) are introduced in Ruby 2.6.0 and are still fairly new.

 

Like many other of the proposed 2.7 changes (pattern matching, anonymous block arguments) this is controversial and seems a bit pointless:

  • Anonymous block arguments look like instance variables even though they are not and don't save much typing ({ |a, b| a + b } vs { @1 + @2 }).
  • Pattern matching is somewhat nice, but IMHO works better in languages where you can overload functions/methods, which replaces a lot of conditionals in the body with pattern matched arguments.
  • The "pipeline operator" (|> in Elixir/Ocaml/F#, ->/->> in Clojure) makes a lot more sense in languages where it turns code from something like f(g(h(x)))) into x |> h |> g |> f which reduces cognitive overhead by making order of application clearer. This is generally not a problem in Ruby where the x.h.g.f form already takes care of it.

I've been an ardent Rubyist since 2004 and involved in running both the regular meetup and a RubyConf in Bangkok. But I haven't used Ruby for any new private projects in about a year now and looking at the recent direction the language is taking this probably won't change any time soon.

 

Anonymous block arguments look like instance variables even
though they are not and don't save much typing
({ |a, b| a + b } vs { @1 + @2 }).

You pick a single example and discount other examples.

I already refuted this in issue trackers, so I will refute it here too, just so that people don't singularize on these erroneous claims made by people who write clearly in the sense of how they dislike something.

Consider:

array.each {|cat, dog, hare, some_strange_name_here, some_other_strange_name|

In the middle of writing code, you add:

pp dog
pp some_strange_name_here

And in these examples, it is so much easier to write:

pp @4
pp @5

Since you decided to focus on only a trivial example, I am happy to extend your example with another one.

I am getting hugely tired of people being so single-minded.

It is totally fine to dislike a change; for example, I don't like the |> pipe operator and will not use it. But I do not think it is good to be critical AND only mention what you dislike, without finding ALL possible advantages/disadvantages.

But I haven't used Ruby for any new private projects in about a
year now and looking at the recent direction the language is
taking this probably won't change any time soon.

I am writing lots of ruby and while I dislike some of the changes,
it is not as if you are forced to use any of it. Nor would this
affect the way I write code either. I am just selective in what I
use and what I don't use.

I find it strange of you to consider wanting to use ruby or not based on new features. That is already a HORRIBLE use case if you don't use the language as-is. None of the changes made me stop writing ruby code altogether - I just don't use the changes I dislike. Are you FORCED to use anything that is added?

There is only one thing I agree with - the explanation part, which is often missing or incomplete. I think it is fine to make changes; sompe people will always dislike something. It is still important to explain, in english, what is added, why and so forth - otherwise the language is a black box. Discussions at the ruby dev meeting are also summarized in english, which is good - this should be for all changes that are discussed in japanese too, because many people do not understand japanese. I don't for instance, so I depend on english.

 

array.each {|cat, dog, hare, some_strange_name_here, some_other_strange_name|

If your blocks take this many arguments your code may have other issues.

I find it strange of you to consider wanting to use ruby or not based on new features.

I don't. I haven't picked it up for private projects for many reasons, but to me a lot of the new changes feel out of place and not like the language I fell in love with 15 years ago.

I am getting hugely tired of people being so single-minded.

You also conveniently ignored the part where I mentioned I'm still co-organizing a monthly Ruby meetup and a conference later this year. πŸ€·πŸ»β€β™‚οΈ

BTW, if you often find yourself this upset about stranger's opinions on the Internet I recommend camomile tea, mindfulness practice and going outside more.

 

I already refuted this in issue trackers

I'll point out the obvious, you can't refute an opinion. You've just presented a counter-opinion.

 

I love Ruby, it's the first programming language I ever learned (with Rails), but that is a toneless and disturbing response. Despite using Ruby for a few years now, I had no idea the project was "owned" by Matz. This is a huge blow for my love of the language :'(

 

I would say this is not a very charitable view of Matz's comment, and it would be wise to remember that English is not his first language. Perhaps we should ask for clarification on what he means by this before jumping to conclusions.

 

Exactly. People love taking things out of context too.

You can see this on reddit, where people keep on claiming "omg omg omg this is not following the principle of least surprise". It seems no matter how often I correct these statements, people do not listen and will repeat these erroneous statements. It is as if some do not want to learn, so they have a closed mind.

 

On the face of it, yours seems like a very charitable response. But I see you've met him before so point taken.

Some of this could just be a shattering of the rose-tinted view I've (possibly mistakenly) held of the Ruby language.

It sounds like you've already had a more grounded understanding of Ruby's development process:

I believe you listen, and take our opinions, but ultimately decide on what your vision is. That is your right as the creator of the language.

That's not the community driven process I had imagined. This doesn't help.

 

I don't see a problem here. Matz created ruby. And it is not as if matz is not listening to feedback either - see the issue tracker.

People who dislike something will always be super-vocal about something, and often use their EMOTION first. You can see it with the numbered parameters where people write terrible claims such as "this makes my code be like perl" or "this leads to sloppy programming". I refuted these statements but it does not change the fact that people are not objective - they are emotional and will make their statements based on emotions.

Language design has to be about making decisions. You can always reason based on objective statements, but not on emotions. And I write this while also saying that I think oldschool ruby is better than modern ruby, even though I write a lot more ruby code these days. Ruby 1.8.x was the best ruby IMO.

 

I think Matz is forcing the community's hand on this to create a discussion. A little counter intuitive maybe but it's working ;-)

I don't understand the point of the current implementation but if it becomes a way to move from LHS to RHS like in the JS and Elixir examples... I'd be okay with it.

 

The pipe operator should support a FP approach to development where the dot syntax supports an OOP approach:

Should call the method bar/2 on the object returned

from the function foo/0

foo().bar(1, 2)

Should pass the object returned by function foo/0

as the first param on the function bar/3

foo() |> bar(1, 2)

 

Since when is taking a one easy to type character and making it two annoying to type characters "syntactic sugar" πŸ€”