https://dev.to/tokoyax/a-little-story-about-the-no-command-1am
Continuation of the above article
Execution speed problem
Command no is slower than theyes command like this.
> yes | pv -r > /dev/null
[31.0 MiB/s]
>./no | pv -r > /dev/null
[85.6 KiB/s]
I got the following comment.
you've better to use buffer for writing. Actually, yes command does not call write (2) for each lines.
It seems to be caused by a large number of I/O, so it is better to use a buffer.
Try to improve.
Using IO List
https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1-building-output-efficiently
According to the above article, when passing strings to IO.puts, Instead of passing only one combined string, storing the part of the string to the list has better performance.
If the same string is stored in the list, it's efficiently because the same string refers to the same address.
I fixed 'no.exs' to use a buffer.
no.exs
defmodule No do
@moduledoc """
Documentation for No.
"""
def main(args) do
args
|> parse_args()
|> run()
|> wait()
end
def say(args) do
{_, _, _, debug} = args
message = buffer(args)
case debug do
true -> IO.inspect({:debug, message, self()})
_ -> IO.puts(message)
end
say(args)
end
defp buffer(args) do
{argv, process_num, _, debug} = args
message = case debug do
true -> "#{expletive(argv)} from process no.#{process_num}"
_ -> expletive(argv)
end
buffer(args, message, [])
end
defp buffer(args, message, list) do
# faster than append last
# https://hexdocs.pm/elixir/List.html
list = [message | list]
cond do
IO.iodata_length(list) >= 4096 -> list
true -> buffer(args, message, list)
end
end
defp expletive(_ = ''), do: "n\n"
defp expletive(value), do: value
defp parse_args(args) do
{opts, argv, _} = OptionParser.parse(
args,
swithces: options_list(),
aliases: aliases_list()
)
{argv, opts}
end
defp options_list() do
[
processes: :integer,
debug: :boolean,
]
end
defp aliases_list() do
[
p: :processes,
d: :debug,
]
end
defp run(args) do
{argv, opts} = args
num = case opts[:processes] do
nil -> 1
_ -> String.to_integer(opts[:processes])
end
Enum.each((1..num), fn(n) ->
spawn_link(No, :say, [{argv, n, self(), opts[:debug]}]) end)
end
defp wait(args) do
wait(args)
end
end
Measure
before
> ./no | pv -r > /dev/null
[85.6 KiB/s]
after
> ./no | pv -r > /dev/null
[802KiB /s]
SUCCESS :D
Try parallel execution
> ./ no - p 4 | pv - r > /dev/null
[1.33 MiB / s]
Okay, let's run in parallel more.
> ./ no - p 10000 | pv - r > /dev/null
[5.00 MiB /s]
[3.16 MiB /s]
[1.84 MiB /s]
[458 KiB /s]
[466 KiB /s]
[569 KiB /s]
[724 KiB /s]
[805 KiB /s]
[1.27 MiB /s]
[6.17 MiB /s]
[15.3 KiB /s]
[1.13 MiB /s]
Although the result which speeded up was obtained, the output result was not constant.
I don't know why.
Summary
- I got knowledge of speed improvement such as IO List.
- It is simply fun to make faster.
- The
yescommand is still faster.
Github: [https://github.com/tokoyax/no:title]
(reference)
Top comments (0)