DEV Community

Donald Feury
Donald Feury

Posted on • Originally published at donaldfeury.xyz on

How to create the content hash for a Dropbox file upload in Elixir

Dropbox has a specific way they want you to calculate the content hash for a file you are uploading. You can find the instructions here but to summarize:

  • Split the file into blocks of 4 MB (4,194,304 or 4 * 1024 * 1024 bytes). The last block (if any) may be smaller than 4 MB.
  • Compute the hash of each block using SHA-256.
  • Concatenate the hash of all blocks in the binary format to form a single binary string.
  • Compute the hash of the concatenated string using SHA-256. Output the resulting hash in hexadecimal format.

We can convert each of these steps into parts of a pipeline!

First we need to start reading in a file, split into chunks of the specified size. File.stream! is a good way of achieving this.

chunk_size = 4 * 1024 * 1024

File.stream!("my_file.txt", [], chunk_size)
Enter fullscreen mode Exit fullscreen mode

Next, we need to create a new hash from each chunk. Iterating over the stream chunks with Enum.map and passing those chunks into the Erlang module :crypto.hash will get us our chunk hashes.

chunk_size = 4 * 1024 * 1024

File.stream!("my_file.txt", [], chunk_size)
|> Enum.map(&:crypto.hash(:sha256, &1))
Enter fullscreen mode Exit fullscreen mode

After that, we'll combine all individual hashes into one big chunky string of hashes. Enum.join will let us that list of strings and slap them together

chunk_size = 4 * 1024 * 1024

File.stream!("my_file.txt", [], chunk_size)
|> Enum.map(&:crypto.hash(:sha256, &1))
|> Enum.join()
Enter fullscreen mode Exit fullscreen mode

Getting warmer, time to take that big boy and get one final hash using :crypto.hash again. We'll have to wrap it inside an anonymous function since we need to pass the string in as the second argument.

chunk_size = 4 * 1024 * 1024

File.stream!("my_file.txt", [], chunk_size)
|> Enum.map(&:crypto.hash(:sha256, &1))
|> Enum.join()
|> (&:crypto.hash(:sha256, &1)).()
Enter fullscreen mode Exit fullscreen mode

Home stretch! Lastily, base 16 encode the final hash using Base.encode16. Don't forget to lowercase it by passing in case: :lower.

chunk_size = 4 * 1024 * 1024

File.stream!("my_file.txt", [], chunk_size)
|> Enum.map(&:crypto.hash(:sha256, &1))
|> Enum.join()
|> (&:crypto.hash(:sha256, &1)).()
|> Base.encode16(case: :lower)
Enter fullscreen mode Exit fullscreen mode

There ya go! You should now be able to create the content hash for a file that you intend to upload to Dropbox.

Top comments (0)