When it comes to website speed performance, limiting the amount of HTTP calls is crucial, even if we're sparing a couple of bytes.
Since v4.2, Wordpess introduced support for emoji right into the core for older browsers, but it's likely you don't need it at all. So here's a little gist to help you through, removing any reference to the wp-emoji-release.min.js file.
Found a typo? Make a PR