We want to make the web better. I believe fetch() upload streaming is a key to making the web simpler, safer and faster.
Available, but giving up to shipping ...
Looking back in history, the document of fetch() upload streaming seems to have started 7 years ago. The latest document is under whatwg/fetch.
Main browser vendors have positive reactions as follows.
https://github.com/whatwg/fetch/pull/425#issuecomment-531680634
At last, we have the fetch() upload streaming feature since Chrome 85.
chrome://flags/#enable-experimental-web-platform-features
However, the Chromium team is going to give up to shipping the feature. That is the reason why the post is here.
We worked with a parter but we failed to show benefits of the feature, so we're giving up shipping the feature.
https://bugs.chromium.org/p/chromium/issues/detail?id=688906#c57
The origin trial ends today, Nov 10, 2021.
Why the Web browser needs fetch() upload streaming
HTTP is the core transfer technology on the web. It is an open, widely used and trusted protocol. And we are building more rich and complex applications on the web. Some of them may deal with high-resolution videos and others may provide high-level security with E2E encryption. The fetch() upload streaming allows us to do both upload and data transformations at the same. So, we can upload while encoding rich videos in client-side. We can upload while encrypting files in client-side, which archives E2E encryption. In addition, it is easy to combine multiple transformations and build a complex one. The fetch() upload streaming is more efficient in terms of both time and space than transforming whole data.
The most important thing is that it has high compatibility. Our world is made up of all kinds of software that are integrated together. Streaming upload in HTTP is already available in common software such as Java (JVM languages), Node.js, Go, Swift, curl, etc. These common software has built-in HTTP streaming upload. I believe that built-in availability is the biggest difference from WebSocket, WebRTC and upcoming WebTransport. The streaming upload coming to web browser makes web server simpler, instead of creating another WebSocket API, etc. only for web browser. Eliminating extra entrances or branches can reduce code size, reduce unexpected behaviors and improve maintainability and safety. HTTP is a widely accepted protocol and provide practical API than raw TCP. As far as I know, HTTP is the only one that has this power.
Simple, but powerful
API is very simple as follows. You can pass ReadableStream to fetch(). The streaming upload is the most general transfer in HTTP, which allows us to transfer conventional fixed-length data and infinite data stream.
await fetch(url, {
method: "POST",
body: myReadableStream
});
Here is an example of how to create ReadableStream emitting infinite random bytes.
const myReadableStream = new ReadableStream({
pull(ctrl) {
// random bytes
ctrl.enqueue(window.crypto.getRandomValues(new Uint32Array(128)));
}
});
Here is how to convert MediaStream to ReadableStream. This can apply screen share, web camera, voice or other sources which are from MediaStream.
// Convert MediaStream to ReadableStream
function mediaStreamToReadableStream(mediaStream, timeslice) {
return new ReadableStream({
start(ctrl) {
const recorder = new MediaRecorder(mediaStream);
recorder.ondataavailable = async (e) => {
ctrl.enqueue(new Uint8Array(await e.data.arrayBuffer()));
};
recorder.start(timeslice);
}
});
}
// Usage:
const mediaStream = await navigator.mediaDevices.getDisplayMedia({video: true});
const myReadableStream = mediaStreamToReadableStream(mediaStream, 100);
Everything over HTTP
As another real example, here is a stream encrypting another stream using OpenPGP. The demo shows end-to-end encrypting (E2EE), encrypting, uploading, downloading and decrypting at the same time. In the demo below, the client is a web browser. Yet, curl and gpg are also compatible with sending and receiving since behind-the-system is a pure HTTP streaming upload.
https://github.com/nwtgck/piping-ui-web/issues/433#issuecomment-830734031
Here is another example. This is SSH over HTTP. Everything including SSH client is complete in web browser. SSH client is also implemented in web browser powered by SSHy project. So, this is also E2E encryption under SSH.
https://github.com/nwtgck/piping-ssh-web
Here is VNC over HTTP. VNC is for remote control. VNC client is complete in web browser powered by noVNC project. The application below also provides E2E encryption using openssl aes-256-ctr-command-compatible way which encrypts streamingly.
https://github.com/nwtgck/piping-vnc-web
Those two applications tunnel a TCP port over pure HTTP. So, simple commands using curl and nc or socat can also be a client, not only web browser.
With built-in software
It is worth noting that in those examples, the web browser is just one of clients. This means java.net.HttpURLConnection in the standard library in JVM languages can be a client, net/http in the standard library in Go can be a client, URLSession#uploadTask(withStreamedRequest: request) in Foundation in Swift can be a client. The fetch() upload streaming allows our web browsers to be one. Without it, this highly compatibile feature is not available since there is no alternative and it is impossible to create a polyfill.
Conclusion
The feature allows us to transfer infinitely in real-time and give efficiency in both time and space, which was never allowed before and it is impossible to create polyfill. It also gives us to communicate with other built-in software seemly and keep web servers simpler, safe and more maintainable.
I love the feature. I love the fetch() upload streaming. Infinite stream has infinite possibilities. I hope this post would be a good one for you to know how great this feature is.
More about the feature
The links below would be helpful to know more.
- whatwg spec
- Chromium issue
- Firefox
- Safari
- web.dev
- Chrome Platform Status
- Chromium Design Doc
- my post
- my post in Japanese (SSH)
- my post in Japanese (VNC)
Photo by Robert Zunikoff on Unsplash
Latest comments (0)