Sending data through Mixed Content filters

tl;dr / bottom line up front

Using overt steganography (phanerography?), you can retrieve data on a cooperative HTTP server from an HTTPS hosted site while only triggering Mixed Passive/Display Content warnings on the browser.

But why tho?

The most basic use case is including data from a server on the local network in a page hosted on the Internet via HTTPS. Local servers have trouble getting CA-issued HTTPS certificates because HTTPS certificates require a domain name, and local servers generally don't have one. But there still might be a use case for including data from a local server on a site hosted on the Internet, perhaps a configuration page for IoT devices.

If you load non-secure data on an HTTPS page, one of two things could happen. If your content is in an <img>, <audio>, <video>, or some <object> tags, it will be loaded as Mixed Passive/Display Content. This means the site will lose its lock icon and the console will throw warnings about mixed content, but otherwise the site will work. However, if your content is loaded in any other way (<script>, <iframe>, XMLHttpRequest, etc) your unsecure content will fail to load as it'll be considered Mixed Active Content.

Most IoT devices or other network appliances simply forgo HTTPS. Plex has another solution, but it costs $5k-$10k USD per year. This article covers an alternative to those.

Binary data transmission via portable network graphics

Out of images, audio, and video, images are the easiest to programmatically create and have the lowest overhead. And of the more-or-less universally supported image formats, PNGs are ideal in that they have a grayscale mode where one byte is one pixel, they include gzip compression, they have a very low overhead, and they are not lossy.

The overhead is a constant 66 bytes for up to 2 gigabytes of data, which means even without compression (which you could apply to both), it'll be more efficient than base64 for transmitting binary data bigger than about 200 bytes, at the cost of some cpu cycles.

The server (Kotlin/JVM)

Let's start with the server. The server must receive all requests as an HTTP GET request and so all options have to be in a query string or param string. How to do that is outside the scope of this article, but it's super easy.

After it receives a request, it has to transform some data into a PNG, then return that to the requester.

This manually creates a PNG file from a string - it could have been an array of bytes, but I wrote it as a string for this example. The output PNG is a single row with a width equal to the size of the input data and each pixel represents a byte in greyscale. The cover image for this article is "Hello World" run through this, but blown up a bunch so it's visible.

Note: *arrayName is not a pointer, it's the Kotlin spread operator.

fun makePNG(data: String): ByteArray {
    val dataAsByteArray = data.toByteArray(Charsets.UTF_8) // string (utf8) as a byte array

    return (pngSignature() +
            pngIHDR(dataAsByteArray.size,1) +
            pngIDAT(dataAsByteArray) +
            pngIEND())
}

// PNG Signature - https://www.w3.org/TR/PNG/#5PNG-file-signature
fun pngSignature(): ByteArray {
    return byteArrayOf(-119,80,78,71,13,10,26,10)
}

// PNG IHDR chunk - https://www.w3.org/TR/PNG/#11IHDR
fun pngIHDR(width: Int, height: Int): ByteArray {
    val ihdrLength = byteArrayOf(0,0,0,13)
    val ihdrType = byteArrayOf(73,72,68,82)
    val ihdrData = byteArrayOf(
        *intToBA(width), // width
        *intToBA(height), // height
        8, // bitdepth - 8 so each pixel is a byte
        0, // color type - 0 is greyscale
        0,0,0 // compression, filter, and interlace methods - must be 0
    )
    val ihdrCRC = getCRC(ihdrType, ihdrData)

    return (ihdrLength +
            ihdrType +
            ihdrData +
            ihdrCRC)
}

// PNG IDAT chunk - https://www.w3.org/TR/PNG/#11IDAT
fun pngIDAT(data: ByteArray): ByteArray {
    val idatType = byteArrayOf(73,68,65,84)

    val idatData = deflate(byteArrayOf(0, *data)) // filter type 0 (no filter)

    val idatCRC = getCRC(idatType, idatData)

    val idatLength = intToBA(idatData.size) // compressed data length

    return (idatLength +
            idatType +
            idatData +
            idatCRC)
}

// PNG IEND chunk - https://www.w3.org/TR/PNG/#11IEND
fun pngIEND(): ByteArray {
    return byteArrayOf(0,0,0,0,73,69,78,68,-82,66,96,-126)
}

I know that was a lot, but out of all the code above, probably 95% of it is boilerplate to create a basic PNG. The IHDR is interesting, but only because it uses bitdepth 8 and color type 0 to allow exactly 1 byte per pixel. The rest of the chunks aren't anything special unless you're interested in the PNG file format and implementing it in the JVM.

The convenience functions getCRC(), intToBA(), and deflate() create a CRC using Java's CRC library, convert an integer into a byte array, and DEFLATE data using Java's Deflater library, respectively. They're included in the full server code.

The site (javascript)

The website hosted on HTTPS needs to solve two problems, the first is to send data along with the request to an HTTP server and then get that data.

It sends data via a query string because of course the data communication has to go through an <img> tag. This limits the request data to 1KB according to most browser restrictions.

The second problem is getting the data. The server solves the problem by sending a png that essentially wraps and DEFLATEs the data, but now the browser has to make some sense of it. It does so by drawing the img onto a <canvas> element, then reading each pixel's red value (red, green, and blue are all the same in a greyscale image) and pushing that into an array:

function pngGet(url, fn) {
    const canvas = document.createElement('canvas')
    const ctx = canvas.getContext('2d')
    const img = new Image()
    img.crossOrigin = 'anonymous'
    img.onload = function() {
        canvas.width = img.width
        canvas.height = img.height
        ctx.drawImage(img, 0, 0)

        const utf8 = []
        for (let x = 0; x < img.width; x++) {
            const byte = ctx.getImageData(x, 0, 1, 1).data[0]
            utf8.push(byte)
        }
        fn(stringFromUTF8Array(utf8), img)
    }
    img.src = url
}

This could be improved by cleaning up the <canvas> and <img> elements this creates in the DOM, but in this example it's actually outputting it for the user to see. There are also some optimizations both here and in the server code that could be done (e.g. pre-allocating the utf8 array's length).

The stringFromUTF8Array() function used above was written by Ed Wynne. I didn't modify it at all for this example. Of course if you wanted to transfer binary data, you wouldn't need to translate the byte array into a string.

A huge caveat with this implementation

The provided code only allows the creation of a 2,147,483,647 pixel wide PNG file with a single row, which has a problem... while it's technically allowed by the PNG spec, programs like Photoshop only allow 300,000 x 300,000 pixel images while Chrome and Firefox have a max <canvas> width of 32,767 pixels. So even if more rows than one were implemented, it'd only allow about 1 gigabyte per PNG. It shouldn't be a difficult fix, but this is only a proof of concept so it wasn't implemented in the code.

Regaining confidentiality and data integrity

The major problem with this is that it lacks confidentiality and data integrity. In other words, people sniffing your network traffic via unsecured Wi-Fi or Man-In-The-Middle can theoretically read and/or change the image containing your data. This is a problem with all Mixed Passive/Display Content.

One way to solve this is to roll your own encryption/decryption via something like asmCrypto.js or the Stanford JS Crypto Library. You can then encrypt the response data via the normal Java crypto libraries and decrypt the response after reading the bytes from the <canvas>. You'd have to pass the key in a side channel, with both the HTTPS site/server and HTTP server talking to a HTTPS server to post the key in a database. That HTTPS server+db could also be hosting the HTTPS website.

Closing remarks

In the future browsers may throw more blatant warnings for Mixed Passive/Display Content, or they may start treating such content as Mixed Active Content and just block it outright. Essentially, this is a work-around that may not exist forever. We at GameGlass have decided against implementing it in production for that reason.

This may have applications in sending large amounts of data because it's a bit better than base64, but why not just send the binary directly instead of wrapping it in a .png?

This may also have an application in exfiltrating data from a compromised machine, but I can't think of any situation in which this would be the preferred solution over any of the more established methods, including just sending the binary data.

This could be used for obfuscating a payload I guess, but that'd last about as long as it takes for someone to read this article.

But even if it's not super useful nor that ground breaking, I think it's pretty neat. Thanks to Ron Karroll and the rest of the guys at GameGlass for letting me bounce ideas off their heads!