Mastering The Heap: How to Capture and Store Images from Fetch Responses

As software developers we at times do not invest the time to understand how things work.

We always hear the adage "Don't reinvent the wheel."

This mindset can stifle creativity and a failure in advancement of our technical knowledge. This will result in less rewarding and fulfilling personal and professional lives.

To build up my previous post where our API presented an image to the browser, we will attempt to look into fetching and saving an image from an API. Why do we need this? We don't, but why not 🤷🏿.

There are many posts related to this topic, and not many approach the topic in a runtime agnostic fashion. Many examples use the DOM (Document Object Model) to accomplish this task. Today we will use a set of JavaScript runtime agnostic utilities; let's get started.

When in doubt READTHEDOCS

TLDR;

Security Considerations:

While we will attempt to use best practices, any code should be scrutinized. Do not use this in production without a approval from your organization, pentesting (penetration testing) to verify it follows your organizations safety guidelines.

Requisite Knowledge:

1. Data encoding (Hexadecimal, base64, ascii, utf-8, utf-16).
2. Fetch API (outside libraries will require additional steps).
3. System I/O (input/output) calls.
4. Asynchronous vs Synchronous programming.
5. Types of media (image, audio, and video)

The Setup

• Install depencies.
  1. curl -fsSL https://bun.sh/install | bash
• Fetch/AJAX.
  1. Native Fetch
  2. node-fetch.
  3. axios

• Setup a bun-flavored project.

  mkdir sample
  cd sample
  bun init
  

• Setup Testing

  • create a feature folder to house our testing. This should help prevent your shell's autocomplete suggestions when running the "test" command.

  mkdir features
  

  • sample test module.

Technology

• ECMA Script

  1. Request Interface
  2. Fetch API
  3. URL Interface
  4. Headers Interface

• Data Producer

  1. Random image generator API. picsum

The Code

Helper Function

1. Change from header keys from kebab-case to camelCase.

   • Package Dependents Change Case
   • DIY
     • Create a replacer function that will handle our results from the String.replace function.
     • create the case change function
     • create a function to convert case for the header's keys.

2. Create a jpeg image processing function. The call to the API will return a JPEG/JPG image, so we will not worry with other images.
   

3. Create a helper function to extract the content disposition header.
   

Fetching The Data

• Prepare the object to request a new random image.
• Make network request. We will be reading the raw data from the response. I do not recommend using any other library as you will have to contend with encoding as well as creating a Buffer, Blob, or TypedArray. So lets just cut the extra steps out.

Save The Data

• the penultimate step is to save the data. You will have to figure this part out on your own (It's just import statements, and calling the function). I am not gonna give you everything silly.
• The last step is to open the image and see what you got.

Testing

Maybe Later

Example Code

import { createWriteStream } from 'fs'
class FileNameError extends Error {
    constructor(filename:unknown) {
      super(
        `Incorrect data give for file name. Expected a string received: ${!filename ? typeof filename : filename.constructor.name}`,
      );
    }
  }
type ParsedResponse = {
    [x: string]: any;
    body?: Response;
}
/**
 * see https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types
 */
const mediaExtensionyTypes = new Map([
    ['jpg', 'image/jpeg']
])

const headerCaseChange = (sentence: string) => sentence.replace(/\-\w/g, (word) => word.toUpperCase().slice(1))
const getFileName = ({ contentDisposition }: Record<string, string>) => contentDisposition.split(";").reduce((accum: Record<string, string|null>, current: string)=> {
    const [key, val] = current.split("=")
    const value = JSON.parse(val?.trim()||"null")
    return {...accum, [key.trim()]: value }
}, {})
const prepareFetch = async () => new Request(new URL('https://picsum.photos/536/354'),{ mode: 'cors' })
const getHeaders = (headers: Headers) => [...headers.entries()].reduce((accum, [key, value]: string[]) => ({...accum, [headerCaseChange(key)]: value}), {})

const processJpegImage = async ({ value }: ReadableStreamReadResult<Uint8Array>) => {
    if (!value?.byteLength) return 'none'
    const [ a, b, c] = value
    if (a === 0xff && b === 0xd8 && c === 0xff) return 'jpg' 
}
const fetchImage = async (): Promise<ParsedResponse> => {
    const request = await prepareFetch()
    const {headers, body} = await fetch(request)
    const reader = body?.getReader()
    const data = await reader?.read()
    if (typeof data === 'undefined') return {}
    const extension = await processJpegImage(data)
    return { extension, headers: getHeaders(headers), data}
}
const saveResponse = async () => {
    const { headers, data, extension } = await fetchImage()
    const mediaType = mediaExtensionyTypes.get(extension) ?? 'application/octet-stream'
    const { filename } = getFileName(headers)
    if (!filename) return new FileNameError(filename)
    const stream = createWriteStream(filename)
    stream.write(data.value)
    return { extension, mediaType, filename, }
}

Inspiration was for this post came from this blog post. Saving and Image...

Resources:

OWASP File Upload Cheatsheet