Automate Highlighting in Wysiwyg Editors

Working in What-You-See-Is-What-You-Get editors (WYSIWYG) can be very tiring and also taxing on your wrists. Things like highlighting the same thing over and over again can often not be done with search-and-replace, because there is just no button for it 😤

I recently had to put a transcript of a podcast into such an editor and had to bold the speaker hints, so that you clearly know who was speaking in that part of the podcast. It looks like this:

JD: Welcome to J^2 Talk AI

JH: I think first of all we should understand...

It is always alternating between JD and JH as that is the nature of a conversation 😋.

Bolden the beginning speaker hints would need me to double-left-click the abbreviations and then hit the shortcut cmd+b (I am on macOS) in this particular editor.

I do not want to do this by myself as that would mean to do roughly 20*2 = 40 actions on average per episode. And I had 5 episodes... Extremely straining on your wrists and arms and it is dull work also!

In this tutorial I will show you how I automated this task with AskUI and also how to work around some kinks of WYSIWIG-Editors like microanimations and strange behaviour.

Prerequisites

• AskUI installed: Windows, Linux, macOS
• Open your favorite WYSIWIG-Editor

The Problem

So what do we need to do?

• Find all the JD and JH
• Double-left-click each one
• Press cmd+b (macOS) or (ctrl+b Linux and Windows)

This is how it looks:

Let Us Develop It Step-By-Step

So the first thing that needs to be done is finding all the JD and JH. This can be done with the get() command. As you can see in the demo above there are possibly multiple pages, so we have to scroll somehow. The easiest way to do this is to press the pagedown-key. So let us be conservative and press that key three times.

Every time we press it there is a scroll animation that might mess up the inference. So let's wait two seconds with waitFor(2000). That should be enough to finish the animation.

for (let i = 0; i < 3; i++) {
  const jds = await aui.get().text().containsText("JD:").exec();

  const jhs = await aui.get().text().containsText("JH:").exec();

  await aui.pressKey("pagedown").exec();
  await aui.waitFor(2000).exec();
}

This already looks like it would be a good idea to refactor it and put the actual highlighting code into a function highlight().

for (let i = 0; i < 3; i++) {
  highlight();

  await aui.pressKey("pagedown").exec();
  await aui.waitFor(2000).exec();
}

async function highlight() {
    const jds = await aui.get().text().containsText("JD:").exec();

    const jhs = await aui.get().text().containsText("JH:").exec();
}

Great, now we need to mouse-double-left-click the text JD and JH one by one with a for-loop. But when we try it with await aui.moveMouseTo(jds[0]).exec() it will move the cursor to the middle of the line where JD/JH is at the beginning. We have to use the bounding box x-min and y-min values to get to the beginning of the line. Also we have to add like 10 pixels to the values to actually move the cursor over the JD/JH.

async function highlight() {
    const jds = await aui.get().text().containsText("JD:").exec();
    for (let i = 0; i < jds.length; i++) {
        const bbox = jds[i].bndbox;
        await aui.moveMouse(bbox.xmin + 10, bbox.ymin + 10).exec();
    }

    ...
}

Now we can do the highlighting by mouse-double-left-click and press the bold shortcut.

async function highlight() {
    const jds = await aui.get().text().containsText("JD:").exec();
    for (let i = 0; i < jds.length; i++) {
        const bbox = jds[i].bndbox;
        await aui.moveMouse(bbox.xmin + 10, bbox.ymin + 10).exec();
        await aui.mouseDoubleLeftClick().exec();
        await aui.pressTwoKeys("command", "b").exec(); // Use CTRL+b on Linux/Windows
    }

    ...
}

That should highlight all the JD. Let's do the same with JH. Also we need to add a keypress of escape here because the mini-menu that pops up when double-left-clicking the last JD might cover a JH

async function highlight() {
    const jds = await aui.get().text().containsText("JD:").exec();
    for (let i = 0; i < jds.length; i++) {
        const bbox = jds[i].bndbox;
        await aui.moveMouse(bbox.xmin + 10, bbox.ymin + 10).exec();
        await aui.mouseDoubleLeftClick().exec();
        await aui.pressTwoKeys("command", "b").exec();
    }

    await aui.pressKey("escape").exec();

    const jhs = await aui.get().text().containsText("JH:").exec();
    for (let i = 0; i < jhs.length; i++) {
        const bbox = jhs[i].bndbox;
        await aui.moveMouse(bbox.xmin + 10, bbox.ymin + 10).exec();
        await aui.mouseDoubleLeftClick().exec();
        await aui.pressTwoKeys("command", "b").exec();
    }
}

Two More Tweaks - Complete Code

Another problem you might run into is missing focus on the editor. This is why adding the first two lines to your code is beneficial. It moves the mouse cursor into the editor and with the left-click gets the focus.

Also extracting a function boldenText(bbox) that does the highlighting given a bounding box bbox avoids duplicate code.

Here it is, the complete code:

it('highlights all the JD and JH', async () => {
    await aui.moveMouse(500, 500).exec();
    await aui.mouseLeftClick().exec();

    for (let i = 0; i < 3; i++) {
        await highlight();
        await aui.pressKey("pagedown").exec();
        await aui.waitFor(2000).exec();
    }

})

async function highlight() {
    const jds = await aui.get().text().containsText("JD:").exec();
    for (let i = 0; i < jds.length; i++) {
        await boldenText(jds[i].bndbox);
    }

    await aui.pressKey("escape").exec();

    const jhs = await aui.get().text().containsText("JH:").exec();
    for (let i = 0; i < jhs.length; i++) {
        await boldenText(jhs[i].bndbox);
    }
}

async function boldenText(bbox) {
  await aui.moveMouse(bbox.xmin + 10, bbox.ymin + 10).exec();

  await aui.mouseLeftClick().exec();
  await aui.mouseDoubleLeftClick().exec();
  await aui.pressTwoKeys("command", "b").exec();
}

Conclusion

In this tutorial you learned how to automate tedious tasks like highlighting some recurring text in a WYSIWYG-Editor.

You also learned how to overcome some of the challenges user interfaces throw at you, like mini-popups and micro-animations.

Photo by Pankaj Patel on Unsplash