DEV Community

Takuma Otake
Takuma Otake

Posted on • Updated on

How to "Copy" text contained in PC screens and image files

Hello everybody.
This is an English translation and modification of the article (link) posted in Japanese in advance.
* Because this is the first DEV post, there may be various things that are not good ... I would be grateful if you could let me know in the discussion.


Sometimes when using a PC, this happens.
A message box that says something incomprehensible

(Image: EmEditor message dialog)

"I don't know what it says..."
At times like this, most people try to understand the meaning using a translation website. But this is a message dialog.
After dragging to select a range, right-click and click "Copy"...
You can't do that.
So, normally, you have to manually enter it on the translation site, but when it comes to a somewhat long text, it can't be helped anymore. In that case, human beings are creatures who want to make things easier, so you think, "Is there a way to read the characters on the screen?"
This time, I would like to introduce software that makes it easy.

Free software: Cloud OCR Snip

The introduction has become long, but this time I will introduce a free software called "Cloud OCR Snip".
The method of reading the characters on the screen is introduced in other blogs, but most of them have problems such as many steps and poor recognition accuracy.
Character recognition results with many misrecognitions

(Image: The result of character recognition on the language selection page of Wikipedia with GT Text)

In the above results, the texts in Latin alphabet languages are well recognized, but the other texts are broken beyond imagination.
In general, all software that returns the above results use the local character recognition engine for character recognition. Certainly, if it is a simple sentence, it may be read properly, but if it is a slightly complicated sentence, the accuracy will drop at once.
Let's try to recognize the same sentence with Cloud OCR Snip this time.
Character recognition result with almost no misrecognition

(Image: The result of character recognition on the language selection page of Wikipedia with Cloud OCR Snip)

This one reads so well that I can't find any mistakes.
The reason for the difference so far is that the software like the one in the previous example processes the data locally, while this software uses the cloud service to process the data.
Each cloud service requires a computer to do a lot of learning work to improve recognition accuracy, so it performs far better than when it is done locally.
Since this kind of character recognition is a fairly advanced technology, most services charge for it, but the Google Cloud Vision API used in the example above is free up to 1000 times per month (beyond 1000 times, it costs US$1.5 for every 1000 times).

* The above information is as of December 2021. Please check here for the latest information.

You can also start character recognition by clicking the icon on the right side of the taskbar or pressing a hotkey, so you can easily use it even when you want to check a little.
In summary, Cloud OCR Snip and Google Cloud Vision API make it free and easy to use very accurate character recognition (up to 1000 times a month).

Get ready to use

First of all, regarding installation, since this is a developer community site, I will omit the details. Download the installer from here and run it (latest version recommended).
Next is the initial setting, but basically you can finish it by following the instructions on the screen. However, setting up a cloud service is a bit complicated, so I'd like to briefly explain it here.
・When using Google Cloud Vision API
If you search the web with "How to Create Google API JSON Credential" etc., you will reach the support page of paid software with the same function. See it.
* To use the service, you need to set the payment method (credit card and PayPal can be used).

Try using

If you left-click the icon in the task tray or press the hotkey (default: Alt key, Shift key, S key at the same time) to specify the area to read, the text on the screen can be recognized and copied.
Also, right-click the icon in the task tray and left-click "Read text from image file" in the displayed menu to display the image file selection dialog. Then select an image and you will be able to recognize and copy the text in the image.
In addition, right-click the icon in the task tray and left-click "Read text from clipboard image" in the displayed menu to recognize the text in the image on the clipboard and copy it.
* If there is no image available on the clipboard, the operation will be blocked.
* This function can also be called by hotkeys (default: Windows key, Shift key, D key at the same time).

Use in combination

If you want both the result of character recognition and the scanned image, first take a screenshot with Snip & Sketch (hotkey: Windows key, Shift key, S key simultaneously), and then cloud OCR the image. Load with Snip.
* If you take a screenshot using the hotkey in Snip & Sketch, it will be automatically copied to the clipboard. You can also save the screenshot to a file by clicking the notification that appears when the shot is complete.

Top comments (0)