OCR analysis of images

I had a very interesting time using Google OCR on a flytipped machine over the weekend:

https://twitter.com/JanetGunter/status/1046015710495936512

Google OCR told me this:

машина АА АААА Антеонтов А
калинка-М
2 03 04 [0]
see A A. an era
Ansert AssArter Add

I put the most accurate line, the second line, into Google Translate and learned the model of the Bulgarian machine

I did a Google search for this, and ended up on this video

I screen grabbed the first couple of seconds, and put into Google OCR again and got this:

КАЛИНКА | ) МАЛКіровли АНАя
МАШИНА ГЛАДИЛЬНАЯ
ЭЛЕКТРОБЫТОВАЯ
руководство по эксплуАТАЦИИ

Which means

KALINKA | ) МАЛКіровли АНАя
IRONING MACHINE
ELECTRIC EQUIPMENT
manual

Lesson: Google OCR is often good enough to capture a model or a brand name, even in Bulgarian

Wondering how we can look to incorporate this in a more automated way into our Fixometer database? Is the technology mature enough? @Karel_De_Schepper

Nice to know, Janet!
Currently, I am planning to work the following months on computer vision based device identification for the fixometer, in collaboration with Neil.

In our repair event in September, I noticed that it is quite difficult for everyone to properly register their devices and specifics about the problem/solution. I hope to bring these tools inside an app to register devices in an easier and more consistent manner. I aspire to retrieve, brand, model number, and use OCR for “extra” label information, as a first take.

In terms of maturity, the performance of computer vision techniques are not 100%, currently. However, a combination of these techniques will/should result in a more robust identification, which is one of the main topics of my research.

Looking forward in collaborating with you guys on the prototype testing!

2 Likes

Interesting!

I still believe for us, it would be a shame to fully ditch the waiting list. But your comments provoked me to think this through better.

Perhaps a public waiting list should become a more “social” document, focusing on the person - and a very simple repair status of their item :slightly_smiling_face: :neutral_face: :frowning_face:

This would be complemented by a parallel, session-based mobile photographic registration of the item, including more technical comments and details about the repair. The data we’d like to have a greater policy voice.

Benefit of maintaining both: the waiting list could serve as a short-term “back up” in case some digital data gets lost.

Benefit of developing the new feature: the photographic registration of the item could eventually allow for easier self “pre-registration” of items by event-goers - this is a feature many groups have pushed for.

2 Likes

I still believe the public waiting list has a purpose during events, especially in the first stage/real time communication to participants. I feel that the application should not, in first instance, replace this. However, it should be more convenient for repairers and hosts (or even participants?) to collect proper data collection (devide_id, problem, solution).

Nice to know that there is feedback from the community for such a feature. My aim is to test out the application during a few selected events sometime in January to receive feedback and measure the performance of the system, i.e. the potential benefits.

I a month or so, I’d like to have call again with Neil on this. Can be nice if you want to join as well in terms of “user/event interaction”. I’ll forward some slides from the last call we had!

2 Likes

Just as an aside, IMO we don’t have to think of image analysis and photographic registration as necessarily coupled together. i.e. it could be beneficial used pre, during or post event, whenever someone is able to upload the image.

But it could certainly be useful at registration, if it can quickly suggest category, brand and model, as from there you can then think about pulling in useful repair info to help with the fix during the event.

For context the way we were thinking is that Karel can work on an app that can interact with the Fixometer database via API calls, as a good means to test out the image analysis in R&D environment yet still recording event/device data in the Fixometer.

2 Likes

I applied OCR using Tesseract4 with the new deep learning engine, and from this MacBook Pro (courtesy of iFixit) if was able to retrieve the following text:

Designed by Apple in California Assembled in China Model A1989 EMC 3214 Rated 20V===3A max.
FCC ID: BCGA1989 and IC: 579C-A1989 CAN ICES-3 (B)/NMB-3(B) Serial C17X14X9JHC8

Ce I) & Re

Not bad at all!! Can be quite interesting to use as a post-processing step too, as @neil pointed out.

1 Like

Great stuff, @Karel_De_Schepper - now all we need to do is teach the machine to scrub the serial numbers, privacy and all… :wink: