So I was tasked to develop a C#-based stand-alone testing tool using camera to capture the GUI screen (device-under-test) for image verification and OCR (Optical Character Recognition) comparisons.
I do not have any prior experience in image processing applications so I spent a lot of time researching and reading up on the various libraries and techniques available.
I kicked things off by playing around with Microsoft Office Document Imaging (MODI) which has a OCR component by itself. I also looked into the Universal Windows Platform (UWP) OCR API, but realised that it only supports Windows 10.
After reviewing all the available OCR options, I decided to go with Tesseract, which is no doubt the most mature and robust solution in comparison.
On the hardware aspect, the actual shooting and capturing of the screens, my supervisor had already placed an order for the camera to be used during the testing process, the Canon PowerShot SX720HS.
Such a beautiful piece of hardware but there’s only one problem: There is no support for tethered shooting.
Believe me, I’ve tried searching. Very hard. The following list comprises of all leading tethering solutions which did not work for my camera model (but might be of use if you stumble upon this list for yours):
I even emailed PSRemote’s company enquiring about a custom solution and I was willing to pay, no luck;
The only workable tethered shooting option available was Canon’s Camera Connect app. So remote shooting is possible, but only for your iPhone or Android. Considering that I’ll have to run the OCR processing in succession to the capture, NO GO.
My next idea was to run the Camera Connect app via an Android emulator on the PC. Before I got to thinking I’m such a hot shit coming up with such a briliiant idea, I discovered that the method was not workable due to the emulators’ (ALL of them: Android Studio, Bluestacks, Genymotion, Nox) inability to detect the actual WiFi network. NO GO.
What if I look up the PowerShot SDK to see if I’m able to come up with something?
Oops. I downloaded the 9-year old last release SDK anyway but as expected, NO GO.
Imagine my joy and happiness when I discovered CHDK (Canon Hack Development Kit), a software application developed by enthusiasts that enables extra features for ported Canon™ “Point&Shoot” cameras.
The firmware dump for the SX720HS was available but unfortunately, the software has yet to be ported to support my model. Reading up on the porting procedure, I’m clearly ill-equipped in this area and it was frankly just too complex for a project of this size. NO GO.
So I clearly have a huge problem here, the primary intended purpose of capturing the screen, simply cannot be achieved with this model. At this point, I had started looking at alternatives which are within the project budget.
My research led me to the Yi HD Action Camera, which was the cheapest (SGD 119) camera available with generally positive reviews and with support for WiFi tethered shooting, something my supervisor brought up several times which he felt was important to have.
I quickly set up the camera and was able to get the WiFi tethering running. I’m relatively pleased with this option but there were 2 minor issues;
- Being an Action Camera, there was an in-built fisheye effect. Fisheye correction lens are actually available but the lens themselves are actually more expensive than the camera itself.
- Barely noticeable delay of screen capture via WiFi, but could be an issue? Testing conducted with a digital clock.
Source code is available here.
Links and resources referred to throughout the entire course of the project (in no particular order).
Stroke Width Transform
Canny Edge Detection