Support KenColour App on Generosity by IndieGoGo
Technically correct colour and pattern recognition is a very challenging task to solve. The main computational obstacles include:
- Producing an initial good quality image in poor lighting conditions.
- Recognising complex patterns.
- Implementing correct linguistic and cultural description of colours, hues, and patterns.
- Full image sampling. Because visually impaired people are unable to aim a small area of the focus field towards the object of interest KenColour will analyse the entire image.
- Motion and focus detection. KenColour will use iPhone's motion detection capabilities in order to detect when the iPhone remains relatively still. When that happens the app will assume that the camera is pointing towards the object of interest and will perform rapid image acquisitions and analysis to resolve colours. Fine lateral movements of the device won’t affect this process. On the contrary, the iPhone can still be moved towards or away from the object (zoom in/out) without causing the data acquisition to stop.
- Determining lighting conditions. Depending on the iPhone model, various tests will be performed to determine as close as possible the exact lighting conditions (e.g. exposures with different ISO settings, exposure time, use of device specific low-level APIs, etc.). The exact set of calculations will be determined case by case - the poorer the lighting conditions, the more complex the set.
- Image calibration. In rare cases of very poor non-natural lighting conditions (e.g. almost dark room with weak incandescent spotlight) extra calibration is required. The app will ask for the use of a calibration card (purchased separately, or downloaded as a PDF file).
- Producing high quality images. KenColour needs high quality images in order to determine the correct colour composition and colour patterns. To do that the app will use the same well proven methods astronomers use to produce deep-sky photographic images, such as stacking and averaging of multiple images, and subtracting from the result the combined - device and lighting specific - calibration images.
- Identifying colour clusters. In most cases in a single image there are lighting gradients and shadows. By calculating gradients and comparing histograms produced for different samples, KenColour will determine the set of visible colours regardless of those lighting effects.
- Speed vs. precision. When KenColour is used in familiar environments (expected object arrangement, e.g. "a plate with zucchini in different colours", "a row of clothes hanging in the wardrobe", etc.) fast scanning is essential. By briefly holding the iPhone pointed at the object, only the simple colour (e.g. "red", "yellow", "brown", etc.) will be announced. If you hold the iPhone still for a little longer, the detailed colour name and - optionally - the pattern, will be described (e.g. "sulphur yellow", "blue and white stripes", etc.). This mode is important for unfamiliar environments.
- Describing colours. KenColour will be used by people with different language and cultural backgrounds, and different levels of visual impairment. The app will support different mapping dictionaries, and will be able to enhance the dictionary with recorded by the app colour names and pattern descriptions.
- Pattern detection. There are patterns that are relatively easy to detect, such as stripes, plaid or dots. But others can be extremely difficult, for example a flower pattern. Most of the pattern recognition algorithms will be based on a combination of edge detection, cluster analysis, and repetition detection. As a minimum, KenColour will list the dominant colour names.
- Additional learning heuristics. In order to minimise the need of using a calibration card, a set of heuristic learning algorithms will allow KenColour with the time to remember specific lighting environments and object compositions (e.g. "home in the evening", "my kitchen desk", "my wardrobe", "the office shelf").
- Aim the iPhone towards the direction you would like to scan.
- When the device is roughly aimed towards the object, the VoiceOver will announce the name of the prime colour.
- Hold the iPhone still for a little while, and the VoiceOver will announce first the precise colour name, followed by the pattern description.
For any further information, you may contact us at: firstname.lastname@example.org