Optical Sensors - Activity Detection

For challenging location detection tasks, image sensors offer a great potential. Against this backdrop, Fraunhofer IIS/EAS has developed an energy-efficient system with activity sensors using image sensor systems-on-chip with multimodal, texture-based image sequence processing. This enables robust, spatially resolved detection of movements and gestures without the need to output image data. In other words, there is no violation of privacy. The data obtained can serve as the basis for scene analyses or motion tracking, for example.

Our Approach

The approach taken by Fraunhofer IIS/EAS is based on robust texture-based, multimodal algorithms for motion detection and location. An image sensor system-on-chip (SoC) was implemented for extremely low-power data processing. Alongside the low energy consumption, “privacy by design” played a fundamental role in the development of the SoC. The derivation of area-based object features (texture) at the level of the image sensor SoC allows for maximum protection of privacy. Because no real image data is output, it is not possible to visually recognize individuals. The requirements for additional hardware are also minimized, which lowers the overall costs.

In comparison, passive infrared motion detectors (PIR sensors) are only able to detect that a movement is occurring, not what type of movement it is or where exactly it is taking place. The spatial resolution could conceivably be improved by combining multiple PIR sensors, but the tracking of individuals in order to derive a position and movement pattern is not possible. By contrast, the image sensor hardware developed at Fraunhofer IIS/EAS and the implemented software contain algorithms that have been adapted for this purpose. These algorithms allow textures to be detected and classified independently of the lighting and with useful spatial and temporal resolution. For example, people can be automatically differentiated from equipment that is displaying motion or engaging in movement (televisions, fans, etc.).