This project will develop models and algorithms for estimating actionable, physical properties of surfaces from their appearance for applications in scene understanding, robotic action planning, and efficient visual sensing. The research will address the fundamental question of how computer vision can anticipate the physical properties of a surface, laying the foundation for computational vision-for-action. The research activities are centered on four specific aims: 1) large-scale data collection of actionable physical properties and appearance measurements of everyday surfaces, 2) derivation of prediction models for deducing physical properties from local surface appearance, 3) integration of global semantic context including object and scene information, and 4) development of efficient appearance-capture optics and hardware for use in novel physics-from-appearance sensing.
This project will develop models and algorithms to accurately and efficiently capture the 3D geometry and surface properties of objects on the Earth. Collaborating with Columbia University, Purdue University, Raytheon and Kitware. Our contribution is semantic material segmentation of satellite images.
This project develops the first material camera, or MatCam, that outputs a per-pixel label of object material and its properties that can be used in visual computing tasks. In the everyday real world there are a vast number of materials that are useful to discern including concrete, metal, plastic, velvet, satin, water layer on asphalt, carpet, tile, wood, and marble. A device for identifying materials has important implications in developing new technologies. For example, a mobile robot may use a MatCam to determine whether the terrain is grass, gravel, pavement, or snow in order to optimize mechanical control. In e-commerce, the material composition of objects can be tagged by a MatCam for advertising and inventory. The potential applications are limitless in areas such as robotics, digital architecture, human-computer interaction, intelligent vehicles and advanced manufacturing. Furthermore, material maps have foundational importance in nearly all vision algorithms including segmentation, feature matching, scene recognition, image-based rendering, context-based search, and object recognition and motion estimation. The camera brings material recognition to the broader scientific and engineering communities, in a similar way that depth cameras are currently used in many fields outside of computer vision.
unique visitors since Dec 2016