Meta open-sources multisensory AI model that combines six types of data

Meta has released an open-source AI model that combines six types of data, including text, audio, visual data, temperature, and movement readings. The model is called ImageBind and is a research project with no immediate consumer or practical applications. The core concept of the research is linking together multiple types of data into a single multidimensional index. This idea underpins the recent boom in generative AI. Meta's ImageBind model is the first to combine six types of data into a single embedding space. The six types of data included in the model are visual, thermal, text, audio, depth information, and movement readings generated by an inertial measuring unit. Future AI systems will be able to cross-reference this data in the same way that current AI systems do for text inputs. The research brings machines one step closer to humans' ability to learn simultaneously, holistically, and directly from many different forms of information. Meta is open-sourcing the underlying model, which is an increasingly scrutinized practice in the world of AI. The immediate applications of research like this will be much more limited, but it shows how future versions of the system could incorporate other streams of data.


All Comments

Recommended for you