Recently, the research team led by Prof. Wang Hongqiang from the Hefei Institutes of Physical Science of the Chinese Academy of Sciences proposed a wide-ranging cross-modality machine vision AI model.
This model overcame the limitations of traditional single-domain models in handling cross-modality information and achieved new breakthroughs in cross-modality image retrieval technology.
Cross-modality machine vision is a major challenge in AI, as it involves finding consistency and complementarity between different types of data. Traditional methods focus on images and features but are limited by issues like information granularity and lack of data.
Compared to traditional methods, researchers found that detailed associations are more effective in maintaining consistency across modalities. The work is posted to the arXiv preprint server.
In the study, the team introduced a wide-ranging information mining network (WRIM-Net). This model created global region interactions to extract detailed associations across various domains, such as spatial, channel, and scale domains, emphasizing modality invariant information mining across a broad range.
Additionally, the research team guided the network to effectively extract modality-invariant information by designing a cross-modality key-instance contrastive loss. Experimental validation showed the model’s effectiveness on both standard and large-scale cross-modality datasets, achieving more than 90% in several key performance metrics for the first time.
This model can be applied in various fields of artificial intelligence, including visual traceability and retrieval as well as medical image analysis, according to the team.
More information:
Yonggan Wu et al, WRIM-Net: Wide-Ranging Information Mining Network for Visible-Infrared Person Re-Identification, arXiv (2024). DOI: 10.48550/arxiv.2408.10624
arXiv
Chinese Academy of Sciences
Citation:
New AI model breaks barriers in cross-modality machine vision learning (2024, September 24)
retrieved 24 September 2024
from https://techxplore.com/news/2024-09-ai-barriers-modality-machine-vision.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
