HeAR: A new AI Model aims to Diagnose Human Illnesses through Sound

Google has partnered with an Indian AI startup to introduce a groundbreaking bioacoustic healthcare model that can diagnose diseases based on human sounds. This innovative approach leverages bioacoustics, a fascinating fusion of biology and acoustics, to interpret sounds produced by humans and animals. By combining this with the power of generative AI—the same technology behind ChatGPT—Google is adding, HeAR to healthcare diagnostics.

Using AI to Diagnose Human Ailments Through Sound

One of the most promising developments is a foundational AI model developed by Google’s parent company, Alphabet Inc., that uses audio signals to predict early signs of disease. This technology could be integrated into smartphones, making it a portable diagnostic tool for people in hard-to-reach areas. Where traditional medical equipment like X-ray machines or diagnostic expertise is unavailable, this AI, paired with a mobile phone’s microphone, could offer a lifeline.

The AI system is already making a significant impact by tackling one of the world's deadliest infectious diseases: tuberculosis (TB). According to the World Health Organization, TB claims around 4,500 lives daily and infects 30,000 people. Although TB is treatable, millions remain undiagnosed each year. In India alone, TB kills nearly 250,000 people annually, making early detection essential for stopping its spread. Google's AI model aims to address this by identifying the subtle audio markers of TB before it advances.

How Can AI Revolutionize Disease Detection?

Google’s Indian partner in this endeavor is Salcit Technologies, a Hyderabad-based startup specializing in respiratory healthcare. Salcit is using Google's AI model to enhance the accuracy of TB diagnoses and overall lung health assessments. Their machine learning system, called Swaasa (from the Sanskrit word for "breath"), works in conjunction with Google’s AI to make these diagnoses more precise and accessible.

Google’s AI model, known as HeAR (Health Acoustic Representations), was trained using 300 million audio samples, including coughs, sneezes, and breathing sounds. These two-second audio clips were sourced from public domains, such as YouTube, and from real patients screened for TB in hospitals, including those in Zambia.

The model’s database includes 100 million cough sounds, allowing it to detect TB based on nuanced variations in coughing patterns. The AI can be loaded onto a smartphone, enabling healthcare workers to screen for TB in remote and underserved populations.

The HeAR AI model offers an affordable and efficient solution for disease screening. Instead of relying on expensive diagnostic tools, the AI can assess a patient’s cough in just 10 seconds.

With 94% accuracy, Swaasa allows users to simply cough near their phone, and the audio sample is processed in the cloud. This screening test is available for just 200 rupees ($2.40), a fraction of the cost of traditional spirometry tests, which can cost up to 3,000 rupees in clinics.

How is it going to make an impact on global health?

The ability to detect diseases using just sound opens up incredible opportunities, particularly in areas with limited access to healthcare. Google's research director, Shravya Shetty, emphasized that the AI is designed to assist even minimally trained health workers in diagnosing respiratory illnesses. The system analyzes subtle differences in cough sounds, helping to triage patients for further medical evaluation.

The HeAR AI model is already in use by some of India’s top healthcare providers, including Apollo Hospitals and the nonprofit Healing Fields Foundation, who are screening patients in rural and underserved areas. Salcit has also received regulatory approval from India's medical device authorities, marking the first time a software tool has been certified as a medical device in the country.

The Future of AI-Driven Healthcare

With the successful deployment of HeAR, the potential for AI-driven diagnostics is immense. What was once only achievable with costly diagnostic machines can now be done with a smartphone and AI, making healthcare more accessible to populations that are otherwise out of reach. By analyzing everyday sounds like coughing, AI models can offer a simple yet effective way to diagnose not only TB but potentially a wide range of respiratory and other health issues.

This technology could revolutionize healthcare in resource-limited settings, bringing diagnostic capabilities directly to people in need, reducing the need for expensive tests, and ensuring earlier detection of life-threatening conditions. As the model continues to evolve, the hope is that AI will become an integral part of global health strategies, offering scalable, cost-effective, and reliable solutions for diagnosing a variety of diseases.

In the future, HeAR could expand its capabilities beyond respiratory illnesses, utilizing bioacoustics to detect other diseases through human sounds, thereby transforming the way healthcare is delivered around the world.