The convergence of acoustics and machine learning is unlocking new frontiers in predictive maintenance. Historically, automotive diagnostics relied on the seasoned intuition of an experienced mechanic—the ability to pinpoint component wear simply by recognizing a characteristic rattle or knock. Modern neural networks are now automating this expertise, transforming raw acoustic streams into structured data regarding the health of mechanical assemblies. The cardiag project demonstrates how open-source data harvested from social media can serve as the foundation for a functional tool dedicated to primary fault triage.
At its core, cardiag is designed as an intelligent assistant—not to replace the skilled technician, but to significantly narrow the diagnostic search space. The workflow is seamless: a user records engine noise via smartphone and uploads the file for a preliminary analysis. Acting as a first-level filter, the system identifies the presence of a fault and pinpoints the likely problematic component, allowing the mechanic to focus their attention on a specific area of the vehicle.
The project's tech stack is characterized by a striking pragmatism. Rather than opting for the computationally expensive fine-tuning of massive neural networks, the developer implemented a cascaded architecture. First, the audio stream undergoes a cleaning stage where algorithms strip away technical noise, human speech, music, and ambient interference. The refined signal is then processed by a frozen CLAP (Contrastive Language-Audio Pretraining) model, which maps the sound into high-dimensional vector representations—embeddings. Lightweight linear classifiers operate atop these vectors. This approach reduces the weight of the trained portion of the system to a negligible 100 KB, even though the base CLAP model itself occupies approximately 2 GB.
The training dataset was assembled through an unconventional pipeline: data was harvested via scraping YouTube and TikTok videos where users showcase their vehicle failures. Performance on such "noisy" data shows moderate efficacy, with an AUROC (Area Under the Receiver Operating Characteristic curve) of 0.79. While significantly better than random guessing, there is clear room for improvement. In 76% of cases, the problematic node—whether the braking system or the chassis—falls within the system's top three suggestions.
Interestingly, the primary bottleneck is not the analytical method, but the quality of the source material. When tested against clean recordings from an independent benchmark, accuracy jumps sharply to an AUROC of 0.93. The research also revealed a saturation effect: the learning curve plateaus after processing just 500 clips. This suggests that simply increasing the volume of social media data without improving its quality yields diminishing returns in precision. To ensure the reliability of these results, a rigorous methodology was employed: cross-validation was grouped by source video to prevent fragments from the same clip from appearing in both training and test sets, while statistical significance was verified via permutation testing.
cardiag pays particular attention to the issue of neural network "hallucinations" and model overconfidence. Probabilities are calibrated so that a stated confidence level of 70% corresponds to actual accuracy in 70% of cases. If a recording is too noisy or ambiguous, the system transparently returns an UNCERTAIN status rather than suggesting a random guess.
A critical analysis of the project also highlighted its vulnerabilities. For instance, the engine knock detector showed near-perfect results on familiar data (0.99) but proved useless when faced with external recordings. This classic case of overfitting to a specific dataset led to the feature being downgraded to a simple hint. Such an honest approach to error analysis transforms what could have been a mere pet project into a serious case study on the application of audio analytics in the real sector.
The Paradox of Absolute AI Alignment
