The artificial intelligence (AI) system developed by Google Health more accurately identifies breast cancer than human experts, reports The Guardian (and others). The system has been tested in the UK and the US, and the results are published in Nature, one of the most prestigious scientific journals in the world.
The system produced 5.7% fewer false positives and 9.4% fewer false negatives in the US data. (False positives are cases where the system mistakenly labelled the image as cancerous while the patient in fact did not have cancer. False negatives are the opposite: these are the cases where the system missed the tumors.)
The UK results were more modest: there were 1.2% fewer false positives and 2.7% fewer false negatives compared to human experts.
The difference in performance seems to do with how breast cancer screening works: in the US, women are seen every one to two years and the mammograms are examined by one radiologist. In the UK, the screening period is every three years, and two radiologists review the results. (A third expert usually gets involved in cases of disagreement.)
It is also worth keeping in mind when considering the results that the AI system received less information than the human experts: it had no access to patient histories, for example, or prior mammograms, notes Google Blog on this topic.
The system was developed over two years in collaboration with DeepMind (a Google subsidiary), Cancer Research UK Imperial Centre, Northwestern University, and Royal Surrey County Hospital.
While the results are quite promising, two things are important:
First, the performance of the AI system slips when tested on images from different mammogram systems. The study reported in Nature used images from machines provided by a single manufacturer. This is known as data capture bias, and it is a common problem in image recognition systems.
Google and the University of Toronto researchers write in an unrelated paper that “the performance of a model can vary depending on what instruments were used to capture the input to the model. For example, a face detection model may perform differently depending on the camera’s hardware and software, including lens, image stabilization, high dynamic range techniques, and background blurring for portrait mode. Performance may also vary across real or simulated traditional camera settings such as aperture, shutter speed and ISO. Similarly, video and audio input will be dependent on the choice of recording instruments and their settings.” And since mammograms are a type of image, factors impacting the performance of an image recognition system also apply to those interpreting mammograms.
The second point that many people make, including the creators of the Google AI system, is that the breast cancer recognition system is not meant to replace radiologists. It will rather complement them, as a “second pair of eyes and a safety net,” says Dr. Caroline Rubin, Vice President for Clinical Radiology at the Royal College of Radiologists.
And some countries simply don’t have enough radiologists – in the UK, for example, 8% of hospital posts for breast radiology are unfilled, with many older radiologists retiring and not enough new ones joining in, perhaps scared away by the AI advances such as the one reported here.
Looking to learn where else AI and ML are used in healthcare? See our blueprint Get Started With AI.
To learn about data capture bias and other machine/algorithmic biases, including data and design biases, consult our blueprint Mitigate Machine Bias.
Transparency, explainability, and trust are pressing topics in AI/ML today. While much has been written about why these are important and what organizations should do, no tools to help implement these principles have existed – until now.
Recently I attended the inaugural Emotion AI conference, organized by Seth Grimes, a leading analyst and business consultant in the areas of natural language processing, text analytics, sentiment analysis, and their business applications. So, what is emotion AI, why is it relevant, and what do you need to know about it?
SortSpoke’s novel approach to machine learning answers a longstanding problem in financial services – how to efficiently extract critical data from inbound, unstructured documents at 100% data quality.
Amazon is offering its cashierless store technology to other retailers. The technology known as “Just Walk Out” eliminates checkout lines, offering an “effortless” shopping experience and shifting store associates to “more valuable activities”.
As the COVID-19 pandemic is shutting down whole countries, a few of you may be wondering whether AI can help create a vaccine for the virus responsible. After all, AI is magic, right?
Alphabet is facing backlash from its shareholders over its approach to digital privacy, reports the Financial Times. And not for the first time. This time, however, things will need to change.
The EU plans to invest €6 billion to build a single European data space, reports EURACTIV. The envisioned space will house personal, business, and “high-quality industrial data” and create the infrastructure for data sharing and use across businesses and nations.
“Facebook quietly acquired another UK AI startup and almost no one noticed,” reported TechCrunch on February 10. We looked into why.
In a landmark ruling, a Dutch court has ordered an immediate halt to the government’s use of an automated system for detection of welfare fraud.