The final major driver of product adoption is acceptance by the stakeholders, mainly being clinicians and patients. In the theoretical findings, the author suggested that radiologists experience some reluctance toward software adoption because of mistrust and threat of job replacement.
However, the findings from the interviews suggest, that this acceptance of algorithms is much more a patient concern, or even more so a societal question. It is important to highlight that the perception of radiologists’ acceptance of software solutions is not necessarily accurately representing the radiologists’ true attitude toward the subject. For this reason, it would be important to build on this study to further investigate on radiologists’ take on CAD solutions and the drivers for their acceptance. As especially for society as a whole, psychological factors carry fundamental importance, it would also be interesting to investigate in further research how these effects can be mitigated through e.g. communication strategies. In summary, the acceptance of these two stakeholders jointly will also have an effect on the legal frame set for such technologies and are most decisive for their development in the future.
To conclude this discussion section, there are apparently some differences in the perception of the drivers or rather inhibitors of more autonomous software adoption. It is to be expected that employees that have different core focuses in their day-to-day work are more aware of the issues they face in absolute proximity, rather than the ones that develop at different points of the product chain. Yet, it is important to be sensitive to issues that might be faced by other stakeholders and
that there is no oblivion towards them. To assess if this balance is still asserted, it could be interesting to replicate this study again in a larger frame or question stakeholders that interact less.
6 Limitations of the Study
After establishing the findings of the study, it is important to mention that the generalizability of the of this thesis is limited to some aspects. First and foremost, one must note that the insights obtained are based on five interviews. The contributions made by these five respondents may not be representative for the entire market of software vendors or might miss some smaller aspects. It could therefore be interesting to replicate this study on a bigger scale.
Nevertheless, one may notice that the respondents covered similar opinions with some (profession caused) deviations, which pleads for the coverage of the most pressing topics that concern AI vendors in the medical imaging field.
The research question of this paper is “What are software vendor’s challenges for the diagnosis of breast cancer with AI-CAD?”, which is why all respondents work either directly or indirectly for software vendors. As previously mentioned, they make up for only one out of five groups of main stakeholders (the others being IT environment vendors, such as PACS vendors;
hospital management; radiologists and patients). To understand the barriers for AI-CAD as seen in the market jointly, one must also consider the perceptions and opinions of those groups. This could be done in the frame of a follow-up study. Especially the perception of patients, PACS vendors and hospital management have not been discussed in depth in the current academic literature.
The exclusion of these perspectives also decreased the attention paid to the legal and ethical discussion, which will have a big impact on the adoption of autonomous AI-CAD solutions.
Ethical perceptions will very likely have a significant effect on customer acceptance and legal decision making, which is why they are very important to consider. Another aspect which has also not been discussed in this research is the financial impact of AI-CAD on hospitals. Hospitals often work with very limited budgets, which is why investment decisions are also largely driven by the financial impact they will have on the hospital.
Picking up on one insight gathered during the pilot interview, this study has shed a light on the regulatory situation and cultural perception of CAD within Europe. Health regulations as well as cultural factors affecting the perception of CAD software might be very different in a different geographical area, where there may be different necessities for alternative healthcare solutions or where there is overall a different perception of technologies used in healthcare. Consequently, one might gather very different and novel insights by duplicating this study in a different geographical area.
Lastly, it should be mentioned that the literature research, questionnaire design, interview execution, data analysis and paper writing process were executed by the same person. While the researcher was supervised by an independent party throughout this process and took precautionary measures to ensure objectivity, this aspect must be kept in mind when evaluating the findings of this study. Figure 9 in appendix A portrays in more detail what proactive measures the researcher has enforced to ensure academic rigor and research quality.
7 Conclusions
As previously stated, the goal of this research was to get an understanding of what challenges there are to breast cancer detection with AI-CAD from a vendor’s perspective. The findings above suggest that people on the development side and the integration side of software
development eventually see a very similar future for such software. They believe that its importance will grow over the years to come and it may well take over the work of a radiologist in the European double-reading standard of mammography interpretation. However, both sides identify different key challenges on the way. The integration side sees the biggest challenge in the low standardization of products in the market and consequently with complications of seamless product integration in the clinician’s workflow. Meanwhile, the development side believes end-consumer (patient) acceptance to be the key to autonomous breast-cancer detection. Both sides believe that product performance is already at a high standard, but that it could be further improved.
It is important to understand that algorithm performance constitutes of how the algorithm is designed and what data is available for training purposes. Both sides indicate that the key to even better algorithm performance is the improvement of training datasets. Furthermore, it appears the options for additional features that AI-CAD solutions could entail in the future are limitless.
Participants are aware that for instance multi-tasking learning (combining the predictions from not only mammogram analysis, but also e.g. MRI or ultra-sound exams) could improve the diagnoses made, but they are also subject to regulatory standards.
These findings suggest different actions for the academic, governmental and business world. Considering the academic perspective, one must point out that this research deliberately focused on one of five key stakeholders along the supply chain, being the software vendors. Next to the software vendors, it would be essential to understand the challenges seen by the PACS vendors, the IT managers in hospitals, the radiologists and the patients concerning this topic, which is to this point a gap in the academic literature. Additionally, this study suggests that consumer acceptance and product integration are important factors for product adoption, but it does not provide proven strategies on how to address these issues. Simultaneously, other important factors
such as legal and ethical questions have received limited attention, but are most likely very important in the discussion. Further academic research could provide better insights on these aspects and improve actionable suggestions for governments and businesses.
As optimal patient healthcare is a common interest, governmental institutions must understand their role in the discussion. This paper highlights the importance of data availability for algorithm performance and explains how regulatory rigidities limit the accessibility of the same. It is the governments’ responsibility, and a question to us as a society, to find the right balance between data protection and innovative progress. A similar obligation exists in the light of regulatory standards concerning the different types of permitted algorithms.
Finally, businesses that sell breast cancer detection software must be aware of the challenges ahead and how they can address them. Misconceptions of the underlying issues can be costly and harm a company’s competitive advantage. If the main challenges are known, companies can mitigate them by adapting their product strategy accordingly. Taking integration solutions, the best integration will be the one that works best for the client and their IT infrastructure. It is therefore crucial to understand the individual requirements of the client and to communicate how a company can serve their needs best. Simultaneously, patient acceptance surely is driven by ethical believes and various psychological factors, yet there may well be communication and marketing strategies that can influence a patient’s perception of the product and affect acceptance.
Furthermore, new business opportunities arise in the market due to dissent in integration strategies that emerged from findings in the academic literature as well as the expert interviews. Thus, companies might spot their chance to develop alternative products such as plug and play platforms that serve as “translators” and facilitate the integration of the product with the client.
REFERENCES
2D vs. 3D Mammography: What’s the Difference?. (2021). Retrieved 17 June 2021, from https://www.elcaminohealth.org/stay-healthy/blog/2d-vs-3d-mammography-difference
Abnormal Interpretations: BCSC. Bcsc-research.org. (2017). Retrieved 11 April 2021, from https://www.bcsc-research.org/statistics/screening-performance-benchmarks/abnormal-interpretations.
American Cancer Society. (2021). History of Cancer Screening and Early Detection. Retrieved 17 June 2021, from
https://www.cancer.org/cancer/cancer-basics/history-of-cancer/cancer-causes-theories-throughout-history11.html
Bland, K., & Copeland, E. (2010). The Breast: Comprehensive Management of Benign and Malignant Diseases (4th ed., pp. 1-62). Saunders.
Breast cancer statistics. World Cancer Research Fund. (2021). Retrieved 11 April 2021, from https://www.wcrf.org/dietandcancer/cancer-trends/breast-cancer-statistics.
Brosky, J. (2015). Is PACS ready to expand beyond radiology images?. Retrieved 17 June 2021, from
https://healthcare-in-europe.com/en/news/is-pacs-ready-to-expand-beyond-radiology-images.html
Castellino, R. (2005). Computer aided detection (CAD): an overview. Cancer Imaging, 5(1), 17-19. doi: 10.1102/1470-7330.2005.0018
CE Marking Medical Device Standalone Software – MDR & IVDR Guide. (2021). [Blog].
Retrieved from https://blog.clevercompliance.io/ce-marking/ce-marking-medical-device/
DenseBreast-info., Berg, W. (2021). Cancer Detection in Fatty vs. Dense Breasts [Photograph].
https://densebreast-info.org
Devers, K. J. (1999). How will we know “good” qualitative research when we see it? Beginning the dialogue in health services research. Health Services Research, 34(5), 1171.
DICOM. DICOM. (2021). Retrieved 11 April 2021, from https://www.dicomstandard.org.
Dietvorst, B., Simmons, J., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal Of Experimental Psychology: General, 144(1), 114-126. doi: 10.1037/xge0000033
Euler-Chelpin, M., Lillholm, M., Napolitano, G., Vejborg, I., Nielsen, M., & Lynge, E. (2018).
Screening mammography: benefit of double reading by breast density. Breast Cancer Research And Treatment, 171(3), 767-776. doi: 10.1007/s10549-018-4864-1
European CE Marking Strategy for Medical Devices. (2021). Retrieved 17 June 2021, from https://www.emergobyul.com/services/europe/ce-certification
Federation of European Academies of Medicine, European Academies Science Advisory Council, All European Academies. (2021). International Sharing of Health Data for Research. 7-15. doi: www.doi.org/10.26356/IHDT
Firmino, M., Angelo, G., Morais, H., Dantas, M., & Valentim, R. (2016). Computer-aided detection (CADe) and diagnosis (CADx) system for lung cancer with likelihood of malignancy. Biomedical Engineering Online, 15(1). doi: 10.1186/s12938-015-0120-7
Gill, A. (2020). Top Five Drivers of the Global Teleradiology Market. Retrieved 17 June 2021, from https://www.signifyresearch.net/digital-health/top-five-drivers-global-teleradiology-market/
History of computer-aided detection (CAD). (2018). Retrieved 17 June 2021, from
https://www.siemens-healthineers.com/mammography/news/history-of-computer-aided-detection.html
Huisman, M., Ranschaert, E., Parker, W., Mastrodicasa, D., Koci, M., & Pinto de Santos, D. et al. (2021). An international survey on AI in radiology in 1,041 radiologists and radiology residents part 1: fear of replacement, knowledge, and attitude. European Radiology. doi:
10.1007/s00330-021-07781-5
Iansiti, M., & West, J. (1998). Technology integration: Turning great research into great products Marco Iansiti and Jonathan West, Harvard Business Review (May-June 1997), pp. 69-79. Journal Of Product Innovation Management, 15(2), 194-196. doi: 10.1016/s0737-6782(98)90102-3
Jorritsma, W., Cnossen, F., & van Ooijen, P. (2015). Improving the radiologist–CAD interaction:
designing for appropriate trust. Clinical Radiology, 70(2), 115-122. doi:
10.1016/j.crad.2014.09.017
Kohli, A., & Jha, S. (2018). Why CAD Failed in Mammography. Journal Of The American College Of Radiology, 15(3), 535-537. doi: 10.1016/j.jacr.2017.12.029
Lebovitz, S., Levina, N., Lifshitz-Assaf, H. (2021). Is AI ground truth really "true"? The dangers of training and evaluation AI tools based on experts' know-what. Management
Information Systems Quarterly Forthcoming Special Issue on "Managing AI".
Leichter, I., Gallimidi, Z., Heyman-Reiss, A., Merlet, N., Ratner, E., & Abramov, M. et al.
(2010). Is CAD Able to Assist in the Detection of Subtle Breast Findings – Lobular Cancers, and T1a/T1b Masses in Dense Breasts?. Digital Mammography, 474-480. doi:
10.1007/978-3-642-13666-5_64
Lomonaco, V. (2021). Why Continual Learning is the key towards Machine Intelligence [Blog].
Retrieved from https://medium.com/continual-ai/why-continuous-learning-is-the-key-towards-machine-intelligence-1851cb57c308
Mammography. (2019). Retrieved 17 June 2021, from https://www.radiologyinfo.org/en/info/mammo
Mammograms and More: Breast Cancer Screening Guidelines. (2021). Retrieved 11 April 2021, from
https://www.hopkinsmedicine.org/health/conditions-and-diseases/breast-cancer/mammograms-and-more-breast-cancer-screening-guidelines.
Mammography Outcomes Improve When Doctors Compare Prior Screenings. (2016). Retrieved 17 June 2021, from
https://www.itnonline.com/content/mammography-outcomes-improve-when-doctors-compare-prior-screenings
National Cancer Institute. Dense Breasts: Answers to Commonly Asked Questions. (2020).
Retrieved 17 June 2021, from https://www.cancer.gov/types/breast/breast-changes/dense-breasts
Metz, C. (2019, August 16). 5 Options for AI integration into your radiology workflow. Quantib B.V. https://www.quantib.com/blog/5-options-for-ai-integration-into-your-radiology-workflow
PACS: What It Is and How It Works Together With Radiology Information Systems (RIS).
(2019). Retrieved 17 June 2021, from https://www.adsc.com/blog/pacs-radiology-information-systems
Ratner, C. (2002). Subjectivity and Objectivity in Qualitative Methodology. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 3(3), Art. 16, doi:
https://doi.org/10.17169/fqs-3.3.829
Rosselli del Turco, M., Hendriks, J., Perry, N., Azavedo, E., & Skaane, P. (2006). European guidelines for quality assurance in breast cancer screening and diagnosis (4th ed., p.
189). Luxembourg: Office for Official Publications of the European Communities.
Sarvamangala, D., & Kulkarni, R. (2021). Convolutional neural networks in medical image understanding: a survey. Evolutionary Intelligence. doi: 10.1007/s12065-020-00540-3
Sechopoulos, I., & Mann, R. (2020). Stand-alone artificial intelligence - The future of breast cancer screening?. The Breast, 49, 254-260. doi: 10.1016/j.breast.2019.12.014
Shin, D. (2021). The effects of explainability and causability on perception, trust, and
acceptance: Implications for explainable AI. International Journal Of Human-Computer Studies, 146, 102551. doi: 10.1016/j.ijhcs.2020.102551
U.S. Breast Cancer Statistics | Breastcancer.org. Breastcancer.org. (2021). Retrieved 11 April 2021, from https://www.breastcancer.org/symptoms/understand_bc/statistics.
Walsh, F. (2020). AI 'outperforms' doctors diagnosing breast cancer. Retrieved 17 June 2021, from https://www.bbc.com/news/health-50857759
Was ist ein PACS? - Itz-medi.com. (2021). Retrieved 17 June 2021, from https://www.itz-medi.com/was-ist-ein-pacs/
What is CE Marking? - CE Mark Certification vs. Self Declaration | ASQ. (2021). Retrieved 17 June 2021, from https://asq.org/quality-resources/ce-marking
Valtorta, M., & Sessions, V. (2006). The Effects of Data Quality on Machine Learning
Algorithms. In International Conference on Information Quality. MIT, Cambridge, MA, USA.
APPENDIX A. FIGURES AND TABLES Figure 5
Project Workflow
External Input
Actions Researcher
Conducting the literature review
Pilot Interview
Identify status quo Identify possible
challenges according to academic literature
Design interview
Start of Project
Building the Methodological
Framework
Data Collection Data Analysis End of Project
Reach out to interviewees, provide participant information and consent
form and arrange interview
Execute interviews
Evaluate interview results
Reflect results
Interaction University Supervisor
Design of research proposal Acceptance of proposal
Revision of Interview Questions
Revision of Conclusions Revision of
LR
x-axis – project stage y-axis – key player
Submission of project
Figure 6a Top-level codes
Figure 6b All codes
Dimensions Key Quote Participant Product Excellence
Algorithm performance “ […] The weaknesses are the most important so that you can try to improve your algorithm all the time. And I would say that almost all AI vendors have that process to get back some of the cases internally to try to improve the algorithm in this way.”
“We built a product that brings confidence to radiologists when assessing mammograms, so that they can really rely on the assessment and I would say even discharge some of their work off of them in the end.”
“[…] it has been proven that AI is very well capable of predicting whether there is a tumor or not inside of the breast and they are doing a pilot right now to see if they can replace one of those radiologists. Also, the algorithms keep learning, so they keep getting better and better […]”
“There are definitely some types of lesions that the AI is very good at detecting, for instance radiologists say that it is very good at detecting microcalcifications, but it is mostly because microcalcifications are by definition micro and very difficult to see […] It [the product] is good at actually everything […] there are other very useful parts of the system where they see value, for instance, it helps them to confirm that there is nothing to see. And not only with calcifications, it reassures them you haven’t missed anything, so women can go home safely.”
“There is some improvement, because there are still really these cases that they usually can dismiss, because they are obvious things and those are things related to very specific types of findings that maybe are not often enough in our training datasets. So you need more data or you need to train in different ways to give it more importance. Or things that our system doesn’t do, like taking into account the history of the patient, including the previous mammograms. Those are the things that are important to improve further, but also to get better acceptability, so really to take into account all the information that the radiologist is taking into account.”
P3; P3; P1; P3;
P4
Data availability “I think the majority of the data that we use to train our product is not publicly available”
“We sometimes give away our product for free, meaning that the customer would have to hand over the data for us to train the algorithm, so that will sometimes happen indeed.”
“[…] very little of public datasets, or well, public in a way so there are some dataset where you can, let’s say lease, data if you pay.”
“[…] the product […] doesn’t do data collection on its own. […] Having said that, in some situations, costumers are interested to help us on this or it’s part of the negotiation.”
[On feeding back customer data to the algorithm for further training]: “That depends on how you set up the integration with the AI vendor. So that’s what I was talking about with the accept/reject workflow that a radiologist accepts or rejects the result from the AI algorithm and, if we want to, feedback that to the algorithm again so it can learn from the results.”
[On feeding back customer data to the algorithm for further training]: “Yes, definitely. This is a very important thing for all AI manufacturers. It’s to know your weaknesses and of course what you are good at. The weaknesses are the most important so that you can try to improve
P2; P2; P4; P4;
P1; P3
Figure 8b
Derivation of the Drivers of AI-CAD Adoption – Extended
your algorithm all the time. And I would say that almost all AI vendors have that process to get back some of the cases internally to try to improve the algorithm in this way.”
Research affiliations “I think what makes our product different from the ones of the competitors is that we base our products on a lot of research, so they have been trained with a huge cohort of data, meaning that we are convinced that our product will perform better than the ones of our competitors.”
“I think the majority of the data that we use to train our product is not publicly available. So indeed, that data would be gathered in a research kind of agreement, with a research hospital.”
“Some others are datasets that were previously built through research projects. […] we had some history on already working on datasets, so we have access to those datasets that are used and some others are indeed via costumers.”
P2; P2; P4
Integration in Hospital IT and Clinician’s Workflow
Dependency on PACS vendors
“I think one of the most challenging parts is the different integrations that we have. For instance there are a lot of different PACS systems that we use. Each PACS system will require Company X to do the integration in as specific way. So there is not really a guideline or a golden standard.”
“I: So the software that you for instance offer is also somewhat limited to the PACS vendor you integrate it with? If the functionality is not guaranteed [supported] by them, then..
P: Yes, correct! That’s also making it hard. Sometimes we will give a presentation to a new customer, showing them some screenshots from different PACS systems how the product can look like, but the thing that we don’t handle is that their PACS may not be able do to anything at all […]”
“We have different partnerships running with different companies that are within our platform, so next to that platform we always need other integrations for other vendors, so that needs to be set up separately from the platform.”
P2; P2; P1
Low standardization “[…] it would be very relevant to have a preset list of conditions, making it very easy for a customer to understand if they can use their PACS with Company X or not. So I think some improvements can be made. I think each PACS system will do. But it will invent its own wheel and I think there would be ways to do it more centralized or more standardized.”
“I think one of the biggest rooms for improvement would then be standardization. So if there was more standardization between PACS systems, that would make things a lot easier. So on the one hand of course all PACS systems support DICOM, Company X supports DICOM, so technically we should be able to integrate. But it should be easier I think also for customer to know whether a certain algorithm can be used by them or not. I think that is one of the things that can really be improved in that aspect.”
P2; P2; P2
Creation of plug and play platforms
“[…] a lot of platforms are in development. I know that also in Netherlands there is an initiative.
I cannot recall the name of the platform, but it’s really a platform that will be offered to a customer with any PACS system and the platform will handle all the communication with the AI algorithm. Meaning that when a customer wants a certain AI algorithm, the platform will be installed, the PACS will only communicate with that platform and not with the algorithm.
So there will be a solution in between handling all the exotic stuff and making sure that the integration in the customer PACS is working. I think that is quite a strong development.”
P2
Acceptance of stakeholders
Clinicians “The good thing with radiologists is they are pretty techy people, because they are used to technology since the beginning. […] Radiologists, they embrace the technology much more than surgeons, so there is a tendency to love technology. Still, some have some fears, so they
P3; P1; P2