😷 Clinical decision-making and AI, Product manager mistakes during discovery, and more

Hiya 👋

We’re back with another edition of the digital pharmacist digest!

Here are this week's links that are worth your time.

Thanks for reading,
Kevin

📖 What I'm reading

Road map for clinicians to develop and evaluate AI predictive models to inform clinical decision-making
🤖 Artificial Intelligence and 🩺💻 Health informatics

“The nine stages for developing and evaluating predictive AI models:
Stage 1: clarifying the clinical question or outcome(s) of interest (output).
Stage 2: identifying appropriate predictors (features selection).
Stage 3: choosing relevant datasets.
Stage 4: developing the AI predictive model.
Stage 5: validating and testing the developed model.
Stage 6: presenting and interpreting the model prediction(s).
Stage 7: licensing the AI predictive model.
Stage 8: maintaining the AI predictive model.
Stage 9: ongoing evaluation of the impact of the AI predictive model.”

Creation and Adoption of Large Language Models in Medicine
🤖 Artificial Intelligence and 🩺💻 Health informatics

"Current evaluations of LLMs also do not quantify the benefits of novel collaboration between humans and artificial intelligence that is at the core of using these models in clinical settings. The methods for evaluating LLMs in the real world remain unclear. Concerns with current evaluations range from training dataset contamination (such as when the evaluation data are included in the training dataset) to the inappropriateness of using standardized examinations designed for humans to evaluate the models….

The purported benefits need to be defined and evaluations conducted to verify such benefits. Only after these evaluations are completed should statements be allowed such as an LLM was used for a defined task in this specific workflow, it measured a metric, and observed an improvement (or deterioration) in a prespecified outcome. Such evaluations also are necessary to clarify the medicolegal risks that might occur with the use of LLMs to guide medical care, and to identify mitigation strategies for the models’ tendency to generate factually incorrect outputs that are probabilistically plausible (called hallucinations)."

11 lesser-known mistakes PMs make during discovery
👨‍💻 Product management

"#1 - Conducting discovery without building the culture to support it.
#2 - Attempting discovery without a strategic vision
#3 - Working on discovery reactively
#4 - Talking to customers only
#5 - Talking to EVERY kind of customer
#6 - Failing to frame the problem correctly
#7 - Being over-sensitive to intriguing feedback
#8 - Limiting discovery to live interactions
#9 - Not spending enough time distilling information
#10 - Using discovery to market the solution
#11 - Using discovery to explore just function and not messaging"

Thunking vs Thinking: Whose Job Does AI Automate?
🤖 Artificial Intelligence

"Thunking is the term I’ve coined to describe those tasks that are repetitive, predictable, and don’t require a high level of cognitive engagement, creativity, or critical thinking. These are the tasks that you can do almost on autopilot once you’ve figured out what needs to be done."
...
"No matter how fancy-sounding your job title is, you almost surely spend at least some — if not most — of your day thunking. It’s the data entry, the scheduling, the responding to frequently asked questions in the same way for the millionth time."
...
"That’s exactly what AI is poised to automate: thunking is on the chopping block."

Are you enjoying this digest? It would mean a lot if you'd consider forwarding it on to someone that you think would also appreciate it!