The Importance of Evidence

A place to get your questions answered from McDougall staff dietitian, Jeff Novick, MS, RDN.

Moderators: JeffN, carolve, Heather McDougall

Re: The Importance of Evidence

Postby JeffN » Wed Jun 28, 2017 1:35 pm

'What Used to Be Fraud Is Now Alternative Medicine'
Perry Wilson, MD, discusses the new quackery with Steven Novella, MD
F. Perry Wilson MD, MSCE
June 28, 2017

https://www.medpagetoday.com/PrimaryCar ... cine/66316
(Video and transcript)
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Tue Jul 25, 2017 9:18 am

How sad. Information has been in a steady decline of being dumbed down and it's about to fall off a cliff.

In Health
Jeff


We may have seen the future of TV news this week
USA Today
7/22/2017

https://www.usatoday.com/story/tech/tal ... 500043001/

"The episodes I watched were five minutes in length, with five stories each (roughly 60 seconds a piece).

That’s about on par with most made-for-web video, which assumes that people’s attention spans won’t last much longer than a minute before clicking away.

...

Beyond the flashy graphics, we like Snap’s approach to storytelling, which could teach us a lot about how to communicate in all walks of life. Tell us what we need to know in quick, easy to understand bites, and you’ll get and keep our attention."
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Fri Sep 08, 2017 6:36 am

While this article takes a political spin, which is not why I am posting this, it's comments about expertise are as relevant and as true in regard to diet, nutrition and health.

In Health
Jeff


How We Killed Expertise - (And why we need it back.)
By TOM NICHOLS
September/October 2017

"What is qualitatively different today is that ordinary citizens seem increasingly confident in their views, but no more competent than they were 30 or 40 years ago. A significant number of laypeople now believe, for no reason but self-affirmation, that they know better than experts in almost every field. They have come to this conclusion after being coddled in classrooms from kindergarten through college, continually assured by infotainment personalities in increasingly segmented media that popular views, no matter how nutty, are virtuous and right, and mesmerized by an internet that tells them exactly what they want to hear, no matter how ridiculous the question."

"And as it stands now, attacks on expertise often amount to a demand from ordinary citizens—sometimes encouraged by politicians and hucksters—that their views, no matter how contradictory or hazardous, be considered equal to those of the most experienced expert."

http://www.politico.com/magazine/story/ ... 1?cmpid=sf
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Mon Mar 12, 2018 2:31 pm

Excellent article on deciphering health news

In Health
Jeff

The Appropriate Evaluation Of The Anecdote

Nutrition Education And Behavioral Science
American Society for Nutritional Sciences
MARCH 11, 20180
BY COLBY VORLAND

"I spend a lot of time reading social media postings of various dietary tribes, and constantly see success stories of weight loss and a wide range of other health observations. To build on my last post on social media misinformers, the anecdote is a key feature of binding these groups together and reinforcing their dietary supremacy. Anecdotes about any particular diet’s efficacy are often trumpeted by highly motivated individuals in which the diet may align with their personal ideologies. But when thinking about health and nutrition in a scientific way, how should we internalize them? At extreme ends, people will interpret anecdotes in different ways. For example, if a particular diet helped someone lose more weight than compared to other diets in research trials, some will say that the diet can’t be appropriately studied in the research setting. On the other end, someone with a training in science might suggest we disregard anecdotes completely as unreliable. I’ve been pondering a lot lately how much weight we should give to diet anecdotes that we read online. In this post, I’ll try to argue why a mixture of both in the appropriate contexts is probably the best approach, and explore several cognitive biases that make us exaggerate the importance of anecdotes in the interpretation of health and nutrition."

Read more....

https://nutrition.org/the-appropriate-e ... -anecdote/
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Wed Mar 28, 2018 12:16 pm

The Person Who’s Best at Lying to You Is You

https://qz.com/1231534/the-person-whos- ... ou-is-you/

Some highlights...

In 1999, together with social psychologist Justin Kruger, Dunning identified the co-eponymous Dunning-Kruger effect: people who are incompetent and lack knowledge in a field tend to massively overestimate their abilities because, quite simply, they don’t know enough to recognize what they don’t know. So hugely unqualified people erroneously believe that they’re perfectly qualified.

Our false confidence in our own beliefs also deters us from asking for advice when appropriate – or to even know to whom to turn. “To recognize superior expertise would require people to have already a surfeit of expertise themselves,”

“We’re living in a world in which we’re awash with information and misinformation,” says Dunning. “We live in a post-truth world.”

There’s no quick fix to this, but there is a key step we could take to avoid being so willfully misinformed. We need to not only evaluate the evidence behind newly presented facts and stories, but evaluate our own capability of evaluating the evidence.
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Sun Sep 16, 2018 8:42 am

Some great points in this


Is Nutrition Research Seriously Flawed? Can Hazelnuts Really Add Years to Your Life? -
Medscape - Sep 14, 2018.

Ioannidis argues that the nonstop barrage of associations reported between various foods and chronic diseases inappropriately implies causation and potentially causes more harm than good to the public.

"Unfounded beliefs that justify eating more food, provided 'quality food' is consumed, confuse the public and detract from the agenda of preventing and treating obesity," Ioannidis writes. Current guidelines may even contribute to some obesity, he suggests, if people consume too many "healthy food" calories, but don't spend them.

https://www.medscape.com/viewarticle/902024
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Mon Dec 24, 2018 7:46 am

https://www.metacausal.com/CLAIMS/

STUDY

The CLAIMS (Causal language and strength of inference in academic and media articles shared in social media) study is a systematic review of the state of causal inference in health research as it reaches the consumer through social media. This site serves both as a public explainer of the study and a public access repository of the full protocol, review tool, dataset generated during the review process, analysis code, reviewer profiles, and results from the CLAIMS study. In addition, we will be discussing this study and related topics in the main blog section.

The full, published version of this study is in PLOS ONE, and can be found here.
Citation: Haber N, Smith ER, Moscoe E, Andrews K, Audy R, Bell W, et al. (2018) Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review. PLoS ONE 13(5): e0196346. https://doi.org/10.1371/journal.pone.0196346

What did we find..

Among the scientific articles most likely to be consumed in 2015, only 6% of the scientific articles were rated as having strong enough causal inference that it should be utilized in practice. In other words, very few studies show that a change in X would actually cause the change in Y at the level reported, just that they were related in some way. When looking at how the academic authors talked about their own results, we found that 20% of them strongly implied causal results, and that 34% of them used language that was too strong given our reviewers’ assessment of strength.

When looking at the media articles, we found that 44% of the media articles used causal language that was stronger than the academic articles, remembering that many of those studies were overstated to start with. Furthermore, 58% of the media articles contained at least one substantial inaccuracy about the study.
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Tue May 07, 2019 8:59 am

:idea: Caveat Emptor!

Level and Prevalence of Spin in Published Cardiovascular Randomized Clinical Trial Reports With Statistically Nonsignificant Primary Outcomes - A Systematic Review
Original Investigation Medical Journals and Publishing
May 3, 2019
JAMA Netw Open. 2019;2(5):e192622. doi:10.1001/jamanetworkopen.2019.2622

Key Points

Question Do authors of cardiovascular randomized clinical trials present statistically nonsignificant primary outcomes accurately and objectively?

Findings In this systematic review that included 93 reports of randomized clinical trials from 6 high-impact journals, positive spin of statistically nonsignificant primary outcomes was found in 57% of abstracts and 67% of main text of the published articles.

Meaning Despite peer review, manipulation of language in the cardiovascular literature is common and may have implications for scientific integrity, patient care, peer review, and medical progress.

Abstract
Importance Clinical researchers are obligated to present results objectively and accurately to ensure readers are not misled. In studies in which primary end points are not statistically significant, placing a spin, defined as the manipulation of language to potentially mislead readers from the likely truth of the results, can distract the reader and lead to misinterpretation and misapplication of the findings.

Objective To determine the level and prevalence of spin in published reports of cardiovascular randomized clinical trial (RCT) reports.

Data Source MEDLINE was searched from January 1, 2015, to December 31, 2017, using the Cochrane highly sensitive search strategy.

Study Selection Inclusion criteria were parallel-group RCTs published from January 1, 2015, to December 31, 2017 in 1 of 6 high-impact journals (New England Journal of Medicine, The Lancet, JAMA, European Heart Journal, Circulation, and Journal of the American College of Cardiology) with primary outcomes that were not statistically significant were included in the analysis.

Data Extraction and Synthesis Analysis began in August 2018. Data were extracted and verified by 2 independent investigators using a standard collection form. In cases of disagreement between the 2 investigators, a third investigators served as arbitrator.

Main Outcomes and Measures The classifications of spin type, severity, and extent were determined according to predefined criteria. Primary clinical outcomes were divided into safety of treatment, efficacy of treatment, and both.

Results Of 587 studies identified, 93 RCT reports (15.8%) met inclusion criteria. Spin was identified in 53 abstracts (57%; 95% CI, 47%-67%) and 62 main texts of published articles (67%; 95% CI, 57%-75%). Ten reports (11%; 95% CI, 6%-19%) had spin in the title, 35 reports (38%; 95% CI, 28%-48%) had spin in the results section, and 50 reports (54%; 95% CI, 44%-64%) had spin in the conclusions. Among the abstracts, spin was observed in 38 results sections (41%; 95% CI, 31%-51%) and 45 conclusions sections (48%; 95% CI, 38%-58%).

Conclusions and Relevance This study suggests that in reports of cardiovascular RCTs with statistically nonsignificant primary outcomes, investigators often manipulate the language of the report to detract from the neutral primary outcomes. To best apply evidence to patient care, consumers of cardiovascular research should be aware that peer review does not always preclude the use of misleading language in scientific articles.
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Thu May 09, 2019 6:08 am

'No-Spin Zone'? Not in Cardiology
High proportion of abstracts and papers up-sell their findings, study finds
by Nicole Lou, Contributing Writer, MedPage Today
May 06, 2019

https://www.medpagetoday.com/cardiology ... tion/79637

Even when published in the top journals, cardiology research is not immune to investigators putting a positive spin on results that are neutral at best, according to a systematic review.

Among 93 randomized trials for which the primary endpoints missed statistical significance, 57% of abstracts and 67% of main report papers were subject to some spin, or "the manipulation of language to potentially mislead readers from the likely truth of the results," reported Richard Krasuski, MD, of Duke University Health System in Durham, North Carolina, and colleagues.

Main reports featured a spin in 11% of titles, 38% of results sections, and 54% of conclusion sections, while abstracts had a spin noted in 41% of results sections and 48% of conclusion sections, Krasuski's group reported online in JAMA Network Open.



JAMA paper


Level and Prevalence of Spin in Published Cardiovascular Randomized Clinical Trial Reports With Statistically Nonsignificant Primary Outcomes - A Systematic Review
May 3, 2019
JAMA Netw Open. 2019;2(5):e192622. doi:10.1001/jamanetworkopen.2019.2622

https://jamanetwork.com/journals/jamane ... le/2732330

Abstract
Importance Clinical researchers are obligated to present results objectively and accurately to ensure readers are not misled. In studies in which primary end points are not statistically significant, placing a spin, defined as the manipulation of language to potentially mislead readers from the likely truth of the results, can distract the reader and lead to misinterpretation and misapplication of the findings.

Objective
To determine the level and prevalence of spin in published reports of cardiovascular randomized clinical trial (RCT) reports.

Data Source
MEDLINE was searched from January 1, 2015, to December 31, 2017, using the Cochrane highly sensitive search strategy.

Study Selection
Inclusion criteria were parallel-group RCTs published from January 1, 2015, to December 31, 2017 in 1 of 6 high-impact journals (New England Journal of Medicine, The Lancet, JAMA, European Heart Journal, Circulation, and Journal of the American College of Cardiology) with primary outcomes that were not statistically significant were included in the analysis.

Data Extraction and Synthesis
Analysis began in August 2018. Data were extracted and verified by 2 independent investigators using a standard collection form. In cases of disagreement between the 2 investigators, a third investigators served as arbitrator.

Main Outcomes and Measures
The classifications of spin type, severity, and extent were determined according to predefined criteria. Primary clinical outcomes were divided into safety of treatment, efficacy of treatment, and both.

Results
Of 587 studies identified, 93 RCT reports (15.8%) met inclusion criteria. Spin was identified in 53 abstracts (57%; 95% CI, 47%-67%) and 62 main texts of published articles (67%; 95% CI, 57%-75%). Ten reports (11%; 95% CI, 6%-19%) had spin in the title, 35 reports (38%; 95% CI, 28%-48%) had spin in the results section, and 50 reports (54%; 95% CI, 44%-64%) had spin in the conclusions. Among the abstracts, spin was observed in 38 results sections (41%; 95% CI, 31%-51%) and 45 conclusions sections (48%; 95% CI, 38%-58%).

Conclusions and Relevance
This study suggests that in reports of cardiovascular RCTs with statistically nonsignificant primary outcomes, investigators often manipulate the language of the report to detract from the neutral primary outcomes. To best apply evidence to patient care, consumers of cardiovascular research should be aware that peer review does not always preclude the use of misleading language in scientific articles.



Combating Misrepresentation of Research Findings
Stephan D. Fihn, MD, MPH1,2
JAMA Editorial
May 3, 2019
JAMA Netw Open. 2019;2(5):e192553. doi:10.1001/jamanetworkopen.2019.2553

https://jamanetwork.com/journals/jamane ... le/2732324
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Thu May 30, 2019 11:24 am

Does the news reflect what we die from?
May 29, 2019
Hannah Ritchie

Except

"How over- or underrepresented are deaths in the media?

As we can see clearly from the chart above, there is a disconnect between what we die from, and how much coverage these causes get in the media. Another way to summarize this discrepancy is to calculate how over- or underrepresented each cause is in the media. To do this, we simply calculate the ratio between the share of deaths and share of media coverage for each cause.

In the chart below we see how over- or underrepresented each cause is in newspaper coverage.4 Causes shown in red are overrepresented in the media; those in blue are underrepresented. Numbers denote the factor by which they are misrepresented.

The major standout here – I had to break the scale on the y-axis since it's several orders of magnitude higher than everything else – is terrorism: it is overrepresented in the news by almost a factor of 4000.

Homicides are also very overrepresented in the news, by a factor of 31. The most underrepresented in the media are kidney disease (11-fold), heart disease (10-fold), and, perhaps surprisingly, drug overdoses (7-fold). Stroke and diabetes are the two causes most accurately represented."


Image


Image
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Sun Feb 16, 2020 11:26 am

I thought these two article would make a great addition to this thread. I have posted a few highlights but would recommend reading both articles in full.

In Health
Jeff


The Cancer Industry: Hype vs. Reality
Cancer medicine generates enormous revenues but marginal benefits for patients
By John Horgan on February 12, 2020
https://blogs.scientificamerican.com/cr ... -reality/#

SOLUTION: GENTLE CANCER MEDICINE?

Cancer boosters commonly point to improvements in survival rates, the length of time between diagnosis and death. Survival rates for some cancers have indeed grown as a result of more widespread and higher-resolution testing, which detects cancer earlier. But as a 2015 analysis points out, in general people do not live longer as a result of early detection. They simply live longer with a diagnosis of cancer, with all its harmful emotional, economic and physiological consequences.
...

So what’s the solution to all these problems? Some health-care experts espouse “conservative medicine” as a way to reduce health-care costs and improve outcomes. In “The Case for Being a Medical Conservative," a manifesto published last year, four physicians (including the aforementioned Vinay Prasad) urge colleagues to recognize the human body’s “inherent healing properties and to acknowledge “how little effect the clinician has on outcomes.” Physicians will thus protect themselves “against our greatest foe—hubris.”

Medical conservatives happily adopt new therapies “when the benefit is clear and the evidence strong and unbiased,” the authors emphasize, but many alleged advances “offer, at best, marginal benefits.” Conservative cancer medicine, as I envision it, would engage in less testing, treatment, fear-mongering, military-style rhetoric and hype. It would recognize the limits of medicine, and it would honor the Hippocratic oath: First, do no harm.

Physicians cannot bring about a shift toward conservative cancer medicine on their own. We consumers must help them. We must recognize the limits of medicine and the healing capacities of our bodies. We must resist tests and treatments that have marginal benefits, at best. We may never cure cancer, which stems from the collision of our complex biology with entropy, the tendency of all systems toward disorder. But if we can curtail our fear and greed, our cancer care will surely improve.




The Case for Being a Medical Conservative
Mandrola, John et al.
The American Journal of Medicine, Volume 132, Issue 8, 900 - 901

https://www.amjmed.com/article/S0002-9343(19)30167-6/pdf

"The medical conservative, therefore, is pragmatic about human nature and the prevailing business model of medical science. To wit, content experts, professional societies, or jour- nal editors who too harshly criticize an industry product jeop- ardize future funding. Motivating biases need not be considered nefarious, only considered.

In the end, the medical conservative stands in awe of the human body. We recognize that our knowledge and best models only rarely predict the success of a new intervention. We see true medical progress as slow and hard, in large part because nature has provided the human body with inherent healing properties.

The wisest of conservative physicians understand and embrace how little effect the clinician has on outcomes. While many may call this frame of reference nihilistic, the conservative clinician sees it as protective against our greatest foe—hubris."
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Tue Jun 22, 2021 4:35 pm

Quoting the author “We have a new paper out in PNAS today, in which we address the harm wrought by dramatically restructuring human communication of the span of a decade, with no aim other than selling ads. It might be the most important paper of my career.”


Stewardship of global collective behavior
Joseph B. Bak-Colemana,b, et al

Edited by Bonnie J. McCay, Rutgers, The State University of New Jersey, New Brunswick, NJ, and approved May 17, 2021 (received for review December 14, 2020)

https://www.pnas.org/content/118/27/e2025764118

Collective behavior provides a framework for understanding how the actions and properties of groups emerge from the way individuals generate and share information. In humans, information flows were initially shaped by natural selection yet are increasingly structured by emerging communication technol- ogies. Our larger, more complex social networks now transfer high-fidelity information over vast distances at low cost. The digital age and the rise of social media have accelerated changes to our social systems, with poorly understood functional consequences. This gap in our knowledge represents a principal challenge to scientific progress, democracy, and actions to address global crises. We argue that the study of collective behavior must rise to a “crisis discipline” just as medicine, conservation, and climate science have, with a focus on providing actionable insight to policymakers and regulators for the stewardship of social systems.
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Sun Jul 10, 2022 2:55 pm

Op-Ed: When Everyone is an Expert, No One is an Expert.
Tim Holt

https://elpasoheraldpost.com/op-ed-when ... an-expert/

In Health
Jeff
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Re: The Importance of Evidence

Postby JeffN » Thu Jul 20, 2023 6:50 am

Nature
18 July 2023

Medicine is plagued by untrustworthy clinical trials. How many studies are faked or flawed?

Investigations suggest that, in some fields, at least one-quarter of clinical trials might be problematic or even entirely made up

https://www.nature.com/articles/d41586-023-02299-w
User avatar
JeffN
 
Posts: 9412
Joined: Tue Jan 08, 2008 5:56 am

Previous

Return to Jeff Novick, RD

Who is online

Users browsing this forum: No registered users and 7 guests


cron

Welcome!

Sign up to receive our regular articles, recipes, and news about upcoming events.