{"id":39672,"date":"2023-10-20T15:03:46","date_gmt":"2023-10-20T15:03:46","guid":{"rendered":"http:\/\/startupsmart.test\/2023\/10\/20\/how-to-spot-dodgy-science-in-minutes-startupsmart\/"},"modified":"2023-10-20T15:03:46","modified_gmt":"2023-10-20T15:03:46","slug":"how-to-spot-dodgy-science-in-minutes-startupsmart","status":"publish","type":"post","link":"https:\/\/www.startupsmart.com.au\/uncategorized\/how-to-spot-dodgy-science-in-minutes-startupsmart\/","title":{"rendered":"How to spot dodgy science in minutes – StartupSmart"},"content":{"rendered":"
I haven\u2019t got time for science, or at least not all of it. I cannot read 9000<\/a> astrophysics papers every year. No way.<\/p>\n And I have little patience for bad science, which gets more media attention than it deserves.<\/p>\n Even the bad science is overwhelming. 700 papers are retracted annually<\/a>, and that\u2019s a gross underestimate of the bad science in circulation.<\/p>\n I, like most scientists, filter what I read using a few tricks for quickly rejecting bad science.<\/p>\n Each trick isn\u2019t foolproof, but in combination they\u2019re rather useful. They can help identify bad science in just minutes rather than hours.<\/p>\n Good science is often meticulous and somewhat anxious. You discover something new or find something unexpected, and frankly you worry a lot about screwing up.<\/p>\n Identifying and addressing what could plausibly go wrong, and then writing that up succinctly, takes time. Lots of time. Months. Even years.<\/p>\n If you\u2019re taking the time to do meticulous science, why not take the time to prepare a good manuscript?<\/p>\n Make nice-looking figures, proofread it a couple of times, and the like. It seems obvious enough, which is why a sloppy manuscript or poor grammar can be a warning sign of bad science.<\/p>\n Recently, Ermanno Borra and Eric Trottier<\/a> claimed to have detected \u201csignals probably from extraterrestrial intelligence<\/a>\u201d.<\/p>\n I thought this was far-fetched, but still worth looking at the paper preprint<\/a>. An immediate red flag for me was some blurry graphs, and figures with captions that weren\u2019t on the same page.<\/p>\n<\/p>\n Was my caution justified? Well, as I dug into the paper more there were other warning signs.<\/p>\n For example, the results relied on Fourier analysis, a mathematical method that can be powerful but is also notorious for picking up artefacts from scientific instruments and data processing<\/a>.<\/p>\n Furthermore, the surprising conclusions relied on a tiny subset of data, and there was no attempt to confirm<\/a> the conclusions with additional observations.<\/p>\n If they were being meticulous, wouldn\u2019t they have taken the time to collect more data and properly format their manuscript?<\/p>\n I\u2019m very sceptical of Borra and Trottier\u2019s aliens, as are many of my colleagues<\/a>.<\/p>\n Of course, there are exceptions to good-looking good science. The announcement of the Higgs boson, which featured fantastic science, included slide designs that did not impress Vincent Connare, the creator of Comic Sans.<\/p>\n<\/p>\n To be honest, I\u2019m with Connare on the slides<\/a>. However, this is a reminder that tricks for quickly flagging bad science are imperfect shortcuts, not absolute rules.<\/p>\n \u201cThat\u2019s obvious, why didn\u2019t someone think of that before?\u201d<\/p>\n Well, perhaps someone did.<\/p>\n It has recently been claimed that the expansion of the universe may not be accelerating<\/a>, which seems at odds with some Nobel Prizewinning research<\/a>.<\/p>\n That claim relies on a statistical analysis of supernova data. However, such analyses are nothing new.<\/p>\n Enter keywords into a search engine<\/a>, and you find many previous studies, but without the unexpected conclusions. That\u2019s a red flag right there.<\/p>\n So what happened? Well, one can study up on supernovae or cosmology, but there are experts on Twitter providing succinct explanations and informed responses.<\/p>\n<\/p>\n<\/p>\n In plain English, you can only claim that evidence for the accelerating expansion of the universe is marginal if you make make incorrect assumptions about supernova properties and brush aside other key observations. And thus facepalm.<\/p>\n Cosmologist Tamara Davis has noted<\/a> that such omissions, accompanied by emphasis on a contrarian conclusion, tend to be misleading spin.<\/p>\n Unfortunately, such omissions and erroneous assumptions turn up elsewhere too<\/a>.<\/p>\nOkay, this looks bad<\/h3>\n
Obviously<\/h3>\n