RG: Do we actually need (or understand) more than basic statistics?

Link to the original Q&A thread at ResearchGate

This is another topic worthwhile looking at, and especially thinking about. I copy here, my answer, that is to some extent off-topic (you will need to follow the link above to read the original post and other answers):

Frequently students that I have supervised, seem to think that statistical tests come first, rather than being a source of guidance on how far we can stretch the inferences that we can make by “looking at the data” and derived summaries. They just describe effects as statistically significant or not. This results in very boring “results” sections lacking the information that the reader wants to know. When I read a paper I want to know the direction and size of an effect, what patterns are present in the data, and if there is a test, then statistical tests should help us decide what amount of precaution we need to use until additional evidence becomes available. Many students and experienced researchers which “worship” p-values and the use of strict risk levels ignore how powerful and important is the careful design of experiments, and how the frequently seen use of “approximate” randomization procedures or the approach of repeating an experiment until the results become significant invalidate the p-values they report.

[edited 5 min later] As I read again what I wrote it feels off-topic, but what I am trying to say is that not only the proliferation of p-values and especially the use fixed risk levels, but also many times how results are presented, is the reflection of a much bigger problem: statistics being taught as a mechanical and exact science based on clear and fixed rules. Oversimplifying the subtleties and degree of subjectivity involved in any data analysis, especially in relation to what assumptions are reasonable or not, and how any experimental protocol relates to which assumptions are tenable or not, is simply not teaching what would be the most useful training for anybody doing experimental research. So, in my opinion, yes we need to understand much more than basic statistics in terms of principles, but this does not mean that we need to know advanced statistical procedures unless we use them or assess work that uses them.