On Social Science Research

“Research” is a funny animal. What passes as social science research can be fascinating, funny, tragic, or maddening depending on its quality and presentation. To ensure high quality, there are standards researchers are ethically bound to observe which serve as guardrails to ensure they do not fly off track into the abyss of speculation. 

One of those is transparency. Researchers are expected to explain their methods in such a way that any other researcher could do the same study, using the same research methods, with a reasonable likelihood of getting the same results. Now, if they do, and they do not get the same results, it does not mean the original research was wrong. It could mean that there were important differences in how the method was used produced a different result. If it is determined that the methods were the same and used in the same way but with different results, it is time to consider other possibilities for the difference, including bias. Transparency makes it possible for researchers to keep themselves, and one another, honest.

Flowers for Fun! From my Saturday walk.

Another guardrail is integrity. An important aspect of integrity in research is the commitment to show the whole truth of the findings, even the inconvenient truth. Nobody is free of bias. Each researcher goes into a project with the expectation, or even hope, of finding certain results. However, we are disappointed as often as we are rewarded. Research results are usually a “mixed bag.” Some results we are pleased with and others we wish had not appeared. However, integrity means we are ethically bound to report the good, the bad, and the ugly. 

A third guardrail is peer review. An important function of professional peer-reviewed journals is to provide a rigorous review of research before it is published for the world to see. The peer review process means that a panel of other researchers examines the work of the author and determines the research and presentation is of sufficient quality to appear in the journal. The PhD dissertation process is essentially a very intense peer review process. A committee examines you at every step of the process and determines whether, and when, you are ready to present and defend your research. It is more than a rite of passage for the PhD, it is a pragmatic rehearsal for a process researches will go through again and again in their postdoctoral careers. 

The fourth guardrail has to do with research data. There are basically two considerations regarding data which researchers need to give attention to for producing solid research. One is the quantity of data and the other is the quality of data. Both are important whether the researcher’s methodology orientation is as a quantitative researcher or as a qualitative researcher. The standards are slightly different for each, but the concern for excellent data is common to both. The quantitative researcher, who is concerned with collecting and analyzing numerical data, usually needs lots of data (numbers) from reliable sources. The qualitative researcher, who is concerned with collecting and analyzing non-numerical data (e.g., interviews, text, videos, etc.) goes in-depth with fewer sources but those sources need to be highly reliable and verifiable. 

Here are two research secrets: Google and Wikipedia. Of course, if you cite either Google or Wikipedia in a school paper, you are likely to lose points and rightly so. Neither is considered a primary source. Researchers try very, very hard to use primary sources. So why do so many researchers go to Google and Wikipedia first? Because you can get a general overview of the topic and each leads you to primary sources. The links and citations in both will usually take you back to primary sources, if you are willing to patiently explore them. 

If you have read many of my blogs you will notice that I often include links to Wikipedia. If it is not a primary source, why do I link to it? Because, in most cases, I am simply trying to give the reader a basic understanding of the language, term, or concept. More often, though, I try to include the primary sources of information I am sharing. 

By this time you are probably wondering “What the heck does this mini-course in research have to do with anything?” Let me explain. 

I have been reflecting on my process of examining White Evangelical support for Donald Trump. I am using qualitative methods hence I am collecting and analyzing text, video, and audio. I had hoped to share more by this time but it is a slow process. Before I say much more about the topic, I want to feel confident that I am offering solid information and ideas.  

I do not have the benefit of a formal peer review process – though I do try to talk through some of my findings with friends and colleagues who are also researchers. When a peer review process is not readily available, it is even more important to have multiple, verifiable primary data sources. This takes time because you want to have the highest quality sources you can.

This is a similar challenge faced by journalists. A good example, from the news of last week, is Jeffrey Goldberg’s article in The Atlantic concerning Trump’s denigration of veterans and the military. Goldberg has multiple primary sources for his article. Unfortunately, they are all anonymous hence they are not verifiable. Trump, therefore, has been pushing back on the article claiming he never said any of the things he is accused of saying in the article. Of course, he has also been waving his “fake news” wand over it too. What Goldberg needs is for his anonymous sources to step forward and claim their words in the article. The more the better but even if two or three of the key anonymous informants would own their words, it would dramatically tip the scales. As I wrote earlier, quantity and quality of data matters. Goldberg appears to have more than enough primary sources (quantity) but they are far more credible if they are not anonymous (quality). 

I wanted to give you an update on the process and progress on the White Evangelical research. I am hoping to have more on the topic soon. Stay tuned! 

A Message from Canada

My friend Liz from Canada sent me a Tweet that included a link to a very funny video but, unfortunately, I did not see the video. I thought she was just reminding me to vote in November and so I replied back, “Thanks! We’ll be voting for sure, Liz!” I think she figured out that I had not actually watched the video so she followed up with another Tweet that said simply, “It a fun video.”

I read that Tweet and immediately thought, “Video? What video?” So I dug into my Twitter feed to find the video she had sent me and then I found it on YouTube so I could embed it here. It is a really fun and funny video! Please take a 1 minute and 50 seconds to watch it.

Author: The Driveler

Tom Klaus is the Driveler. On March 16, 2020, the first day of the Novel Coronavirus shelter-in-place order for his state, he started writing a daily blog to keep himself from stressing too much about the pandemic situation. He thought the daily blogging would last for only a couple of week but it stretched on to 77 consecutive days. Then he continued writing daily for a while after that as well. At some point the blog became The Daily Drivel...mostly because he was mostly writing the stream of consciousness drivel that was pouring out of his head, running down his face, and, sometimes, out of his mouth. In November 2020 he launched The Daily Drivel as a free-standing website/blog.

3 thoughts on “On Social Science Research”

  1. Another very important guardrail for primary research is Human Subjects Review by an Institutional Review Board. Yes, it is a form of peer review, which you mentioned, but it is not post hoc, as is most peer review, and it includes very specific rules designed to protect research involving humans. Anyone conducting such research is required to have a current certificate that verifies they have completed the necessary training.

Leave a Reply

%d bloggers like this: