Via Research Blogging, I came across the blog Thoughts of a Neo-Academic, written by an Industrial/Organizational Psychologist. He has started a 10-part review of a recent special issue of the Journal of General Psychology, which examines the research into violence and video games. Part One is here, and worth a read. I tend to agree with this initial conclusion, that the media (and certain researchers) tend to take advantage of the occasional "big time" story to advance the narrative of violence being caused by video games, when in fact the research has concluded no such general, broad-based finding. I look forward to the follow-up installments.
I really liked this post at Brain Posts, not so much because I have an iPad (because I don't), but because of the idea of developing a comprehensive file library for whatever portable computer one uses. The point made by the author is that, with netbooks and iPads, carrying around a large number of pdf files, some of which can be quite informative (and contain charts, graphs, data,etc.), as well as directly related to one's work. The list provided at the linked post appears quite comprehensive, though geared toward the clinician. I've been developing a library of pdf files for my work laptop, primarily for when I am called to testify on competency or responsibility. It is incredibly convenient to have hundreds of journal articles, all in pdf format, located on the hard drive of a laptop; no more dragging around a bunch of paper copies, just on the off chance you may need to look at one of them. At some point, I may make a list of some available pdf files that are handy to put on your portable computer, but of both clinical and forensic utility. In the meantime, I'm definitely going to take a look at some of the links provided at Brain Posts.
Last week, I posted a few times on the subjects of statistics and research. I acknowledged then that I have been a consumer of research and journal articles, but I have not done much in producing research, especially since completing my dissertation. Coincidentally, last week I also started to read How to Write A Lot, by Paul Silvia, Ph.D. It is a short, direct "how to" guide to increasing one's productivity regarding academic writing.
The book itself is easy to read, both in terms of understanding it, and in its style. There is no effort here to empathize with the difficulties associated with academic writing: Silvia's attidue is pretty much, "Yes, writing is hard, and often boring/unrewarding/painful. But, if you want to do it, rather than thinking about doing it without actually doing it, here's what you need to do." Simple and to-the-point suggestions for being accountable tod increasing productivity are what the book offers.
Right off the bat, in Chapter Two, Silvia hits one nail on the head. He notes people often have trouble getting their writing done because of various "specious barriers," which he describes as superficially legitimate reasons for not writing, but which "crumble under critical scrutiny." His very first specious barrier is one I've often succumbed to: "I can't find time to write." He notes, correctly, that to get anything done in a consistent, productive manner, one needs to schedule time for the task, not find time for it. We schedule time for that which we prioritize: If we want to write more, we need to schedule time to do so, not simply throw writing in with other lesser tasks that we get around to now and then. This little section impacted me right away; since I read it, I've been able to take multiple steps towards two different projects I'd been thinking about for some time, but hadn't actually made a move on. Often, it is not that you don't know something, but that you need it put in your face a certain way, in order to make the information relevant and used. In this case, How to Write A lot has started me on tasks I'd been pushing off. I'm looking forward to reading more.
Classification tests are tests used to separate people into separate groups. In other words, they categorize. This is different than, say, an IQ test, which is primarily designed to provide information regarding one's various cognities abilities. yes, IQ tests can be used to classify, but that is not their primary purpose, nor is that the way their data is presented.
A pregnancy test is more of a classification test - women are separated into two groups, eith "pregnant" or "not pregnant." With a classification test, we want to be able to differentiate between two groups of individuals as much as we can, without making errors in our classifications. For example, a pregnancy test with a fifty percent error rate would be pretty useless - you can get those kinds of results simply by flipping a coin. In addition, we'd have a lot of very upset people who had been wrongly classified.
This post will briefly define "True Positive Rate." Simply, the True Positive Rate (TPR) is the proportion of the group of interest who generate a positive score on the classification test being used. In keeping with the example above, let's say our group of interest is women who are pregnant. We want to develop the best pregnancy test possible, to include an excellent TPR. So, let's say that a positive score on our pregnancy test indicates that the test-taker has scored positively - in this case, the test is saying the person is pregnant. The TPR is the proportion of pregnant women who obtain a positive score when they take the test. In other words, if the TPR is .8, then out of 100 pregnant women who take this test, 80 will test positive.
Is this good? It depends upon multiple factors, including the seriousness of the issue, a comparison to other techniques, etc. It also depends upon the False Positive Rate (FPR), or how often an individual who does not have the condition also scores positive. In the current example, if 100 non-pregnant women are administered the test, and 10 test positive (that is, the test says they are pregnant when they are not), then the FPR is .1.
I doubt these ratios would be acceptable in the case of pregnancy testing, but these types of decisions are made in all sorts of situations, including in psychology. We can also adjust the cut-off scores to increase the TPR or FPR, with an understanding that manipulating the cut-off score to enhance one rate will also impact the other. This can get tricky, and it also depends on various factors. here is one more example.
Let's say there is a new cancer screen, based on blood work. The screen predicts the development of cancer in the next five years. It is cheap ($10), easy to administer, and if positive, one can have a more thorough test (%100 accrate) as a follow up for $1000, as well as a series of procedures to then reduce the likelihood of onset by 50%. In this case, the screen will be set to maximize TPR; we would want to capture as many people who might develop cancer as possible, in order to then administer a readily available follow-up test. Yes, there will be more false positives, but the temporary anxiety over the possibility of having cancer would be overshadowed by being able to capture all of the people who do have the marker, along with the treatment intervention. In this case, we would want as many true positives as possible, and tolerate false negatives it would save lives.
Conversely, let's say someone develops a test that categorizes people as "not college material." Here, we would want the exact opposite, in that we would seek to minimize the incorrect classification of someone not being able to complete college, when they actually would be able to. The risk of false positives in this case carries the more significant impact; it would be better for more individuals to go to college and ultimately not succeed (but having been given the opportunity to try), than to prevent individuals would would benefit from attending college (and successfully graduating) from ever having the chance.
These are just random hypotheticals, made up in order to demonstrate the importance of TPR and FPR. In different situations, each can be the primary ratio. Statistically, the important thing to note is that both measure someone scoring positively for the category being measured, either positive (correctly - TPR) or positive incorrectly - FPR). The terms associated with scoring negative on classification tests will be (hopefully) addressed in another post. For a more technical discussion, start here and here.
In psychology, statistics are studied by most, and disliked by most as well. Rarely will you hear "Awesome! I’ve got Advanced Multivariate Stats this semester!" uttered in graduate schools. The problem for many psychologists and psychology students is simply that they have entered the field to do clinical work, and statistics doesn’t seem all that relevant; it’s sort of like taking a play therapy class if you work in corrections. In some cases, people enter this field because of the interpersonal nature, and non-mathematical nature, of clinical work. For others, math was a negative experience when they were younger, and they have a negative view of it, even though they are now older, and the statistics can be learned in a much more applied than theoretical manner. That is, it’s much harder to grasp the relevance of trigonometry as a teen, than to learn about a particular statistical procedure with an accompanying journal article related to one’s area of interest.
In addition, the advent of statistical software and electronic data gathering in general have rendered statistics and research much more digestible than those dark, pre-computer days. I’m not old enough to remember research before any computers at all, but I do remember stories from grad school, when teachers used to discuss how they would need to create numerous "punch cards" with their data in order to complete calculations, and run the cards through what has been described as precursors to computers; I shudder at the thought, but back then, there was no alternative.
It is this technological advancement that actually makes statistics so much more interesting than before. It’s not only that "doing stats" is easier; as long as you know the right statistics to use, the computer does all the work; the more significant development is that much more complex questions can be asked and answered, due to the wider variety of statistical procedures now available.
Back in the good old days, it seems much multivariate statistical ideas, if they even existed, were theoretical. That’s because the math involved in calculating, by hand, a significant amount of data with these more complicated formulas would take a lifetime. T-tests and, if really bold, ANOVAs were the way to go, if you ever wanted to finish something. But those tests limit the range of ways to examine data. Nowadays, with the processing power available, virtually anyone with access to data can answer questions earlier scientists could have only dreamed about.
Confession: I’ve always been more of a digester of statistics and research, rather than an active researcher. I don’t pretend to even be an expert or specialist in statistics, just someone who finds research important to review, and has occasionally considered crunching my own data at some point (beyond the obligatory dissertation, which was completed long ago). I’m hoping I am at a point where actual data collection related to relevant questions I have is not only available, but possible within the purview of my job. However, until it’s done, there’s really no need to talk about what I "might" do; I just need to do it.
Anyway, for those of you considering the field of mental health, or are already in it in some capacity, there’s no need to shy away from statistics. Yes, sometimes the stuff gets complex, and goes over my head, but understanding the fundamentals can help with understanding research, which can, in turn, pique your interest in your particular clinical, developmental, etc. area. I know it works this way for me.
Over at Marginal Revolution, they've got a post up about the meaning of statistical significance, with links to otherposts on the issue. The post discusses the flaws of relying too heavily on alpha levels for determining significance, the reasons why this is a flawed approach, and the particular problems we have in the social sciences with respect to this issue. I remember discussing this in graduate school to some degree, including discussions about broadening one's knowledge of statistics to utilize other measures such as effect size to gain a better sense of the true significance of a finding. I also recall more than a few issues where there might have been statistical significance found with respect to a study, but the clinical significance was negligible. Here is what I wrote about that issue in the comments section:
We see some of this in psychological testing. For example, in IQ testing, we may assess someone with a gap of, say, 10 points between their verbal IQ and their performance (i.e. nonverbal) IQ. While this gap may be statistically significant (happening with a lack of frequency that, statistically, it reaches significance), it doesn't mean much in terms of someone's intellectual functioning. What is of far more concern is whether a gap between the scores approaches what is called clinical significance - at that point, we may see certain issues resulting from a gap that large (say, roughly, 20+ points difference off the top of my head).
In other words, just because something reaches statistical significance (at the .05 level) doesn't mean it is actually telling you anything useful, even if it is infrequent.
I think you sometimes run up against this as you gets tons of data on a particular issue, where the effect size gets bigger and bigger: you may get statistically significant differences for groups that really don't tell you much. As an example (and to stay with IQ), I remember that men outscored women on the nonverbal section by a couple points (on average), while women outscored men on the verbal section by a couple of points. Given the thousands of data sets that were utilized in conducteing such large studies, a difference of 103 versus 100 on Verbal IQ came out as statistically significant, even though a difference of three points is clinically meaningless.
If you are interested in statistics and research, I recommend checking out the links...