Classification tests are tests used to separate people into separate groups. In other words, they categorize. This is different than, say, an IQ test, which is primarily designed to provide information regarding one's various cognities abilities. yes, IQ tests can be used to classify, but that is not their primary purpose, nor is that the way their data is presented.
A pregnancy test is more of a classification test - women are separated into two groups, eith "pregnant" or "not pregnant." With a classification test, we want to be able to differentiate between two groups of individuals as much as we can, without making errors in our classifications. For example, a pregnancy test with a fifty percent error rate would be pretty useless - you can get those kinds of results simply by flipping a coin. In addition, we'd have a lot of very upset people who had been wrongly classified.
This post will briefly define "True Positive Rate." Simply, the True Positive Rate (TPR) is the proportion of the group of interest who generate a positive score on the classification test being used. In keeping with the example above, let's say our group of interest is women who are pregnant. We want to develop the best pregnancy test possible, to include an excellent TPR. So, let's say that a positive score on our pregnancy test indicates that the test-taker has scored positively - in this case, the test is saying the person is pregnant. The TPR is the proportion of pregnant women who obtain a positive score when they take the test. In other words, if the TPR is .8, then out of 100 pregnant women who take this test, 80 will test positive.
Is this good? It depends upon multiple factors, including the seriousness of the issue, a comparison to other techniques, etc. It also depends upon the False Positive Rate (FPR), or how often an individual who does not have the condition also scores positive. In the current example, if 100 non-pregnant women are administered the test, and 10 test positive (that is, the test says they are pregnant when they are not), then the FPR is .1.
I doubt these ratios would be acceptable in the case of pregnancy testing, but these types of decisions are made in all sorts of situations, including in psychology. We can also adjust the cut-off scores to increase the TPR or FPR, with an understanding that manipulating the cut-off score to enhance one rate will also impact the other. This can get tricky, and it also depends on various factors. here is one more example.
Let's say there is a new cancer screen, based on blood work. The screen predicts the development of cancer in the next five years. It is cheap ($10), easy to administer, and if positive, one can have a more thorough test (%100 accrate) as a follow up for $1000, as well as a series of procedures to then reduce the likelihood of onset by 50%. In this case, the screen will be set to maximize TPR; we would want to capture as many people who might develop cancer as possible, in order to then administer a readily available follow-up test. Yes, there will be more false positives, but the temporary anxiety over the possibility of having cancer would be overshadowed by being able to capture all of the people who do have the marker, along with the treatment intervention. In this case, we would want as many true positives as possible, and tolerate false negatives it would save lives.
Conversely, let's say someone develops a test that categorizes people as "not college material." Here, we would want the exact opposite, in that we would seek to minimize the incorrect classification of someone not being able to complete college, when they actually would be able to. The risk of false positives in this case carries the more significant impact; it would be better for more individuals to go to college and ultimately not succeed (but having been given the opportunity to try), than to prevent individuals would would benefit from attending college (and successfully graduating) from ever having the chance.
These are just random hypotheticals, made up in order to demonstrate the importance of TPR and FPR. In different situations, each can be the primary ratio. Statistically, the important thing to note is that both measure someone scoring positively for the category being measured, either positive (correctly - TPR) or positive incorrectly - FPR). The terms associated with scoring negative on classification tests will be (hopefully) addressed in another post. For a more technical discussion, start here and here.
In psychology, statistics are studied by most, and disliked by most as well. Rarely will you hear "Awesome! I’ve got Advanced Multivariate Stats this semester!" uttered in graduate schools. The problem for many psychologists and psychology students is simply that they have entered the field to do clinical work, and statistics doesn’t seem all that relevant; it’s sort of like taking a play therapy class if you work in corrections. In some cases, people enter this field because of the interpersonal nature, and non-mathematical nature, of clinical work. For others, math was a negative experience when they were younger, and they have a negative view of it, even though they are now older, and the statistics can be learned in a much more applied than theoretical manner. That is, it’s much harder to grasp the relevance of trigonometry as a teen, than to learn about a particular statistical procedure with an accompanying journal article related to one’s area of interest.
In addition, the advent of statistical software and electronic data gathering in general have rendered statistics and research much more digestible than those dark, pre-computer days. I’m not old enough to remember research before any computers at all, but I do remember stories from grad school, when teachers used to discuss how they would need to create numerous "punch cards" with their data in order to complete calculations, and run the cards through what has been described as precursors to computers; I shudder at the thought, but back then, there was no alternative.
It is this technological advancement that actually makes statistics so much more interesting than before. It’s not only that "doing stats" is easier; as long as you know the right statistics to use, the computer does all the work; the more significant development is that much more complex questions can be asked and answered, due to the wider variety of statistical procedures now available.
Back in the good old days, it seems much multivariate statistical ideas, if they even existed, were theoretical. That’s because the math involved in calculating, by hand, a significant amount of data with these more complicated formulas would take a lifetime. T-tests and, if really bold, ANOVAs were the way to go, if you ever wanted to finish something. But those tests limit the range of ways to examine data. Nowadays, with the processing power available, virtually anyone with access to data can answer questions earlier scientists could have only dreamed about.
Confession: I’ve always been more of a digester of statistics and research, rather than an active researcher. I don’t pretend to even be an expert or specialist in statistics, just someone who finds research important to review, and has occasionally considered crunching my own data at some point (beyond the obligatory dissertation, which was completed long ago). I’m hoping I am at a point where actual data collection related to relevant questions I have is not only available, but possible within the purview of my job. However, until it’s done, there’s really no need to talk about what I "might" do; I just need to do it.
Anyway, for those of you considering the field of mental health, or are already in it in some capacity, there’s no need to shy away from statistics. Yes, sometimes the stuff gets complex, and goes over my head, but understanding the fundamentals can help with understanding research, which can, in turn, pique your interest in your particular clinical, developmental, etc. area. I know it works this way for me.
Having owned a Nook since February (an acquisition worth a blog post in its own right), I became interested in what sort of books are available in e-book format, particularly older books. I’m often surprised (and frustrated) by what newer releases aren’t available electronically, but less so for older books, which probably don’t sell as well, and are therefore low on the priority list.
It was during one such ten minute period of random searching that I started entering a few of my favorite authors of my youth. This included Elizabeth Boyer, who wrote several books I very much enjoyed from about 6th grade on. In particular, The Elves and the Otterskin was re-read numerous times during a period in which I read as much heroic fantasy as I could my hands on.
During those days, getting these types of books was much harder than today, especially for pre-teens and younger teens. Basically, you went to the mall when you had a chance, and perused the three or four bookshelves dedicated to Science Fiction (mostly) and Fantasy (shoved in here and there). Tolkien was always available, obviously, and terry Brooks was starting to take up space, but there were relatively few other choices.
Though not her first book, The Elves and the Otterskin was the first I read. Elizabeth Boyer wrote fantasy that was heavily influenced by Scandinavian culture and folklore. When younger, I had some initial difficulty tracking the various names and places, but much like watching a film with a heavy accent, you catch up with it after a bit. There is no denying Boyer’s talent as a writer. Character development, setting, pacing, plotting - the four books of hers I read (her first four) were all well-written, at least from what I can remember. In addition, the sense of humor found in her books was superior to most books in this genre - reading her books, you frequently find yourself chuckling.
When I saw that her books were not available on the Nook (no, I really didn’t expect that they would be), I did a regular online search. Sadly, I saw that her books are all out of print, though they are available used. I also noticed via Wikipedia Ms. Boyer ended up publishing 11 books, much more than I had realized. She published from 1980-1995, when she apparently stopped writing (or at least publishing). There was a three-year break in her books from 1983-1986, which means I had probably moved on at that point, after reading her initial four books. Wikipedia notes that she was an early pioneer of "mass-market fantasy," but her specialized writing prevented her from achieving a wider acceptance.
After this bit of research, I scavenged the basement, looking through old paperbacks I’ve kept over the years. The collection gets smaller and smaller, but I did turn up my old copy of The Elves and the Otterskin! None of the others remain, however. I read the book again for the first time in at least 20 years, and it has held up remarkably well. Older now, I can appreciate more the Scandinavian influences and how they enhance the story. This is a book I will definitely encourage my kids to read once they’re a bit older. In the meantime, I may "need" to go on a bit of a quest of my own, to see if I can track down some of her other books, both the early ones I read as a kid, and the later ones I am unfamiliar with.
Poking around the Internet, there isn't much about Boyer. Several short biographies note she attended BYU, and that she moved to a farm near Atlanta after her writing career. A short bio can be read here and here. Also, the reviews of The Elves and the Otterskin on Amazon pretty much say it all, though you can read another description here, that also has a different cover image than my copy. If I end up reading anything else by Ms. Boyer, I'll write a follow-up post. In the meantime, I'll simply note that I loved her books growing up, and only wish she were still publishing.
"The effects of chronic methamphetamine use are numerous and often pronounced. Brecht, O'Brien, von Mayrhauser, and Anglin (2004) found that in a sample of 350 individuals recruited from a publicly funded treatment site, 84% reported weight loss because of their methamphetamine use. Other problems included sleeplessness (78%), financial problems (73%), paranoia (67%), legal problems (63%), hallucinations (61%), work problems (60%), violent behavior (57%), dental problems (55%), and skin problems (36%). Such consequences of continued use of the drug result from the stress the drug places on the physical being of the individual. Also, behavioral outcomes associated with methamphetamine use in terms of impaired cognitions and emotions place additional stress on the homeostasis and well-being of the human organism. In one extreme case, a 44-year old White man bisected his penis in a methamphetamine-induced psychosis (Isreal and Lee, 2002)."
- From pages 50-51 of Methamphetamine Addiction, by Perry N. Halkitis
One last Friday post snuck in! I chose this topic mainly because we are seeing just a ton of guys for evaluations who's past functioning has been chronically impaired by methamphetamine use, so we're really having to study up of the various symptoms associated with methamphetamine use, withdrawal, etc. The other reason is because I am preparing to launch a Breaking Bad marathon within the next couple of weeks, courtesy of receiving Seasons 1 and 2 as a birthday present. Nothing like some light, fluffy entertainment...
I've been using the WAIS-IV exclusively for about nine months, which I think has been long enough to offer a few brief comments on it, as compared to prior versions of the WAIS, as well as tests in general.
Generally speaking, I really like using the WAIS-IV, compared to previous versions. Administration consists of 10 primary subtests, which have been streamlined in terms of discontinuation rules. For example, Block Design is discontinued after two consecutive scores of zero, Matrix Reasoning after three consecutive zeroes, etc. In addition, with each new version of the WAIS, the subtests have gravitated towards ones with easier-to-score items; this version, for example, has relegated Comprehension to "Alternate" staus, and there was much rejoicing. The subtests themselves have been improved in terms of their introduction, as more sample items are provided to give direction to individuals who are unfamiliar with this type of task, or who may have difficulty grasping what they are supposed to do - I believe this has addressed a prior concern of mine; that individuals who already have relatively low intellectual functioning may have needed longer to simply grasp what the tasks entialed, possibly hurting their score.
Conceptually, the test has adopted the four-factor model, rather than the old two-factor model. Now, Processing Speed and Working memory are affored their own spot at the table, and all subtests load separately on the various factors, rather than having overlap. Given that many clinicians reported the four factors anyway, this formalized shift is nice.
Overall, I've been really happy with the latest version of the WAIS-IV. It addressed a few minor concerns I had with the WAIS-III, is streamlined, and relatively quick in terms of administration time.
Interesting article here that reviews research into the assessment of ADHD. The article covers the details, with the main concern being that the youngest children in early grades are being excessively diagnosed (and subsequently medicated) due to comparisons with the older students in the class. Given there is no biological marker for ADHD, and the diagnosis requires the symptoms be present in at least two settings, teachers are often asked to provide their observations of children in the classroom. The difficulty can be when the youngest child in a class such as kindergarten or first grade is compared against the other children, who are older and thus more developed, in terms of attention, impulsivity, etc., the very areas of concern with respect to ADHD. From the article:
According to Elder's study, the youngest kindergartners were 60 percent more likely to be diagnosed with ADHD than the oldest children in the same grade. Similarly, when that group of classmates reached the fifth and eighth grades, the youngest were more than twice as likely to be prescribed stimulants.
The article does not mention gender, but this can also be a factor. If someone compares a just-turned five year old boy against a six-year old girl, they may well appear to be from different planets when it comes to the diagnostic issues related to attention and behavior. However, these comparisons are not appropriate; the evaluator should be examining the child in the context of other children at a similar developmental stage. As adults, the groups are often distinguished by decade. In children, however, differences of as little as three months can have a significant impact. Clinically, it would make sense to consider ADHD only if the youngest of a class were displaying symptoms of ADHD relative to the other younger students, not the class as a whole.
Interestingly, I read somewhere a theory that older boys do well in school, relative to their younger classmates, for an indirectly related reason - sports. Athletic prowess is of particular importance in the perception of a boy's success in school, and at early ages, older boys tend to have a natural biological advantage; they are not only likely to be bigger, stronger, etc., but also more coordinated, and able to focus longer. It would be interesting to examine how the early athletic success afforded the older boys in second, third, fourth grade, etc. relative to the youngest boys of each class (who are, at times a year or more younger, depending on various admission factors), shapes their later attitude towards sports. I would imagine it would be hard to develop a dominating athletic mentality at age 18, when things have evened out (maybe the student is even bigger than his peers), but during those early years, that student was always the least athletic, simply because they were younger and lagged behind their classmates during those early, formative years. Who knows?
Maybe that's what accounts, at least partially, for some of these college and pro athletes who, by all accounts, have "all the physical tools," but simply can't manage to dominate, despite the physical advantages; maybe in their head, they are still smaller than everyone else...David Robinson comes to mind. A late bloomer anyway, he seems like a wonderful guy, with many intellectual and artistic interests. But that's just it - I wonder if he would have truly dominated the NBA if he had developed a Jordanesque attitude, and I wonder if he would have done just that, if he'd been kicking everyone's butt in basketball since an early age. A less well-rounded individual, but a more dominating NBA career. I'm guessing, since he attended Navy, he wasn't the dominating presence in high school his later physique suggested, and in the NBA, he had a really good career, but not what was expected (just ask Bill Simmons). As much as anything, David Robinson didn't dominate more because of his mental approach to the game; clearly, it wasn't his physical skills. Was he a younger guy in class growing up? I don't know, but I'm guessing going from 6'7' to 7'1" after high school hurt his ability to play center in the NBA - he simply wasn't able to develop a center's mentality early enough. Okay, how did I get from ADHD in kindergarten to David Robinson? Oh, that's right - maybe being the smallest guy on the team from the age of 6-12 sours you on sports, or at least impacts your self-view, even when you eventually turn out to be just as big and strong as everyone else.
For that matter, going back to the original article, how much does being diagnosed impact one's self-view related to academics, if you are behind simply because you are the youngest in the class? Interesting stuff...
At the blog Epiphenom, the author of this post reviews recent research into the issues associated with the treatment of delusions containing religious elements. The original article, by Sylvia Mohr and others (University Hospital in Geneva, Switzerland), examined the particular benefits and challenges of working with clients who experience delusions with religious themes. Briefly, delusions, are erroneous beliefs that involve a misrepresentation of perceptions or experiences (as defined in the DSM-IV-TR). Delusions can encompass a variety of themes, with religion being one type. Delusions are a psychotic symptom, and as such, they indicate a break from reality; that is, one is not delusional if he or she is simply mistaken about something. Delusions are a belief system , usually centered on a theme, that involves a distortion of reality. Often, delusional beliefs are theoretically possible, but not grounded in actuality. For example, an individual may believe the CIA has targeted him for assassination, due to some special feature he believes he possesses (certain knowledge, etc.). Could the CIA do this, theoretically? Sure. Are they? Generally, no. Bizarre delusions, on the other hand, are beyond possibility: for example, the belief all of your organs have turned to stone. In this case, not possible. It is helpful to consider these types of beliefs on a continuum, because there can be gray areas - where does suspiciousness end, and paranoia begin? When does something go from far-fetched to impossible? These can be tricky questions, and they often require exploration.
Anyway, back to the original point. Religious delusions often center around a belief one has some special relationship with a religious figure (often God, but also others, such as angels or demons). More extreme beliefs include a belief one is a major historical figure. The link above explores why in some cases, the nature of a religious delusion provides comfort in a way other delusional themes do not, but also why religious delusions may inhibit treatment more than other types of delusions.
As I've posted before (I think), delusions can be difficult to treat. In the case of a delusional disorder (generally absent any other psychotic symptoms, such as disorganization, hallucinations, etc.), an individual will rarely seek out treatment voluntarily, and will typically refuse medication. If the delusion is part of a Schizophrenic presentation, it is the delusion that is often the most resistant to treatment. The delusional beliefs will be the last to respond to medication, and take the longest to abate. When one considers the nature of a delusion, this is understandable. You are basically asking someone to stop believing what they think they know.
Heck, have you ever argued politics with someone? People rarely give up on beliefs, and if they do, it is generally of their own accord, not because someone else told them they are wrong. Imagine if, starting today, everyone began telling you the sky was green. You look outside, and it's still blue. But, everyone else says it's green, and they start to criticize you, patronize you, recommend you get your eyes checked, etc. What would you think? This is the perception of an individual suffering from a delusion - to them, the evidence is clear, and you are the one who is wrong. To the extent research like this further aids in identifying specific isues and recommendations for various types of symptoms, rather than a one-size approach, the better our interventions will become. Read the whole thing.
I recently read an article published in the August 2008 edition of Psychology, Public Policy and the Law, entitled "Clinician Variation in Findings of Competence to Stand Trial" (find abstract here). The authors of the study (Daniel Murrie, Marcus Boccaccini, Patricia Zapf, Janet Warren, and Craig Henderson) note that while most research indicates 20-30% of defendants evaluated for competency to stand trial are found incompetent, there has been little research examining individual patterns with respect to forensic evaluators. That is, do most evaluators tend to cluster within the 20-30% finding, or is there a large spread with lots of outliers. As the authors state, this is extremely important for the integrity of the legal process. While issues related to mental health as it applies to the law can be quite complex, there should be at least a fair amount of reliability associated with the process of evaluating defendants. The purpose of evaluating a defendant for competency to stand trial is to ensure that the defendant is receiving a fair trial, and to reduce the possibility of an erroneous legal finding. As such, the concept of competency is defined, and there should not be excessive variability in findings. Obviously, in the more difficult and complex cases there may be disagreements, but as the article notes, what we don’t want is the result of an evaluation to boil down to who is doing the examining. The authors review the research to this point, which indicates that, generally speaking, overall findings regarding competency are reliable. Still, the lack of data regarding individual variability prompted their study.
In this study, the authors examined the findings of competency evaluations in two states: Virginia (55 clinicians total) and Alabama (5 total), with a different research focus for each group. They discussed the professionals allowed to conduct evaluations (psychiatrists, psychologists, and social workers, primarily), and they reported other pertinent facets as well. For example, both states require completion of a specialized training course prior to be allowed to conduct these evaluations. In addition, the authors limited the evaluators examined in both states to those who had completed at least 20 evaluations, in order to prevent small sample sizes from skewing the results (as well as ensuring that the evaluators being reviewed had sufficient experience in this area, and to allow for an examination of patterns for individuals).
Without going into extensive detail regarding the statistics used, I’ll just point out a few of the findings I found interesting. Within the Virginia sample, there appeared to be significant variability with respect to individual clinician findings. The authors note that almost half of the evaluators had rates of incompetence findings within 10-30%, but that means more than half of the clinicians found defendants incompetent at a rate of either below 10% (18 out of 55, or 32.7%), or above 30% (18.2%). Even when the sample was reduced to clinicians who’ve completed over 100 evaluations, there was considerable variation. In particular, one evaluator who conducted 20 evaluations did not have a single finding of incompetency, while three evaluators had rated higher than 50% (all with 20+ evaluations conducted.
The article also examines 15 evaluators in Virginia who had each completed at least 100 evaluations. The mean rate of this group was 16.1%, but there was also variability within this more experienced group. Three of these evaluators had rates below 7%, while three others had rates above 25%. One evaluator with over 300 evaluations completed had found only 4.2% of defendants incompetent.
The Virginia sample also revealed differences in competency rates based on the evaluator’s profession. Specifically, four social workers were included in the study; their rate for finding incompetence was 46.1%). Conversely, nine psychiatrists were in the study: 7 had rates below 8%, 1 had a rate of 20%, and 1 had a rate of 62.5%. Psychologists fell in between these two other groups, with a mean rate of 20%.
The authors statistically examined the variance in the sample, in order to assess whether/how much of the differences between these various evaluators (and their rates of finding incompetence versus competence) was due to the evaluators themselves, and not due to other sources of variance. The authors note:
"These other sources of variance might include differences in evaluator training, methods used to conduct competence evaluations, party requesting the evaluation (prosecution or defense), individual differences among the defendants who were evaluated, or other sources of systematic or random error. An ideal study would be designed so that all of these potential sources of error could be estimated."
Statistical analyses indicated a significant amount of variance in these differing rates due to differences among the evaluators, above and beyond the other sources noted above. The proportion of the variance attributable to evaluators was calculated to be 12.1%.
In another model, evaluator profession was found to be a statistically significant predictor of competence/incompetence findings. Social workers were found to be 3.51 times more likely to find a defendant incompetent than a psychologist; psychologists were 2.04 times more likely to find a defendant incompetent than a psychiatrist.
Another finding in the article was that the presence of psychosis in the defendant’s presentation, in that it appeared that at least some evaluators were equating psychosis with incompetence. Due to the specifics of the available data, the authors examined the findings of evaluators who had completed at least 100 evaluations. They found that with the exception of the two Virginia psychiatrists, all evaluators found defendants with a psychotic disorder incompetent at a rate higher than 25%. On average, the 11 Virginia psychologists found 39.4% of defendants with a psychotic disorder incompetent. In Alabama, both evaluators found defendants with a psychotic disorder incompetent at a rate higher than 50%.
The authors note multiple possibilities to further explain some of the discrepancies in evaluator incompetence rates, including referral sources (i.e. inpatient versus outpatient), system characteristics (for example, if one correctional facility has particularly strong mental health services, their defendants might be found competent more frequently due to better treatment), and professional discipline. Various issues are raised in the discussion section of this article to further theorize upon the variance issue.
Overall, I found this article to be well considered in the questions it raised. On a macro level, the article points to the ongoing issue of variance, even within an area of mental health that practitioners want accuracy. As the authors point out, the Court's decision regarding a defendant's competence should not be simply the result of which evaluator the defendant is assigned. More individually, the article reminds clinicians they ought to be mindful of any unusual patterns they develop in the course of their work, with the first step being self-awareness. That is, if an individual clinician is arriving at a pattern of conclusions outside of the norm, then he or she ought to consider why. If the explanation is reasonable, fine. However, a lack of awareness can often lead to, at the very least, a blind spot with respect to one’s clinical work. In this case, if an evaluator has conducted 100 competency evaluations, and has found 50 of the defendants incompetent, there is a legitimate question to answer, given that 20-30% is the generally accepted norm. Is the reason based on particular circumstances? It could be; maybe you only evaluate individuals who have a particular (and significantly debilitating) condition. If not, why are your findings outside of the norm? A reasonable question to periodically ask, regardless of one’s profession.