I first learned about this study around the time it was presented at the conference. I'd been following TRI's research for years and was at first shocked and then curious. Coincidentally, shortly before I head of it I'd run across some studies myself on PubMed that showed little or no change in cortisol and, in one case, a rise in cortisol levels. I thought that curious but ignored it at the time. 

My initial reaction was, "Whaaat!?!?!? What about all those TRI studies?" I was able to get a copy of the study and, since I don't understand statistics, didn't understand it but I got the general idea. When the videos came out, it helped me to understand the difference between in group change and between group change. After that, I wondered why TRI had not taken that into consideration?

The study not only taught me about different types of comparisons and meta-analyses, but it also reinforced the importance of not being attached to particular ideas. 

 

Coincidentally, shortly before I head of it I'd run across some studies myself on PubMed that showed little or no change in cortisol and, in one case, a rise in cortisol levels.

Interesting, Alice--I will look for them.

The topic seems to be of enough general interest that it could be the first CAT here. I am designing a poll to find out which subjects are of more interest that others to know where to begin, but massage and cortisol are pretty clearly of wide interest.

Also, we need to remember that "stress" is any demand made on the body, including demands we perceive as positive.  Massage itself is a stressor, and as such it might even cause cortisol to rise in some situations, even as we perceive the massage to be pleasant and beneficial.

Excellent point, Chris. We often use the word "stress" in the sense of bad stress only.

Hans Selye, the endocrinologist who laid the foundation for the study of the stress response, distinguished between eustress (good) and distress (bad). That's a very useful distinction to keep in mind, and we often blur over it by just saying "stress".

Selye's conceptualization of stress is the one I find most useful.

I particularly like Selye's technical term for the total absence of stress - death.

-CM

Thanks for creating this Raven; it's looking great. Some very good comments were made here and I'd just like to add that it might be wise to define anxiety as well as stress in this discussion. The word "stress" in our current culture has become a "tag" word that almost everyone uses to promote their business. It has a broad umbrella and a different meaning for each individual. Most of whom say they have stress in their lives, however, are hesitate to use the term anxiety, as it carries a more clinical meaning.

Thanks Raven; it will be interesting to see the scientific consensus of the definitions of stress and anxiety, but more importantly for massage therapists may be how to address the public's perception of stress (which may in fact be anxiety). We should come up with a working definition in the MT industry but how does that translate to addressing the public's actual meaning?

Does "stress" not imply an environmental trigger for emotional states, whereas anxiety can exist without an environmental trigger and can be a chronic?  (whereas stress is more prone to be a temporary state)?

That is certainly a reasonable distinction to draw in terms of how people may understand the meanings of the terms in their own lives.

And yet, as you are both getting at, that may be a very different way from how scientists interpret the terms, which may also differ from how clinicians interpret the term from the clinician side, as opposed to the client/patient side.

My vertebrate endocrinology prof, John Wingfield, liked to point out how, although we tend in our thinking and conversations, to regard "stress" as a bad thing, that it is only chronic stress that harms us. Acute stress ("flight-or-flight", "flee and pee") is actually very adaptive, as it can save our lives.

Yet, although that is factually correct, we cannot assume that that is how our clients are thinking about talking about it in relation to their own lives. In addition to understanding the science behind what is going on, it is also important to be clear on what someone means when they are telling us what they experience going on in their lives.

It's not for nothing that this is called "translational" healthcare, after all. And how we do it is very important, and you both are asking the right questions to get started on that journey.

Chris Moyer wrote: "I don't think anyone should be too upset that they were previously claiming massage reduced cortisol. "

Thanks for saying that, Chris. I felt like the "confessions" were unnecessary. I wasn't taught it in school because I went to school before TRI existed! However, when I first learned about TRI I was very excited about following their studies because for (essentially) the first time we had someone bringing research into the field of massage therapy. I even met Dr. Field when she came to St. Louis and presented a talk to a conference on infant massage therapy and wrote an article for a newspaper about it.

I feel absolutely no shame about any of that at all, not one whit. It was plausible and it was the best science we had available at the time.

And, new evidence requires us to adapt our thinking. No big deal. Okay, so it's a big deal for anyone running a cortisol reduction service but for the rest of us, it means we adjust our thinking and how we communicate to clients and we just keep making them feel good as best we know how.

The shame would be if this claim *continues* to be perpetuated in light of the evidence we now have. Every massage therapist should know about this study.

I also thank Chris for pointing out that massage is a stressor. Zhenya taught us this and understanding this guides some of our decisions, in Russian massage, about what to do, when to do it, how much to do it, and when not to do anything. People do not think of massage as a stressor because it feels good, but we are interfering with the processes of the body, we are effecting a change, we are making the body adapt to external input. We are creating a stressor.

I still don't understand beans about the numbers and I'm not sure I want to. Yet. Maybe I'm a lazy bum. I want someone else who knows what they mean to just tell me, at least for now. I'm working enough at wrapping my head around the rest of this information which, since it comes in words, is much easier to understand.

p.s. Raven: I think the study that showed a rise was done in a Scandanavian country, but I found it on PubMed while looking for something else.

Regarding whether those giving the massage were trained - most were. It appears that in some cases it is not known. In the one study using untrained people, I'm assuming it's the parents of the children with RA based on that I know TRI has used parents giving massage to their children in at least one other study.

As for the timing of when cortisol was measured, I don't know enough about it to know what would be considered appropriate timing.

"reported results not duplicated in another retrieved article" I'm not sure I understand what that means. Does that mean if TRI did a study and got certain results and I did a similar study and got similar results, that my study would not be included? Or does it mean that if I did an evaluation of, say, three studies, including one done at TRI, and the one done at TRI was included in another article, that only one of them would be used because it's a duplicate i.e. you do not include the same study twice?

What I wondered about was since this is determined by speed of ANS response (in a way), then is the factor of time and measurement not something to be considered. 

Yes, this is very important. The experimental design needs to take this into consideration when determining the measurements, or else, the measurements will be inconsistent, and the researchers will be comparing apples to oranges.

You may have heard of the concept of "half-life" in connection with radiation. "Half-life" is the length of time it takes half of the amount of material present to decay--in the case of radioactivity, to turn into a different (hopefully, harmless) element. In the case of biochemical and pharmacological compounds, the component chemicals re-form and rearrange into different chemical compounds.

This next part is a little bit counterintuitive: what "half of the amount of the material present" means is going to change each time we measure it.

Let's say we have 16 units of a material that has a half-life of one day. After one day has passed, half of the amount of material present is gone, so we have 8 units left.

Then, the amount of material present is 8 units, and half of the amount of material present is 4 units, so after the second day has passed, 4 units of the 8 are gone, and we have 4 units left.

So then the amount of material present is 4 units, and half of the amount of material present is 2 units, so after the third day has passed, 2 units of the 4 are gone, and we have 2 units left.

In the same way, after the fourth day has passed, only 1 unit of the original 16 is left, and then we start getting 1/2 a unit, 1/4 a unit, 1/8/ a unit, and so on.

I remember in my vertebrate endocrinology class, my prof mentioned that the half-life for blood cortisol is rather short--like an hour or so--so they would have had to measure it very soon after the intervention, or else it would drop off too much too fast to get an accurate reading. I tried Googling to get the exact number, but no success with that. But you get the idea.

Only one of the study used blood analysis - the rest used urine or saliva.

I would have thought that urine testing would require more "time" to have an effect on what it is that we're measuring.

Yes, it takes longer to get into the urine or the saliva than into the blood, so along with the half-life, which body material it's in will have an effect on when to measure it.

Back when I was working on the bear vaginal cytology data to determine their reproductive cycle, I didn't have much to do with the endocrinology side of the study (reproductive hormones, not cortisol). However, blood sampling would have been too invasive, and even saliva would have been hard to get. Urine's too hard to collect, since it soaks right into the ground, so poop was the vehicle of choice for measuring circulating reproductive hormone levels in bears. Like I said, though, I didn't participate in the poop collection or hormone measurement.

Also, there is also the question of instrument validation.  Is one method better than another?

I don't know the answer to that, but I do know that within one study, it is important to pick one way to validate it and then stick to it. I suspect, but don't know, that to compare the studies, they had to validate in similar ways; otherwise, the data would just be too different to compare.

I realize that this is nit-picking on the individual studies themselves, but since this is a case of research on research, the top level research is heavily dependent on the the quality of the studies that are included in it.

Exactly right.

There very rarely seems to be any reference to whether or not recipients have received massage previous to the start of types of studies.  Surely this would have a major impact?  Even if you had a recipient who had been getting regular massage for a period of time, then had stopped, it would be a totally different experience for them over someone who was experiencing it for the first time as part of the study - and that different experience could throw a spanner in the works.  There is also the fact that effects may be larger for those who receive massage regularly.  It could be that adequate sample numbers might take care of that, but at the same time, I can't help but think that if 25% of the sample were receiving massage for the first time, it would have different findings from 75% receiving massage for the first time in the trial.

Interesting question. I don't know the answer to that. I have seen studies where they instructed the clients not to get massage outside of the study for the duration of that study, but I don't think I've ever seen people ruled in or out on the basis of whether they have had massage before.

Here is a basic description of SD (standard deviation), and don't be put off by the "kiddy look" of it  - it has some handy wee references for brushing up on basic stats (I think the last time I looked at stats was probably 28 years ago!)  (and who doesn't like references to dogs?). 

Thanks--I was not familiar with this; this is actually quite good. I'll put it in the Links section for reference.

The formulas can look at bit intimidating, but knowing the relevance and "why we should know the concept of it" is the most important.

Exactly--remember Bloom's learning taxonomy, as modified and applied in the InterJurisdictional Entry-to-Practice Competency Profile for Massage Therapists? I think that knowing the relevance and why we should know the concept of it meets their stated goals of reaching

• Cognitive Level 2: Comprehend & apply information
• Cognitive Level 3: Analyze and interpret information

in research literacy.

They don't require Cognitive Level 4: Synthesize (create new) information, which is where I would think using the formulas to generate new data would come in.

comments can get lost when you hit the Save button.

I didn't follow my own advice to save a copy of my work, and as a result, I'm starting over again from zero on the data sharing question.

At POEM, we value your comments and questions, and don't want the same thing to happen to you. Make sure you have a copy of your work saved in case something like what happened to my comment happens to yours.

Back to data sharing, now.

How can a well run peer review be done without providing that?

Typically, peer review operates at a higher level of analysis than poring over the data with a fine-toothed comb. If the study design is appropriate for addressing the research questions, and they've chosen the correct statistical tests for the situation, and there aren't any other glaring issues, then the peer reviewer tends to spot-check calculations, but doesn't sit down and examine every single point of raw data.

This is one reason why trust is so important in science--when someone like Werner Bezwoda [1] or Hwang Woo-Suk [2] falsifies data, and it gets past peer review, it ripples through a huge audience that trusts the peer reviewers and the process to get it right. It harms individual patients, that is true, but additionally, it harms the entire web of science. That's why it's dealt with so harshly.

Kirby Lee and Lisa Bero have advocated data sharing/data audit to cut down on cases like the above [3]. You've independently hit on the same point they are making: the contribution of data sharing to improving peer review, and through that, to improving published science.

Here's a question - am I wasting my time even trying to get people to comment on this (besides Raven?).

Nobody wants to waste their time, and I don't blame you for asking for feedback. But I would not assume that people are not reading and learning, just because they are not commenting--Adrienne has already mentioned that she is, and I have no doubt that others are as well. It is very common that sites have many people who read silently for every commenter--I've seen figures like 10 readers for each commenter.

People read and learn at their own rates, and the degree to which they wish to chime in will also vary. They're also going to want to get a sense of what the site is, and it's early days yet, so I expect that people will chime in more as they get a better feel for it.

So it's absolutely reasonable to wonder, just as a reality check, if you're wasting your time, but I would say both that

• you are definitely not wasting your time, because people are watching and learning at their own rate, and
• you are doing a really good job of touching on several very important issues that are quite relevant to this research--in the study, such as the timing of the assessments, and the larger context, such as data sharing and its effects on the quality of research. 

I am very pleased with how you are doing with facilitating this month's Journal Club, and I would say that you are providing food for thought that others will find useful to consider.

Your process is a good one, and I trust that others will join the discussion at the point at which they feel safe and comfortable doing so. And if that joining means reading and absorbing what other commenters say, that counts for learning as well.

- During the peer review process did any of the reviewers question the viability of having the Ditzen study included?

No, not that I recall.

- Also, did any of them question the inclusion of studies that were dissertation data and not studies that are published in peer reviewed journals?

No. 

- Also, did any of them question the inclusion of studies that involved reflexology?

Again, no.  The inclusion/exclusion criteria were not among the reviewers' concerns, if I recall correctly.

- Also, did any of them question that the methods of the studies varied significantly.  Do you happen to know if all of the reviewers had access to those dissertations in order to see the original study?  That one is probably difficult to answer since it is usually unknown - so maybe I'll ask a different one.  Why didn't you specify that you were just going to include data from articles that have been published in peer reviewed journals?

It would be pretty unusual for the reviewers to go to the time and trouble of looking up the original data in a case such as this.  To the uninitiated, this may sound careless on the part of the reviewers, but it really isn't.  The reason it isn't is because our study provides sufficient detail for *any* reader to check the data herself.  For example, are you wondering if the data from dissertations and theses differ from published studies?  Well, we're giving you the data, in number and in figures and in references, so you can check it yourself if you like.  And if someone does that and finds that my colleagues and I blew it, then they can write their own paper that points this out.  Our methods are totally transparent.

The reviewing history of this paper is interesting in other ways, though.  I am definitely happy that it was eventually accepted for publication in JBMT, but that is not the first journal it was submitted to.  I have wanted to get papers accepted in a wide range of journals, and this paper was submitted to Psychosomatic Medicine and to Psychophysiology before it was submitted to JBMT.  I forget which one of thsoe was first, but I'm pretty sure it was PM (I could look it up if you care).  We got a bogus review from PM, in that one of the reviewers criticized the paper but without giving substantive criticisisms.  The review was so unprofessional that I ended up drafting a letter to the editor of that journal.  Since then, I have learned that I must instruct journal editors that certain potential reviewers cannot be relied upon to have scientific integrity.  Sadly, I have other anecdotes of this type.

Psychophysiology sent the paper back without a review, as they concluded the paper was not a good fit for their journal.  At least they didn't waste our time.

-CM

That review was so unscientific and unprofessional that I felt compelled to let the journal editor know about it.  Here is what I wrote to him.  (Note that the sections which refer to effect sizes without variability data, and studies without control groups, refer to a dataset that we later removed from the final version of the paper - though not in response to this review, but for other reasons.)

December 7th, 2009

David S. Sheps, MD, MSPH
Editor-in-Chief

Psychosomatic Medicine: Journal of Biobehavioral Medicine

Dear Dr. Sheps:

Thank you for your journal's timely review of our manuscript, #PSY09-424, entitled "Cortisol Reductions in Response to Massage Therapy: A Comprehensive Quantitative Review". Logically, we are disappointed that the manuscript was not accepted nor that a revision was requested, but we appreciate that the journal stood by its commitment to review the article in eight weeks, as promised. We are also appreciative of the time and effort that two of the reviewers put in to this review, because we will be able to use their criticisms despite the fact that our study will not be appearing in Psychosomatic Medicine: Journal of Biobehavioral Medicine.

We are writing, however, to express our dissatisfaction with the third of the three reviews. Though it is now water under the bridge, it is unfortunate to consider that the third review may have been the one that tipped the balance from "revise and resubmit" to "rejected." This is because the third reviewer did not actually make substantive criticisms of the study. Though it is now merely an academic exercise to do so, I feel compelled to illustrate this, and even hope you might forward this letter to the third reviewer.

In what follows, the third review appears in its entirety, interspersed with my remarks.

The mission of this paper is important. However, it has many fatal flaws as follows:

1) The logic is flawed if the authors expect as they suggest that anxiety and depression would be reduced by massage therapy but not their biochemical correlate, cortisol. It is not clear what they hypothesize should happen to an underlying biochemistry when mood states like depression and anxiety shift?;

Just because the study does not achieve a result consistent with this reviewer's concepts of anxiety, depression, and the biochemical correlates, it does not follow that the logic is flawed. Anxiety and depression are not perfectly correlated with cortisol or any other biochemical parameter, which is beside the fact that the study makes no attempt to explain precisely how massage reduces anxiety and depression. The goal of the study, as plainly stated, is to comprehensively quantify the relationship between massage therapy and subsequent changes in cortisol levels, which it does.


2) Typically meta-analyses are not possible without variance figures.

We are well aware that meta-analytic results cannot be generated without variability data, and we plainly state that we perform the multiple analyses we have performed are due, in part, to the unavailability of some data.

Most of the studies given in the table do not have S.D.s and the authors claim that they could not get these from the authors of the original papers. That may be that many authors did not have them because of inadequate analyses or did not want to release them to the authors of this meta-analysis who had previously published flawed metaanalyses (one meta-analysis paper by the senior author that was based on 2 studies and another based only on 7 studies, both papers also negating the effects of massage therapy);

The reviewer accuses the first author's other studies of being flawed, but makes to attempt to explain how they are flawed. What type of criticism is that?

The issue of data being unavailable, even when directly requested, has a history in massage therapy research. Often, as the reviewer notes, the data are not available because many of the persons who performed individual studies were not sophisticated researchers. However, it is also true that the laboratory that has done more massage therapy research than any other has claimed, in direct correspondence with me, that much of their data was missing (including some that was not very old) following a relocation of their offices. This was after telling me that they would look for it, and then waiting months to tell me that they never had it. When our meta-analysis was accepted by Psychological Bulletin, the editor of that journal indicated that the set of studies for which variability data was inexplicably missing needed to be mentioned in the text of the article itself. This was done, probably at some embarrassment to that laboratory.

In addition, the reviewer seems to be implying that meta-analytic results based on a small number of studies are somehow completely invalid. This isn't the case, of course. They simply are what they are, a quantitative summary of the best available evidence. If the result is based on a small number of studies and/or has a wide confidence interval, that is a finding in and of itself. In fact, many methodologists have pointed out that, no matter what the number of studies, a meta-analytic treatment is always best.

3) The methodology is fatally flawed, as minimal criteria were used for the meta-analysis in this paper.

This is the old "apples and oranges" criticism of meta-analysis. It's not valid when the category to which we wish to generalize is not apples, nor oranges, but "fruit." We wish to see if massage therapy reduces cortisol. Not only does it appear that it does not, but there is no evidence of variability among the effects (we plainly state the results of having tested for this). If there were differences among different forms of massage therapy, then why is there no variability?

With the criteria being so relaxed, many good studies that were randomized and featured control groups were mixed with many bad studies that did not have those minimal features. 20 of the 48 studies were not randomized studies. 16 of the 48 studies had no control group.

We clearly delineate studies according to those features, which makes this criticism completely irrelevant.

Several studies had as few as 8 or 9 subjects and 2 studies had 1 or 2 subjects.

This criticism is also completely irrelevant, as all the studies are weighted according to their size (either sample size or inverse variance, depending on the analysis).

Different types of massage therapy were used in the different studies.

This is true, as noted above. There are probably no two studies in the massage therapy research literature that have used the same protocol, or even if there are, they would be the exception. This same issue arises in all clinical research.

Values taken from different types of samples were averaged (e.g. blood and salivas) which is not legitimate.

This is certainly not a fatal flaw, and reviewer one even goes so far as to point out that the different methods of cortisol assessment do not differ in their findings. Reviewer one supports that assertion with a citation.

In a word, mixing quality with non-quality studies would be expected to yield a result that is at odds with much of the literature. As stat teachers have been heard to say "garbage in-garbage out."

Our study, more than any study to date, plainly presents the results of massage therapy on cortisol. It does this transparently and quantitatively. The fact that the results contradict some of the literature is the point of the study, not a weakness. Finally, there is no pattern to indicate that our finding is the result of a GIGO situation.

I wish to mention that, prior to submitting this study to your journal, I thought about requesting that it not be reviewed by persons associated with a particular laboratory, as their interactions with me have consistently been unprofessional. I wanted to avoid making such a request, however, in part because I knew that we were able to respond to any reasonable and substantive criticisms of our work, and I was confident that any careless review of our work would be disregarded. I suspect now that I made a mistake in not having made such a request. I suppose I can thank you and your journal for likely having helped me learn a disappointing but important lesson about the subfield of massage therapy research.

In sum, we feel it is very unfortunate that this review was given any weight at all, as it makes no substantive criticisms and even plainly demonstrates the reviewer's poor familiarity with the methods of quantitative review. The third review contrasts starkly with those of the first two reviewers, who plainly took the task seriously and completed it conscientiously.

Sincerely, and on behalf of my coauthors,

Christopher A. Moyer, Ph.D.
Assistant Professor of Psychology
University of Wisconsin-Stout

is that sometimes scientific consensus changes.

Sometimes it may look arbitrary--eggs are bad for you; no, they're good for you; no, they're bad for you again--but here, we're getting a glimpse behind the curtain into the process.

Well, we're giving you the data, in number and in figures and in references, so you can check it yourself if you like.  And if someone does that and finds that my colleagues and I blew it, then they can write their own paper that points this out.  Our methods are totally transparent.

There's no magic and no overriding authority in research. There is only a lot of people's hard work.

And, unfortunately, sometimes people make errors, and then other people come along behind and find those errors.

Chris is saying here that his team's work is subject to exactly the same kind of scrutiny that his team performed on Field's team's work.

If their work is solid, and no one finds methodological flaws in it, eventually, it will become the consensus. If flaws are found, it will be critiqued and redone.

That's one of the strengths of research, but the lack of certainty can be frustrating when it comes time to turn this basic science information into clinical decision-making.

Say there was a study and as part of the methods a family member is trained for about 20 minutes on sitting in a chair, asking a few questions on how someone feels, their thoughts and behavior and taking some notes and this is put forth as "The effects of Psychotherapy on......".

Well, you can imagine how that would go down.  If you replace that with any sort of therapy e.g. physiotherapy or occupational therapy, then you can imagine how people in those professions would react.  It seems to me that a quite a number of researchers regard massage therapy as nothing more than a bit of rubbing that can be taught to someone in a few minutes.  I could take that as an insult, but I won't, because I'm a massage therapist and I'm pretty laid back . 

Yes, that's quite true. On the other hand, we deal in touch, and touch is something that goes way further back than just human history. Many different kinds of animals use touch in ways that aren't simple and obvious, that we have to explore through neurology, endocrinology, and other disciplines in order to understand.

So just touching and rubbing can have beneficial effects. And we use touching and rubbing for just those reasons. And yet, as you point out, what we do is so much more than just that. Which brings up several issues that affect how we apply this research, as well as how we communicate with clients and other healthcare practitioners.

We have not yet talked about the several different types of validity involved in studies, but one of the issues that validity has to do with is "how much can the effects of this study be expected to apply in different populations and treatment situations?".  So does having a family member perform a rigid protocol for 20 minutes every day really tell us anything useful about what we do in the clinic?

In addition to questions about validity, it also brings up questions about how we communicate to other stakeholders about what it is that we really do. Do we have enough consensus on what that is that we can do so reliably, or is that something that we need to agree on in-house first?

There was another post on POEM regarding the "curation of massage therapy".  I think it's time that there was some curation of the use of the term, otherwise we're nothing more than people that give others a wee shoulder rub (I'm also aware that some therapists have t-shrits with "I'm a rubber" on it - yep.  not me though).  I'm not sure about anyone else, but when someone approaches me outside of the work environment and said "can you just give my neck a rub, since you do massage" it sort of irks me.  I politely give them my card and ask them to make an appointment.  There's a reason for that.

It sounds like you are clear on your professional boundaries. I think being clear on that is a good thing--not that everyone draws them in the same places as everyone else, but knowing what's right for you and sticking to it is what I think is good.

The question we're not sure about is exactly where

I don't know about anyone else, but not even having the correct spelling for Therapy, never mind "stress" doesn't scream "high quality" to me.

Yes, that's embarrassing. But I have no doubt that they all know how to spell those words; this is not an issue of their not knowing.

It may have been that they were in such a rush for whatever reason that their process totally broke down. Having turned in both papers and grants on tight deadlines, I can envision such as scenario, although those errors are both so elementary and so flagrant that I have a hard time seeing how they got past even the most frantic deadline rush.

I have a different idea of what may have happened. Remember, this is a 1996 paper. Not everything was so dependably online 15 years ago as it now is.

At that time, a lot of things existed in print only.

I can imagine a scenario where this is a print article that the publisher later decided to scan in, and that the publisher end was sloppy about looking for errors introduced into the text by optical character recognition (OCR) programs.

I will be over at the UW on Friday, and I will see if the article is available there (for me at home, it is behind a paywall). If it is there, I will see whether the article really contains those errors, or if it's only the record pointing to the article that does.

Not that I disagree with your general point that if authors are sloppy about writing, editing, and proofing, that may be a symptom of sloppy labwork as well. But this is so embarrassingly bad, that I actually find it hard to believe it made it past even the most harried team on deadline.

And if you look at it a bit more, you'll see that it's not exactly telling us very much, it's badly formatted and oh, look at the bottom of that to see where it got it's funds. 

It's true that when you get corporate money, you have to be scrupulously careful to avoid even the perception of bias, and that corporations often don't have the institutional safeguards against bias that institutions like the National Institutes of Health (NIH) and the National Science Foundation (NSF) have.

But when only 5% or 10% or 20% of NIH and NSF grant applications receive funding, it's also clear that corporate funding of research is not going to go away any time soon, either.

It's a fact of research life, and it's something we need to be aware of when evaluating research. It's one of the reasons why researchers disclose their funding sources in articles--to give the reader the opportunity to make the determination of how much weight to assign that possibility.

As an example of where I'm coming from:

Say there was a study and as part of the methods a family member is trained for about 20 minutes on sitting in a chair, asking a few questions on how someone feels, their thoughts and behavior and taking some notes and this is put forth as "The effects of Psychotherapy on......".

This is an excellent point.  And yet, studies such as this are sometimes conducted.  They are sometimes known as "analog" studies, because the intervention being used is considered analogous, but not idential, to what really should be studied.

I have never done an analog study, and I try to avoid them.  But, I understand why they are conducted, and I might do one myself someday.  Essentially, they are a 'weak' test of a treatment.  If talking to someone improves their psychological condition, it does logically follow that psychotherapy - in which a highly trained person talks with someone - should work at least as well as having an untrained person.  But if it doesn't work, it is still entirely possible that psychotherapy works, because the specialized aspects of psychotherapy haven't been tested in an analog study.  The same logic applies to massage research.  If *any* touch intended to be therapeutic is helpful, it is reasonably to conclude that true massage therapy will be helpful, but the reverse - if untrained touch is ineffective, legitimate massage therapy training will also not be effective - is not necessarily true.

True clinical research can be very difficult to conduct.  It requires money, resources, and the availability of experts, and that's just for starters.  When any one of those things aren't available, the next best thing might be to do an analog study.  As I stated previously, I have avoided doing analog studies myself, at least up until the present date, because they have important limitations.  However, I do understand why researchers sometimes conduct them, and I wouldn't entirely rule out, ahead of time, the possibility of ever conducting one.

-CM

Important note

UPDATE, 15 September 2011: I'm not one for deleting posts or comments without very good reason, because I think accountability demands you stand behind what you say, rather than just changing the record. However, after I had posted this comment, Chris pointed out a couple of ways in which it was in error.

So don't follow the advice in this comment; from the feedback, I learned that I was thinking about it wrong, by casting it in terms of percentages.

Here's a better explanation that you should use instead of this one.

data can never be 100% accurate.  If you ask people to indicate their own gender, the reliability of that data has been observed to be "only"about r = .96.  In other words, even if the thing you are asking about is something people should know about themselves with practically no ambiguity, there will still be keystroke errors, misunderstandings, and the like.  No data, not even the most basic, can be gathered with perfect reliability.

That makes sense. Information in the real world always contains a certain amount of noise.

Understanding statistics is not a requirement for participating in Journal Club, so I'm not assuming that everyone here is familiar with the statistic r (Pearson's r).

r is a measure of how reliably two things correlate with each other--in this case, the person's gender in real life, and the gender that they indicate in their answer on a questionnaire.

You would expect a perfect correlation--100% reliability--but, as Chris points out, it's been observed that

r = .96

There are several statistical measures that can range in value from 0 to 1*, and which can easily be converted to percentages. We'll talk about those statistics as we encounter them.

r is one of those statistics, so you can multiply it by 100% to find out what that means in percentage terms.

.96 x 100% = (.96 x 100)% = 96% 

So r = .96 means a reliability of 96%, and an error rate of 4%.

If something that relatively straightforward has an error rate of 4%, it would be reasonable to expect the error rate to go even higher when the questions get more complicated, such as medical history.

* r actually ranges from -1 through 0 to 1; we will talk about what the negative values mean later, and just treat the 0 to 1 part of r's range as 0% to 100%.

But even if it is typos and that was corrected - would you regard it as a well written abstract?

With the caveat that I have 15 years of hindsight to draw upon, so it's a little unfair to judge a 1995 article by 2011 standards, I would still have to say I don't think it's especially well-written. There's just not enough information about methods there. It's very general and high-level:

Forty full-term 1- to 3-month-old infants born to depressed adolescent mothers who were low socioeconomic status (SES) and single parents were givn 15 min of either massage (n = 20) or rocking for 2 days per week for a 6-week period. The infants who experienced massage therapy compared to infants in the rocking control group spent more time in active alert and active awake states, cried less, and had lower salivary cotisol levels, suggesting lower stree. After the massage versus the rocking sessions, the infants spent less time in an active awake state, suggesting that massage may be more effective than rocking for inducing sleep. Over the 6-week period, the massage-therapy infants gained more weight, showed greater improvement on emotionality, sociability, amd soothability temperament dimensions and had greater decreases in urinary stress catecholamines/hormones (norepinephrine, epinephrine, cortisol).

One thing I prefer is a structured abstract. Here's an example of a structured abstract for a study that included massage:

Judson PL, Dickson EL, Argenta PA, Xiong Y, Geller MA, Carson LF, Ghebre R, Jonson AL, Downs LS Jr. A prospective, randomized trial of integrative medicine for women with ovarian cancer. Gynecologic Oncology. 2011 Aug 22.

 

Abstract

OBJECTIVES:

Despite increased use of integrative medicine in cancer therapy, little data exist on its efficacy. This prospective, randomized, pilot trial sought to evaluate the feasibility of combined modality integrative medicine (CM-IM) in women with ovarian cancer (OvCA) and evaluate its effects on quality of life (QoL), chemotherapy toxicity and immunologic profiles.

 

METHODS:

Women with newly diagnosed OvCA requiring chemotherapy were offered enrollment. Those randomized to the experimental arm received hypnosis, therapeutic massage and healing touch with each cycle of chemotherapy. The control arm received chemotherapy without CM-IM. All patients completed QoL questionnaires prior to cycles 1, 3 and 6, and 6-months after chemotherapy. Immunologic profiles were measured. Statistical analysis was based on intent-to-treat. Student's t-test and Fischer's exact-test were used to determine differences.

 

RESULTS:

Forty-three women enrolled. All women randomized to CM-IM were successfully treated. There were no statistical differences between the groups in age, stage, grade, histologic cell type, CA125 levels, or surgical cytoreductive status. There was no difference in overall QoL measurements. Re-hospitalization rates, treatment delays, anti-emetic use, and infection rates were similar. Immunologic profiles revealed no difference between arms for WBC or salivary IgA levels. Women receiving CM-IM had consistently higher levels of CD4, CD8 and NK cells, although this did not reach statistical significance.

 

CONCLUSIONS:

Prospective clinical evaluation of integrative medicine for women with gynecologic malignancy is feasible. This first, pilot study of CM-IM in gynecologic oncology demonstrated no improvement in QoL or chemotherapy toxicity. Integrative medicine-associated improvements in immunologic profiles warrant further investigation.

I had never heard the term "analog studies". 

Which, when you think about it, is really odd, since so many of the protocols in MT research would fall under that term.

I learned the term as part of psychotherapy research.  Its usage might be limited to that arena, but I agree that it could be extended to other research areas such as massage.

-CM

So far your presentation of statistical information has been terrific, but with Pearson's r you don't have things quite right.  You suggest that the statistic is called r because that stands for reliability, but that's not the case, and Pearson's r can be used in many ways other than measuing reliability - it's simply a standardized way of quantifying the correlation between any two variables.  (There is a reason it's called r and I used to know it, but I've since forgotten.)

Also, you cannot treat r as a percentage, as you suggest.  In some cases it is appropriate to square an r value to get a new statistic that is the percentage of variance accounted for by a statistical model, but r itself is not a percentage and cannot be interpreted that way.

Thanks for the correction; I've apparently been using r in ways that I shouldn't. I appreciate your pointing out my error, so that I can change that.

I'm not one for deleting posts or comments without very good reason, because I think accountability demands you stand behind what you say, rather than just changing the record. I will, however, put a prominent note at the top of the comment.

Cheers, Chris!

The updated info on correlation is better, but there are still some things I must take issue with.  Your wording of "direct" and "indirect" is imprecise and misleading; what you mean to say in those cases are "positive" and "negative."  Positive correlations are when two variables are moving in the same direction; as one increases, so does the other.  Negative correlations are when two variables move in opposite directions; as one increases, the other decreases.  There is nothing about a negative correlation that is any more or less direct than a positive correlation.  Further, if the scale we are using is arbitrary, we can always reverse one of the scales which will have the effect of changing the sign of a corresponding correlation coefficient.

As noted before, correlations are bounded between -1 (a perfect negative correlation) and 1 (a perfect positive correlation).  A value of zero indicates no correlation at all.

Your example of height and weight in developing children is an excellent example.  As one increase, so does the other tend to increase, but as we all know those two variables are not perfectly correlated.  Sometimes people get taller without gaining weight, and sometimes they gain weight without getting taller.

Your next example, in my opinion, is not a good example, even if we replace the word "indirect" with "negative."

A negative correlation is a situation where two things vary in connection with each other, but in opposite ways--as one variable increases, the other decreases. For example, the faster (more speed) you drive to your destination, the less time it takes you to get there. So speed and the time the drive takes are indirectly correlated.

The reason I do not like this example is because the two variables - rate, and time - are perfectly correlated; if we know the rate at which something is travelling across a given distance, we can perfectly predict travel time.  And when two variables are perfectly correlated, they are not really two variables at all, but are only a single concept. 

To give another example of this, degrees F and degrees C are perfectly positively correlated, because they are actually the same thing (temperature) just on a different scale.  For a negative correlation example, if we know how much individuals weigh in pounds, this will be perfectly and negatively correlated with how much less they weigh, in grams, than the Empire State Building.  Both scales are the exact same thing, conceptually (weight).

A better example of a negative correlation would be a person's current age, and their remaining years of life.  The older we get, the fewer years we have left to live, but we cannot perfectly predict one from the other.

-CM

Also, I sometimes wonder if the fact that we're not taught research literacy skills has lead to poor quality studies.  Take this abstract, for example (one of the studies used).  I don't know about anyone else, but not even having the correct spelling for Therapy, never mind "stress" doesn't scream "high quality" to me.

It's quite reasonable and a very good guiding principle to ask "if they can't get the little things right, how can we trust them to get the big things right?".

If you see elementary errors being made, then you have to wonder what's going on with the team and this research that led to those errors making it all the way through the process into print.

However, these misspellings were so bad that they were over the top. I find it hard to believe that--even if the team were so harried and rushed by deadlines that this slipped through their process, that reviewers/editors at the journal would let something *that* egregious into print.

I count:

• theraphy for therapy twice
• givn for given
• cotisol for cortisol
• stree for stress
• earlt intervention for early intervention
• senssory stimulation for sensory stimulation
• P.O. Bos for P.O. Box

just in the abstract.

That's a level of carelessness I find pretty unbelievable, and so I suspect the explanation lies elsewhere than the authors of the paper.

I got the paper from the library this morning, and here's a screenshot of the abstract:

The mistakes aren't in the paper; the publisher introduced them into the record pointing to the paper. I suspect it happened when someone used a scanner and an OCR program to make their record, but did not bother to proof it after the OCR program had run.

Hi, all--

Just wanted to check in with you about what you'd like to see here.

rchunco has expressed her concerns that people are not interested or don't care, because there hasn't been much discussion lately.

I don't think that's the case at all.

But I think it may be the case that the first time we try something, we find some things that need to be tweaked and adjusted a bit. If there are, then we'll make those changes--there is nothing wrong with making a mid-course correction as needed. If you've seen how sailboats move, then you'll know what I mean by tacking in one direction or another, until we find the right course.

I notice that there was more discussion going on a little while ago than there is now. Is one article per month perhaps too slow a pace? Would two weeks per article be a better speed, and would people have more to say in that case?

Is there anything else we should be taking into consideration?

Journal Club is for you, so if you have any ideas how it can meet your information needs, I hope you feel free to speak up and give your feedback.

I think this has been fine, though I realize I'm not the journal club's intended audience.  And as was noted a while back, I'm sure there are people reading who are not commenting but who are still taking something away from this.

I agree with the idea that the next selection for your journal club should be something very straightforward in terms of research design.  That will give folks who are just beginning to learn about research more opportunity to jump in, maybe.

-CM

Thank you, Sue. You've summed up a number of important points very well.

Rose, your comments have been extremely helpful, well thought out, and show that you have spent a LOT of time on this. A huge and hearty thank you for taking the time to dig in and review this study in such detail. Please do not lose your motivation when people do not respond or comment with as much enthusiasm or intensity as you show when you comment. Your work is appreciated and, in my view, much needed.

Seconded, absolutely. I am very pleased with the work Rose has put into this, including as webmistress as well as in the discussion, and I am blown away by what the other commenters have brought in as well.

It's early days, and we're building something that will last for decades, so I'm pleased with the participation to date. It will certainly add up, as well as picking up over time, as we get familiar with doing this. Remember that we're doing something new here, and we all have to figure it out.

In addition to the intellectual exercise going on, I'm also pleased at how the community is taking shape. It's very kind of you to reach out to Rose like that, and I think the supportive community aspect of POEM is a very important part of the site as well.

That all said, it does take time to read and comment and digest all this so comments will likely trickle in over time. (I just finished reading through this thread today and it's nearly the end of October.) I hope more of us will contribute to POEM, both financially and through commenting and eventually conducting research ourselves.

I won't shut down threads just because "time's up". Think of it more like there's a "featured" article or set of articles each month, but discussion can occur on any article at any time. Comments are welcome even years later--remember, some people haven't even found POEM yet, so they haven't had a chance to comment.

What I will do is mine the comments for content for the e-Books, so I'll have markers that say things like "all content before this point has been adapted for the human systems e-Book", or something like that. But that's not the end of the discussion by any means; it just means that if more discussion happens later, I'll know where to pick up working with the content, rather than redoing what was already done.

I think Journal Club is such a great way to teach MTs research because we are starting with picking apart existing studies so that any research we conduct or review in the future will more likely avoid some of the errors made by those that went before us. Science and research is always evolving, never completely "done."

A very important point, which you've made very well. This exercise is not just about nitpicking, it's to help make future research efforts even better.

I think that that point might not be widely recognized, so it might look more negative than it is. But it's important to remember why we do it, and you've stated it concisely and clearly. 

As to the article under review in Sept Journal Club, I want to point out that it has not "proven" that massage does not reduce cortisol. What it does say is that "The claim that massage reduces cortisol is brought into question. " In just a cursory review of some of the articles published regarding the effect of massage therapy on cortisol, my reaction is, we need to conduct more research on this claim before we can definitively say that massage therapy does not reduce cortisol.

It's an interesting situation.

We've started discussing, but have not yet fully explored, the concept of "burden of proof". Part 1 of the discussion is here; there will be more later.

The gist of it is that positive claims ("Treatment X has Y effect") and negative claims ("Treatment X has no effect") are not treated exactly the same way. The first statement has the risk of false positives, when we mistakenly think that something has an effect that it does not have.

The second statement has the risk of false negatives, where we mistakenly miss a real effect that something does have.

Since there's no way we can always get it exactly right, and we have to choose whether we'd rather live with false positives or false negatives, textbook science--the kind that students study, as opposed to frontier science, where scientists are still trying to figure out what is actually happening--tends to favor false negatives over false positives.

It can be frustrating in its conservatism, but there are advantages to favoring false negatives over false positives, and the idea is that if something really does have an effect that we've missed, the truth will eventually come out. If we favor false positives, on the other hand, we soon get lost in all the possibilities, and no longer can sort out what's really going on from what's a false positive effect.

So, as a positive claim, "massage does reduce cortisol" had a higher burden of proof than the corresponding negative claim, which would be "massage does not reduce cortisol".

It looked like that result was so consistent over so many studies that the claim had met its burden of proof, so people started to cite it as though we were sure it was true.

What Moyer et alia ("and all", meaning his team) pointed out was that the Field team's calculation was done in a non-standard way that increased the risk of false positive. Since so much of the research has been done by the Field team, then so much of what was assumed to meet the claim's burden of proof is now shown to be weaker than thought.

Since the positive claim needed that evidence, because it had the higher burden of proof, then this does strengthen the negative claim, that massage does not reduce cortisol.

It doesn't mean that massage doesn't work, only that the cortisol explanation does not account for massage effects.

But it's interesting, because so much of the research has been carried out by one team, that if that team has made a mistake, the uncertainty about it affects so much. I, too, would like to see more research carried out by more teams, just to reduce this kind of risk.

Correct me if I am wrong. (I often risk being wrong--it allows me to learn!)

That's the spirit!

This is a safe place to learn, and that means taking the risk of being wrong. But while factual corrections are permitted, they will never be allowed to become personal attacks or insults, because we want to preserve this as a safe place for everyone's learning.

Now some may say, "oh my gosh, this is far too overwhelming! It's too much to think about and who on earth has time and energy and the training to conduct such research?" I know I've had this reaction at times. But then I get excited. I think, wow, what an awesome time to be a massage therapist and have the opportunity to learn about resaerch, maybe conduct some of our own and perhaps even help change the profession, increase our credibility, provide other MTs with substantative information about what we do. What an exciting time to be in the field! (I'm hoping more people have the latter reaction.)

It can be overwhelming--but remember, none us of is alone here! All of the things you said are true, and we're here for each other along the journey. So if one of us has trouble with some concept, there are others who can and will help. And then someday the learner will be the one to share with a future learner. So the initial overwhelmingness of it begins to disappear when we work together.

I hope that after reviewing some of these studies, there are those of us in this group that will be able to conduct some new studies. While we can be grateful to Tiffiany Fields and others in the early days of massage research, many of the MT studies out there will not live up to rigorous scientific scrutiny.  Let's give our fellow MTs something more substantial to chew on and use as resources to back up claims about the health benefits of MT. I hope over time that as we all grow stronger in our understanding of research methods, we can substantially contribute to the research available.

We're very happy you're here, and look forward to your participation in the discussion.