Look No Further: Worst Neuromonitoring Paper Ever

Have you ever wondered what’s the worst academic paper ever published on the subject of neuromonitoring? I ponder this question sometimes as I review the literature because I read so many awful papers. I can’t help but wonder how some of these papers ever get published, and especially in high-profile journals. The problems range from bad research design, to inappropriate statistics, to incomplete methodology, to misinterpretation of findings and outright falsification of data. No one seems to notice. It makes me wonder if I’m the only person out here reading. I know I’m not, but you get the point.

Anyway, neuromonitoring research found its new low in 2010 with the publication of a case report, which I submit is the worst academic paper ever published on the subject of neuromonitoring. At least, it’s the worst one that I’ve ever read. Details are below:

Paper Overview

Title: False negative and positive motor evoked potentials in one patient: is single motor evoked potential monitoring reliable method? A case report and literature review.

Authors: Jae-Young Hong, MD, Seung-Woo Suh, MD, PhD, Hitesh N. Modi, MS, Chang-Yong Hur, MD, PhD, Hae-Ryong Song, MD, PhD, and Jong Hoon Park, MD, PhD

Journal: Spine (Phila Pa) 35(18): E912-E916, 2010

Description: A 23-year-old neurologically-intact male with a history of spondyloepiphyseal dysplasia presented with severe kyphoscoliosis. Surgical correction and fusion were performed with transcranial MEP monitoring. During the entire procedure, the MEP did not reveal any signs of cord injury. However, lower limb paralysis and paresthesia was observed when the patient awakened. After 2 additional surgical procedures to recover the neurologic deficit, the MEP did not show any signs of cord injury but the patient’s neurologic status had recovered slightly. At postoperative day 8, the neurologic status recovered, and a third operation was performed to fix the long rods. However, there were abnormal amplitudes in both lower limbs but the patient’s neurologic status was almost normal. It is concluded that undesirable events can occur with use of MEP in scoliosis or other spinal surgery. Therefore, we warn the surgeons too heavily rely on the MEP monitoring, and propose a further prospective study as well as use of alternative method that can improve the reliability of single MEP.

Critique

I could literally spend hours writing about everything that is wrong with this paper, but we’d both get bored. The fact is, at the end of the day, the most important reason why this is the worst paper ever is that all of the conclusions are based on misinterpretation of data. So, my approach here will be to take you through the data and tell you where the authors went wrong.

So, the procedure was T3-L3 posterior spinal fusion and deformity correction with pedicle screw and rod construct. Prior to dissection, the following baseline MEP data were acquired:

A

A: Baseline MEPs recorded before dissection during the initial procedure.

So, this is the baseline study. These are the data to which we compare all future studies in order to provide an ongoing assessment of the patient’s motor function. Here’s the problem: in the image above (A), there were no monitorable MEPs from the legs at baseline. Perhaps there is a blip of a signal in the left leg, but it clearly isn’t of adequate quality for monitoring. From the right leg, the authors monitored 60-Hz noise. So, every judgement made in surgery from here-on is relative to, well, nothing. The authors didn’t see it that way, though. Rather, they saw a 58 μV signal from the left leg, and a 78 μV signal from the right leg. If the authors had any experience in neuromonitoring, then they would have worked to elicit reliable motor evoked potentials from the lower extremities, but I digress.

Following insertion of pedicle screws from T3-L3, the authors performed another test of motor function. Here are the data:

B

“Maintenance of MEPs after insertion of all screws.”

The authors claim that there are no changes in MEPs from baseline after insertion of pedicle screws. I beg to differ. First, the “blip” from the left leg (if it was even real) is now significantly delayed. On top of that, MEPs are now absent from the left arm, likely secondary to malpositioning of that arm. There’s no change in MEPs from the right leg, but they were also absent at baseline. Based on the interpretation of these data as representing no change in spinal cord motor conduction, the authors proceed with correction of the spinal deformity. Following correction, another motor test was performed:

"Maintenance of MEPs after correction of the spinal curvature."

“Maintenance of MEPs after correction of the spinal curvature.”

Again, at this point, the authors are still interpreting the data as being consistent with intact spinal cord motor conduction, despite the fact that MEPs remained completely and totally absent from bilateral lower extremities. Guess what happened next! The patient woke with flacid paralysis of the lower extremities, absence of anal tone, absence of deep tendon reflexes, and absence of distal sensations below T6. The authors conclude at this point that neuromonitoring produced a false negative (predicted intact spinal cord conduction in the face of actual paralysis).

So, the authors do a stat CT which shows abnormally placed screws. The patient is rushed back to surgery, the misplaced screws are removed, and a wake-up test reveals that the patient remains paralyzed. At this point, the authors decide to release the rods for fear that they put too much traction on the spinal cord during correction. Here are the data before removal of rods:

Maintenance of MEPs in 2nd operation, before removal of screws and rods.

“maintenance of MEP before screws and rods removal in second operation”

There may actually be something there from the legs, for once, but the authors still report no change from baseline. This is a good sign! Following removal of the screws and rods, another test of spinal cord motor function. Here are the data:

"maintenance of MEP after screws and rods removal in second operation"

“maintenance
of MEP after screws and rods removal in second operation”

Again, the authors interpret the above data as being consistent with baseline (no change). Well, that’s sort-of true since we had no monitorable data from the lower extremities at baseline (Figure A); however, there is a change from data collected before removal of rods (Figure D). Sadly, the authors are still monitoring 60-Hz noise as if it is an MEP signal.

After removal of rods and screws, the patient awakes with some improvement. Specifically, there 1-2/5 strength in the legs, and distal sensations improved from paresthesia to hypesthesia below T6. The Patient was sent to the ICU for recovery.

At postoperative day 8 the patient’s muscle strength had improved to 4/5, and additional surgery was carried out to fix the rods. The contour of the 2 rods was modified to the present curvature to prevent further neurologic injury. During this procedure, the following baseline data were acquired:

"loss of MEP on lower extremity before rod insertion in third operation"

“loss of MEP on lower extremity before rod insertion in third operation”

So, MEPs are absent from the legs at baseline during this procedure, despite the fact that the patient is relatively neuro-intact (note: this is no different from baseline in the first procedure). After insertion of the rods, a final motor study is conducted and reveals the following data:

"loss of MEP on lower extremity after rod insertion in third operation"

“loss of MEP on lower extremity after rod insertion in third operation”

So, no change in that there is still no MEPs recorded from the legs. The patient wakes from this surgery with no new neurologic deficits. Thus, the authors interpret this as a neuromonitoring false positive (predict neurologic injury when patient is neuro-intact).

Conclusions

In this case study, the baseline neuromonitoring data were incorrectly interpreted in that MEP signals were identified in the legs, when in fact they were absent. Everything went downhill from there. The patient is lucky to have regained function, but it took 3 surgeries and weeks of recovery. The neuromonitoring system used in this paper looks automated or surgeon-directed. The study highlights the critical need for appropriately-trained, board-certified professional interpreters (AuD, PhD, MD, DC).

Like I said before, I don’t really want to point out all of the little problems that surfaced in this study. It would take a lot of time. With problems this obvious, who needs to nit-pick? This paper is the embodiment of malpractice, and that’s sort-of the irony. When people make egregious mistakes, you almost expect them to try to hide it, but certainly not publish it! That’s how I know that these mistakes were secondary to unfettered ignorance. The authors just didn’t know any better. They didn’t know how to appropriately monitor neural function during scoliosis surgery, and, even if they did, the authors didn’t know how to interpret the neurophysiology data.

I think the worst part of this mess, given that the patient recovered, is that the report got published. It shows that peer-review failed, for one reason or another. Maybe the critiquing parties were also ignorant about monitoring, or maybe they were lazy and didn’t vet the data set. The consequence of the publication is that people cite this paper, when in fact the conclusions are invalid. Indeed, this paper is a wet dream for abstract-skimming neuromonitoring haters, and anyone who cites this paper to validate their own conclusions should be ashamed of themselves. People need to start reading beyond the abstract. When papers like this are used as citations, it creates an impenetrable cycle of misinformation, and the academic arena isn’t the place for this type of nonsense in the age of evidence-based medicine.

The entire paper should be retracted.

14 thoughts on “Look No Further: Worst Neuromonitoring Paper Ever

  • Reply Jeremy Bamford September 4, 2014 at 12:02

    This paper is indeed awful. But you’ll notice that two of the authors (Modi and Suh) are also authors on another paper from one year earlier where MEPs were misinterpreted during a scoli case and the patient awoke paralyzed. It appears that they plugged hands and feet backwards into their automated system and believed they still had MEPs from the foot when they actually lost them during the correction. The latency of the responses from hands and feet were reversed, indicating a reversal of the lead placement; something an experienced neurophysiologist would surely have caught and corrected. What fascinates me is that these papers were published in a spine journal. Apparently none of the editors or reviewers were knowledgeable enough to catch something that many in the neurophysiology community saw immediately. Evidence that many surgeons are not equipped to run their own monitoring.

    http://www.ncbi.nlm.nih.gov/pubmed/19910760

    • Reply Richard Vogel September 4, 2014 at 16:14

      Thanks, Jeremy! I didn’t see that other paper, but I’ll give it a read (for sport). It is disappointing that these papers get published, and then cited. I think the problem can be resolved with a combination of education and outreach. In the meantime, we need watchdogs.

  • Reply John N. Gardi September 6, 2014 at 00:34

    Rich,

    I actually think the first paper in this series from Modi was the worst paper ever published in our field. Although several of us in the field wrote a letter to the editor of Spine, they refused to retract the initial paper and then continued by further publishing the paper that you cited a year later in the same journal. See Lieberman, J.A., Berven, S., Gardi, J., Hu, S., Lyon, R., MacDonald, D.B., Schwartz, D., Sestokas, A., Yingling, C. Letter & rebuttal to Modi et al. Spine, 35(6):717-720, 2010. Also you are correct in that the reviewers (of both papers) clearly had no knowledge of neuromonitoring and frankly were probably biased against MEP use in scoliosis surgery to begin with. Finally, they were utilizing one of the Medtronic NIM spine 8-channel units without any attempt to record SSEPs or more than one motor channel from each limb.

    • Reply Richard Vogel September 6, 2014 at 07:16

      I actually didn’t know about this other paper until Jeremy Bamford told me a few days ago. Maybe I’ll review that paper next, since I have an audience. Thanks for your comment!!

  • Reply Tara Stewart September 7, 2014 at 00:48

    This paper is truly appalling. As you point out – there is no shortage of bad IOM papers. The system is built so that anyone can submit a paper and with such an open invitation there are always bound to be poor studies submitted. Ultimately it is the duty of the journal reviewers to hold up a scientific standard and not contribute to the ever increasing noise. In this case the reviewers failed miserably.

    Do we as professionals have a duty to give feedback to the Editors-In-Chief (whether in causal email or a formal comment letter) to give them an opportunity to improve their review process?

    • Reply Richard Vogel September 8, 2014 at 10:08

      Hi Tara, your comments are spot-on. I believe that we have a duty to contact the Editors in these circumstances. In fact, I know that several individuals and groups contacted Spine with formal requests for retraction, which were all denied. I can only assume that the motivations on the part of the journal editorial board were political, which is a shame because it detracts from the legitimacy of the journal (in my opinion). Also, in my opinion, when circumstances like this occur (denied requests for retraction from all-over the world), the reviewers should publicly identify themselves and justify their decisions. Thanks very much for reading/commenting.

  • Reply Bryan Wilent September 22, 2014 at 08:24

    This is painful.

    Also, is that the patient’s name (Lee Tae You) in top left off all the screenshots?

  • Reply Bill Loftus November 11, 2014 at 02:09

    Rich, in addition to reading the Modi et all paper, read the ensuing letters to Spine, including the mind boggling defense by the authors and the definitive take by schwarz et al. Actually, what is most disturbing is that the Journal, as far as I know, still refused to pull the paper. I wonder if they still solicit the services of the same reviewers of the manuscript?

    • Reply Richard Vogel November 11, 2014 at 09:37

      Hi Bill. Thanks for your comment! I did read all of those letters, but didn’t mention them here. You make a good point, though. It isn’t like this paper slipped through the cracks. It seems like the whole world recognized the errors in the paper, except the authors. My suspicion is that the reviewers and the editors made their decisions for political reasons, because they wanted to publish evidence that IONM fails to detect major spinal cord injury, regardless of whether or not the methods or results were valid. Their inability or unwillingness to provide objective criticism undermines the entire scientific process.

  • Reply Mike Williamson September 1, 2015 at 18:43

    I have a question regarding the first paper where the hands and feet were switched on the plug-ins….if the post woke up paraplegic, then you would have expected the MEP responses to have been lost from the hands…does anyone recall enough from that paper/case to know if that was noted?

    Mike

    • Reply Richard Vogel September 9, 2015 at 22:23

      My recollection is that the “hands” did disappear, but the authors didn’t think anything of it. They were only concerned with the “feet”.

  • Reply Luigi Chiacchiari June 17, 2016 at 17:12

    Is there someone that can provide me those papers?
    Thank you!

    • Reply Richard Vogel June 19, 2016 at 22:22

      I can’t distribute the paper because it’s copyright protected.

  • Reply The Dark Side of Neuromonitoring - Neurologic Labs March 19, 2017 at 19:32

    […] research, and I also review papers for several journals. I’ve written previously about the worst IONM paper ever, but bad research is everywhere. Unfortunately, it can sometimes mask as decent research, even […]

Leave a Reply

%d bloggers like this: