Sometimes, Peer-Review Works

While it can be prestigious (and, is required for the CV for younger colleagues wanting to make any sort of academic career), publishing in journals tends to be one-way communication, and not really to the intellectual benefit of the authors, who “report on their work and their findings”.

  • During the peer-review process of a journal submission, often the feedback is only editorial, very rarely is it substantial to the actual science presented – and, by the very nature of “double-blind reviews” where neither the reviewer nor the authors know who each other is, any dialectic process and discussion is impossible.
  • Once published, the totality of “feedback” to a journal publication tends to be possible citations, impacting the authors bibliometrics (citation counts, h-indexes, RG-scores, etc.).
On the importance of coffee
On the importance of coffee

Contrasting with conferences where you go present your work after the review process and (if doing a good job at the presentation) might get questions during/after the presentation – as well as an opportunity to spend time after your presentation with interested colleagues and peers, exchanging ideas and inspiration…hallway-discussions, coffee-breaks, and dinners, are generally where science (and, engineering) gets done. Actually, on the walls in the building at Ecole Polytechnique (the “Pôle Entrepreneuriat et Innovation”) where I have my office, we have a sign hanging on the wall, reminding us that coffee breaks are an important (and, productive) part of our academic lives.

So, generally, I have not been a great fan of publishing in journals. But I may have to change my mind based on recent experiences …

Cisco LoRA Industrial Router
Cisco LoRA Industrial Router

I, a student of mine, and some colleagues, submitted a paper to a journal some time back. The paper contains experimental and analytical studies of an IoT communications technology called LoRa. When we started that work, of course, we did an exhaustive bibliographic search to see what others had done on this topic. Turned out that the answer at the time was “absolutely nothing”. So we dug in, did the work over about a year or so, wrote up a paper, and submitted it to what seemed to be a relevant journal (and, considering the importance of bibliometrics, one with a also a reasonable impact factor >2).

After a handful of months (as is common), we got a reply back from the journal, which essentially went “Revised version needed”. That’s always a bummer, but looking at the reviews quickly turned our frowns upside down.

Sure, there were the usual reviewers making comments such as “you’ve missed a comma here” and “this figure should be bigger/smaller” (which really is the journal editors job to address once the peer reviews are done and dealt with, and not really a task for the scientific peer reviewers).

But, there were also very very  helpful review comments which went “actually, if I were to reproduce this experiment that you did, then I am missing that bit of information” (easy to address, but incredibly important for the repeatability of experimental science), and even pointing out that in the time since we started our study of LoRa, a couple of publications had appeared – even ones providing complementary results to ours, which we could use to put our work and results in perspective.

With the journal editor as a helpful and responsive middle-man (preserving the “double-blind review process”) we even managed to meaningfully iterate with the reviewers during the review process, and the paper is now accepted for publication in an open-access journal, with publication pending the editor doing editorial stuff (fixing commas, making sure all sentences are clear and correct English, doing the formatting, and such).

The academic publishing process (rightfully) can be, and is, decried for many reasons: authors doing the scientific work, required to be signing over copyrights to journal publishers charging exorbitant publication fees while keeping the scientific work “paywalled” … and whilst using volunteer scientific peer reviewers doing the “quality control” for free – and with academic recruitment and tenure processes being contingent on having “published in journals”, these publishers are having a very captive market.

But, in this case the academic publishing process worked: the journal editor selected competent and interesting reviewers, who provided substantial feedback, then ran the double-blind review process in a way ensuring that it did not stand in the way of meaningful exchanges between the authors and the peer reviewers – and, now, the “editing” of the final print.

So, in this particular case, I feel that the journal publisher had a process in place that did add some actual value to the authors – feedback more akin to what I see when attending (good) conferences.  Was that value commensurate with the publication fee? Hard to tell – but enough value to make me consider this journal for future submissions.

One thing is sure: the value experienced by authors is very much dependent on the efforts and qualities of the reviewers.

Professor Rodney van Meter, KEIO University
Professor Rodney van Meter, KEIO University

A good friend of mine, professor Rodney Van Meter from Keio University (who is working on quantum communication, quantum architectures and generally post-More’s-Law computing), recently recounted having spent days very carefully reviewing a submission made to one of the big science journals (I forget which it was, but on the level of “Nature”), and discovering that the authors had committed a subtle, but important, mistake, basically invalidating the work presented.

As such, Rodeny’s efforts as a reviewer represented an immensely important contribution to science (and, given that I know Rodney to be one to provide complete arguments and detailed comments, one of extremely high value to the authors) – but, also, an immensely invisible contribution, which he could only afford to make because of his “academic seniority”: it would not be something that a young faculty member could put on his or her CV for recruitment or tenure purposes, or that a less young faculty members could put on grant applications, for example.