jErOEn jErOEn 9 October 2013

Missed references

Posted in Ethics

no-creditAs a scientist reading a substantial amount of papers I regularly read articles with inadequate referencing. Obviously some papers contain too many self-citations, however my main concern is about missed references. Why are the (co-)authors missing references to crucial and important work of others or, more worrisome on a personal level, to my own work? Here I would like to discuss this issue in more detail.

The most common way to disseminate scientific knowledge is through the publication of peer-reviewed articles. An important aspect of any scientific paper is the inclusion of references. The purpose of references are to acknowledge your sources of knowledge, to guide readers to other papers that explain details of the work presented, and/or to show the context of the work in relation to other peoples’ work. The latter purpose is to indicate similar work, competing methods, and/or alternatives to the presented work. References enable other people to repeat your research, to follow-up on your research by applying your methodology/results to other areas, and/or to use your work outside the context of the original work.

Proper use of references supports and strengthens key scientific values such as

  • originality: who was first with an idea/method/invention
  • research impact: is the work influential or not
  • scientific content: it is not the way you present, but what you present that matters

Not properly referring to the work of your fellow scientists reduces the emphasis of science on the above mentioned values. For many of us writing scientific papers the inclusion of appropriate references is taken with great consideration. However, I regularly observe missed references, something that needs to be opposed. I classify the missed references into three categories:

1. ‘Missed reference’ of similar work

This is the most obvious ‘missed reference’ and should be counteracted strongly. Although it is difficult to have an overview of all literature in your field, it is of critical importance for any author to make reference to the work of others who have done work similar to (part) of the work you present. Moreover, not giving proper reference to other similar work leads to advocating novelty where there is none. If we, as a scientific community, permit missing references to similar work, hence permit experiments/theory to be presented as novel, we devaluate the scientific value of originality, research impact, and scientific content.

2. ‘Missed reference’ of competitors

In the fierce competition between researchers it is important to stay ahead of the competition. To create the image of being the leader in a field it is of importance to emphasize one’s results and de-emphasize the results of competitors and earlier research. For example, an alternative method with similar end results may be ignored to favor one’s own approach to the problem. I have the feeling that sometimes references are missed with the aim to reduce the competitors’ citation rate, and enhance one’s image of innovativeness and originality. This situation is clearly not favorable and it is the task of the journal editor to have papers being peer-review by competitors and/or suggest the inclusion of crucial references relevant to the work.

3. Missed reference’ by ignorance

The ‘publish or perish‘ maxim in science creates a strong publication pressure and therefore results in high throughput writing of papers. As a result, a proper literature survey is sometimes not performed. I observe that papers including several similar key words in the title have not been referenced to. In this age of search machines and online journal availability this can be an indication of sloppiness and/or ignorance.

In general, the peer-review process is a check on the inclusion of appropriate references in articles. However, its emphasis is primarily on scientific content and not on appropriate referencing. As a reviewer of a paper with missing references you can suggest the inclusion of references crucial to the paper. Yet, most of the time you read an article with inappropriate references after publication. The question is what to do with ‘missed references’, especially when they concern your own work?  It is clear that any action is always after the fact and will not result in reclaimed references to one’s own work. Based on the level of reference misrepresentation (see classification) I advocate contacting authors of papers with missing references as it is important to show commitment to your own work. Perhaps, in the future they will properly reference to your work and in case you meet one of the authors at a conference they cannot pretend of not knowing of your work. Finally, it is the responsibility of the editor to promote proper referencing as an editor should have an overview of the work done in his/her topical research area. If you suspect systematic incorrect referencing practices by authors, I recommend contacting the editor as well.

Although the inclusion of references is most of the time based on a best effort of the authors, we scientists should be aware of the value of proper referencing to other work and guard for its appropriate use.

- - - - - -
If you like this post why don't you email subscribe to our new posts. Or subscribe to our RSS feed.
  1. Unregistered

    9 Oct 2013 22:47, moom

    I ignore papers that are simply wrong but somewhat similar so I don’t have to write about why they are wrong. There are large numbers of such papers in my field in my opinion, a lot of them in weak journals. One can’t refer to all this literature. So, this isn’t as clearcut I think. Or maybe things are just different in the social sciences where I work.

  2. Unregistered

    11 Oct 2013 10:53, Bingo Crepiscule

    Good article, Prof. Kalkman! This is a very important issue, which needs to be tackled strongly. The first 2 of the 3 categories fall under the adage: “The secret to creativity is knowing how to hide your sources.” This goes so against the very grain of what we do in science namely try to uncover the truth and report it honestly. The 3rd category is plain sloppiness from the authors, as you mention, since one expects a thorough literature search to be performed from those who wish to advance the science by publishing, and therefore purport to be experts in the field.

    In addition to the 3 categories you mention, there is a 4th and perhaps a 5th and 6th. The 4th is to purposely leave out references of work conducted many a time in labs and universities not familiar to the authors or not famous enough. I think the Russians suffered the brunt of this in the 70s and 80s, even though a part of the work was published in translated English language journals. I think the current victims are labs in China and to a lesser extent India, where author lists are populated with unpronounceable names. I have unfortunate to say also been guilty of this when I OKed a manuscript (letter with limited number of references) from a PhD student where the work of a more famous Chinese American was duly cited and not that of a Chinese author in China. I realized this only after the reviewer pointed this out. In all honesty I think I OKed this because it was more likely that our contribution would be reviewed by the American than the Chinese person. I hope that I will never fall in this thinking again.

    The 5th category is a very devious one when references are used in a misleading way. I recently reviewed a manuscript to a journal from the Nature stables, from a reasonably good group (but a very street smart one). They claimed their new method showed (tested in animal models) far better performance than conventional methods [6 references]. Only they very conveniently omitted to mention that these references deal with work on human patients. You do not mention performance in animal studies in the same sentence as human studies.

    The 6th category is a bit like the 3rd. It is really spreading a lie, due to sloppiness. You read in the work from an influential scientist who has unfortunately followed one of the categories 1 to 5 above. Many authors then take the references from this influential work, and use similar standpoints without checking the literature independently. And this simply goes on and on.

    I think your suggestion to contacting authors of papers with missing references after publication is a very important and useful one. As you say at least in the future the guilty authors will probably properly reference your work.

  3. Otto Muskens

    30 Nov 2013 17:17, Otto Muskens

    A particular problem is that a typical article can only have around 30 references, while scientific literature is growing exponentially. So it is often impossible to cite every relevant paper in the field, and one has to make a decision whether to keep citing the old papers where an effect was first discovered or to include the latest developments which may be more relevant. In order to make their articles look more sexy and give a suggestion of activity of their field to the editor, many authors often choose the latter.

    Also often people cite a review of a field rather than giving credit to the original research papers. However reviews can also help gaining visibility if your own research is included.

  4. Unregistered

    29 Mar 2014 3:53, Willem Vos

    While I mostly agree with Jeroen’s recommendation that in case you feel your own work is not properly cited you contact the relevant authors, I do not like the kind of “letter of complaint”, as it easily turns into or is perceived as “whining”. Whenever it occurred to me, I preferred to personally hand the relevant author a stack of old-school reprints. Usually the recipient will get the point.
    But let us also put a grain of salt on our actions to “protect our babies”: don’t we sometimes have a little tendency to overestimate the impact of our precious paper? Yes I plead guilty too. In hindsight – time is supreme to heal wounds – we may come to realize that our precious paper had inevitably a limited scope; for instance the more speculative aspects in our Discussion section were only truly demonstrated by the subsequent paper of our colleague.

XHTML: You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>


By submitting a comment here you grant this site a perpetual license to reproduce your words and name/web site in attribution.

Subscribe without commenting