"Application Notes" considered harmful
I have thought this for a long time, but I resisted writing on it because I am in general agreement with the values and goals of many of the proponents of Application Notes. This week, however, there was a kerfuffle because biorXiv appears to reject Application Notes style manuscripts so here is my write-up of why I think Application Notes are Bad.
What is an Application Note? I will use the term Application Note to refer to short publications that presents a software tool. The journal Bioinformatics has had these for many years, with a 2 page (1300 word) limit.1 More recently, the Journal of Open Source Software (JOSS) appeared as a venue for publishing similar Application Notes, and not only in bioinformatics.
Application Notes address a real problem. Publishing scientific software tools on Github, making them user-friendly, and providing excellent support may help science as a whole, may get you a lot of name recognition in the community, may even get you invited to give talks and do tutorials, but—at the end of the day—this counts for little compared to publications and citations. Therefore, people are dis-incentivized from producing good tools even though much of modern science is based on software tools. Therefore, we should reward good tools and Application Notes are a solution in that direction. Sometimes they are even called a way to hack the academic reward system, in that they transmute something that is not rewarded (software) into something that is (publications).2
Some people argue back that “software should not be rewarded as this is not what universities are for.” To this I say two things: (1) there is nothing sacred about what institutions should reward3 , institutions are for the people, not the people for the institutions. Historically-speaking, if universities were built for something, they were built for studying theology and we don’t do much of that anymore. But also, (2) in the case of scientific software, producing it is an absolute necessity on the way to the knowledge that everyone agrees scientists should work on.
Application Notes actually devalue research software development. On the surface, the goal of these notes is to make research software development more valuable (a goal I agree is a worthy one), but I think it ends up doing the opposite: it devalues it. Full-length manuscripts would be preferable.4
There may be some tools that really are very trivial, but, for many, I hope that some thought was put into building them. An internal module structure was designed, data structures and algorithms have had to be chosen, an API had to be designed, versioning strategies had to be put in place... All of these have intellectual value, potentially even intellectual innovation. A manuscript on the tool would—in principle—be an ideal place to explain these, but given the short size of the Application Note format, this is impossible.
As a particular weakness, Application Notes have space for only single-digit number of references. This keeps authors from properly acknowledging their debt to other software tools, further contributing to the notion that there are few intellectually-significant elements in a tool beyond using it as a black-box.
In the actual development of a tool, there is always a lot of borrowing from other tools in approach, in API, reuse of algorithms (including often from related areas, where the boundary between tool development and methods development gets fuzzy)... With the Application Note format, the authors from the original tool will not get those citations even if their work was the inspiration for the current tool.
Vaccine dose update: In the period 27th June to 27th of July, almost exactly 1 billion covid vaccine doses were given (996.94 million, according to OWID5). This is similar to the pace of June (1 billion per month) and recall that most of the doses are going to middle-income countries (in fact, as expected, the fraction of doses that go to high-income countries every week keeps going down and that number is now close to their fraction of the world population6).
Link of the Week
Tweets of the Week
Photos
All photos from Luxembourg this week.
It’s not even two pages of text because the journal format will take up almost half a page with the header and the title and author list….
Today, we might say that making software into papers is a way to make software legible to those who evaluate scientific productivity.
I think of the role of universities and other research institutions as “providers of intellectual public goods.” A widely used software tool is an intellectual public good.
Some people will engage in a reductio ad absurdum argument and ask “well, should universities reward developing general-purpose software like Linux?” I will personally bite the bullet and say Of course! In fact, universities should ask themselves why they didn’t develop Linux or any of the other widely-used open source packages. But even if you don’t take my radical “universities should produce intellectual public goods” view, I think there is agreement for bioinformatics and other cutting-edge scientific software, that this is necessary as an intermediate step for things that everyone agrees on.
I imagine that one reason why Bioinformatics started the short format was because paper was expensive, but this justification is now completely gone.
I’m giving the numbers here as they are given in the OWID website, but there is enough of a margin of error that this is indistinguishable from 1 billion.
In the period July 15-25, 22.6% of doses were given to residents of high-income countries and they have 18.5% of the world population. Naturally, the cumulative numbers are still lopsided, with high-income countries having consumed 31.5% of all doses.