You can run it at https://uncommon-ground.xyz/

The rules of the game

This is played by teams (4–6 people each is a good number, 3-6 teams is another good number). There are two phases.

In the first phase, reach team writes down things that everyone on the team has in common. This first is what I had played before, which was kind of fun, but the second half of the game was a bit of a letdown: the teams then read their answers aloud, and the other teams just nodded along.

Therefore, I added the second phase: the answers are revealed one at a time, and the other teams get to challenge them. The rule is that to write down a commonality, every member of the team must share it, but to challenge it, it is enough for all but one of the challenging team to have it.

You get 2 points if your answer is unchallenged, and 1 point for every successful challenge. The team with the most points at the end wins. The challenge mechanism means that while writing something basic like we all had coffee today is a legal move, it is a bad move.

You can run it yourself, if you’d like: https://uncommon-ground.xyz. You need a projector/big screen and an MC to run it and every team needs someone with a phone to join the game (just one).

A few design choices

Only the captain’s phone is in the game. Each team designates one captain, and only the captain’s phone is in the game. In fact, if a second person joins the team, that triggers the first person to be kicked out.1 This is intended to keep the game social and have people look at each other instead of their phones.

We need a human MC. When an answer is revealed, challenges are shouted out and adjudicated by the MC. This was again, about trying to make the game more IRL.

Multiple teams can score challenge points for the same answer. I did not want to make this a first to buzz game. Let people talk it out. Even if another team has already challenged an answer, you can still challenge it as well, so you are incentivised to keep thinking and discussing with your team.

Answers are shown one by one. Showing the answers is also part of the game, so they are revealed one at a time. This probably slows down the game a bit, but you do want to give time for everyone to process information like we all lived in Helsinki at some point and think about whether they can challenge it.

This was obviously vibe coded

This was almost completely vibe coded with Claude. I wrote the rules in bullet points and Claude designed the app. I iterated a few times, but I hardly looked at the code. I asked it for deployment instructions and it gave me a few things to copy & paste into my AWS server and that was it. Almost all my decisions were design decisions.

The biggest category of mistake it consistently made was that it was consistently fragile if users have flaky internet connections. This is hard to encode as tests too.

• The creation of instant coffee. What I found most interesting was not the historical perspective, but that innovation continues almost to this day.

• I have a soft spot for reading about fraud. I love this detail: “He needed something on the scale of a lottery win to bail him out — which is exactly what he got. In November 2012, Lhéritier won €170mn in the EuroMillions lottery. “A huge relief,” he said, ‘which allowed me to inject €40mn of personal funds into the company’ to keep it afloat.”

• Where to Find the Colors Your Screen Can’t Show You

Travels: Aug 16–21, I will be at ISME20 in Auckland (New Zealand), feel free to reach out

Tweets

Photo

• I’m Not Crazy, I’m Just Becoming an Orthodox Jew

• An Indonesian “researcher” faked a number of studies and traveled to Denmark to present them under different assumed names by changing hijabs between presentations (/ht @MishaTeplitskiy)

• The 1994 World Cup in the US is still the highest attendance World Cup, either by total or by average attendance [Wikipedia] (h/t someone on Twitter):

  

Tweets

Photos

In Brazil, there is no standard electrical voltage, so individual plugs will be marked with the voltage that they carry1.

This was motivated by last week’s discussion on authorship and responsibilities. I think that there was a lot of talking past each other because for some people it was not obvious how you could be a co-author on something that you had not, in any meaningful sense, written.

Thus, I thought it would be useful to write down some observations about publications that involve large collaborations, which I am going to call Big Life Sciences. These are partly answering some of the questions that came up (mostly on Twitter) and partly more general observations.

The text is not the paper

In the sciences, and certainly in Big Life Sciences, the text is not the paper.

One can be a co-author without having written a single word of text because the paper is the data, the analyses, the ideas, the figures, and (also) the text. The paper may even include elements such as code on Github, a website, data on a public repository. These are all important parts of the paper and the text may not even be the most relevant element. Thus, one can legitimately be a co-author without having written any of the text (by having contributed to the data analysis or the figures or data collection).

Everyone is a co-author, but some are more co-author than others

The author list can be broken down into three categories: the first author(s), the middle authors, and the last author(s). The first and last authors are the most important and get the most credit, while the middle authors did some crucial work but are not the main drivers of the project.

Typically, the first author is the junior person (student or postdoc) who did most of the actual work and the last author is their supervisor. The middle authors are other people who contributed to the project, but not as much as the first author. The first author is not just the person who did the most work, but also the person whose McDonald’s it is. When the journal administratively rejects the paper and you need to resubmit because the figure files did not adhere to the right naming convention (they need to be called Smith_Figure1.tif and not Figure1.tiff), the first author is the one who will be doing the work to fix the problem and resubmit. If a co-author is flaky, the first author is the one who will be doing the work to chase them down and get them to do their part. It is their McDonald’s.

The last author is the corresponding author (technically these are not the same, but let’s keep it simple). They take responsibility for the paper and act as the point of contact for others to ask questions.

You can have multiple first and multiple last authors, but some are firster and laster. This is signified by an asterisk next to the names of the first authors, and a footnote that says “These authors contributed equally to this work”. Similarly, multiple last (or corresponding) authors can be listed. This is a way to give credit to multiple people who contributed significantly to the work. In practice, the first first author usually gets more credit than the second (or third) first author, and the last last author usually gets more credit than the second (or third) last author. This often emerges from collaborations between different labs where there is not clear hierarchy of who is the first and last author, but can also emerge from collaborations within the same lab where there are multiple people who contributed significantly to the work.

Middle authors get less credit than the first and last authors, but they can be important contributors to the project. In fact, collectively, the middle authors will often have contributed most of the paper!1 They may have contributed specific analyses or specific expertise that the main authors are unable to perform themselves. This is the main reason for collaboration and interdisciplinary work. Bioinformaticians have long bemoaned how they are often crucial contributors to a project but get relegated to middle authorship which is not adequately rewarded. Middle-author disease is a term I have heard from many.

How does such a paper get written in practice?

There is a lot of variability in this process and unlike the question of what it means to be first author, there less of a cultural consensus. I have seen a Google Doc that is shared from the start that everyone can see and edit. At the other extreme, I have seen a first author ask individual co-authors for bits and pieces, then stitch them together before sharing what is very close to a final draft with everyone.

In my group, I tend to advocate for the following: first author writes first draft, which is shared with me and I start to co-edit. Then, we progressively share the draft with individual co-authors, folding in their comments before sharing with the next co-author. You only get someone to read it for the first time once, and those cold reads are very valuable. Eventually, everyone will have seen it and provided comments, and we’ll have one final call for comments before submission.

In any case, it is very easy for someone to end up in a situation whereby they have legitimately contributed to a paper and should, by the norms and ethical standards of the field, be a named co-author, but they have not seen the text until the very end and may even not fully understand all of it. One of the advantages of collaboration is that you can have people with different areas of expertise contributing to a joint effort. This is not per se a problem, but it does cut against strict liability for all co-authors.

Should different sections of the paper be written and signed by different authors?

This was one of the suggestions that cropped up in discussion. After a bit of thinking, I actually believe that this is not a good idea. In as much as there is value in having a coherent structure and voice across a paper, it is harmful to have too many cooks in the kitchen. For example, in our manuscripts, we have decided to use the term habitat to describe different places microbes live. This was the result of a lot of discussion and debate, and we have decided that it is the best term to use. If we had different people writing different sections of the paper, we might end up with some sections using the term habitat and others using the term environment or niche. This would make the paper less coherent and more difficult to read.

When we have everyone edit together, it is easy to end up with duplications or unfired Chekhov’s guns (terms that are introduced and then never mentioned again). Also, people are generally better at identifying problems than solving them, so often co-authors will provide an edit that correctly pinpoints a problem, but their solution is not the best. Thus, it is my experience that the best results are when you combine a large number of people providing input and feedback, but a core group of people who make the final decisions to maintain coherence.

I acknowledge that there is a tension between accountability and coherence, but I think it is best to accept a small loss of accountability in order to get better papers overall.

A few thoughts:

1. Different fields have very different practices & standards. For example, in some fields, papers may represent less than a year of person-work and have 2-3 authors who are all intimately aware of its content. In others, a paper can represent 10+ years of person-work, and have 20-30 authors several of whom were not even active at the same time. Working scientists can be quite field parochial and lack an almost complete awareness of differing standards, even in adjacent fields. I am perhaps especially aware of these because I have lived both under computer science and life science cultures1.

   In Big Life Sciences, it is common to have papers where some authors are not even in principle able to judge everything that is in a paper. This is the point of collaborating: it’s not just about sharing workload, but sharing expertise.2

2. Due process is good. I am not against penalties for bad science. I have written before that scientific fraud should sometimes lead to actual jail time and I stand by that.

   However, I will also stand on the principle that punishing all authors for something that they may not control is bad. This is true even if the action being penalised is also bad. A junior author whose contribution gets flagged will not be able to ever again be a normal co-author on an arXiv preprint for the rest of their career? Responsibility needs to flow to those who have the power to change things.3

3. ArXiv did a terrible job of communication. The announcement uses the modern passive aggressive style of “just clarifying our code of conduct, here, folks” to introduce a sweeping change. Making people go through peer review before posting on arXiv contradicts the whole concept of the preprint! They also go back and forth between claiming this is a general principle and that it only applies to the most egregious cases.

4. The anti-arXiv side also did a terrible job of communication. The biggest objection (to me and I think to all sides) is to holding all the co-authors equally responsible. However, then it does seem that we’re soft on AI-slop and not willing to take responsibility for anything.

Altogether, the whole discussion has been folks talking past each other and neither side has made its points very clear. At the end of the day, though, arXiv is an institution which needs be held to higher standards than a bunch of Twitter posters.

I will be in Brazil from Saturday to Friday (16-22 May) to present at Annual Meeting of the Brazilian Society for Biochemistry and Molecular Biology. I will also spend a few days in São Paulo afterwards to visit the University. Feel free to get in touch if you’d like to meet/chat!

Links

• Calibrating for absolute microbiome abundances without spike-ins by de Wit et al. I mostly want to highlight this calculation: “[If you] sequence 50 million paired-end reads of 150 bp, you would end up sequencing 15 Gbp weighing 16.2 pg.” Somehow, it had never occurred to me to ask how much mass of DNA a typical metagenome represents. 15 Gbp is pretty typical and roughly 1pg/Gbp is a good number to remember.

• Day job link: The elusive resistome: a global comparison reveals large discrepancies among detection pipelines has been a long time in the making, but we have a preprint now: different antibiotic resistance gene pipelines return results so different that we can craft conflicting biological stories out of them!

• This earlier Rabbit Thought is was motivated by AMP detection, but also applicable to the problem of antibiotic resistance gene identification: Why are bioinformatics results so full of false positives? There is a market failure in our attention/citation economy where false positives are rewarded rather than punished.

Tweets

Photo/painting

Generally, job training is a tricky problem in the modern world because if a company invests in training a junior person, the moment the person has improved their prospects, they can leave. There are two common mechanisms to solve this in both the private sector and in academic science: (1) deferred compensation and (2) junior tasks.

When it comes to deferred compensation, companies will often award their workers a bonus that triggers after a certain amount of time: leave early and you will not get this extra payment. In academic science, we use exactly the same system for PhD students! If you leave early, you do not get a degree. At the start of the PhD, students receive a stipend despite producing very little, while in their last year, they are often getting underpaid compared to their abilities.1

The other way to solve the training issue is by assigning junior tasks. These are often simple things (e.g., literature review, website design,…), but require doing. Thus, a win/win trade emerges: we can get them done while also providing a training experience. As Luis Garicano recently wrote, though, in the Age of Claude the junior tasks are exactly those that AI does well. The old bargain whereby you execute junior tasks for me and I train you so that you can eventually do more interesting work breaks down if many junior tasks are not valuable anymore because Claude can do them better than junior people can.2

A PhD program combines training and scientific production. These are not the same goal, but they are synergistic, with the need to produce actual science functioning as a sort of export discipline on the training (see unteaching school again).

I recently read The machines are fine. I’m worried about us citing the wonderful Why do we do astrophysics? by David Hogg:

David Hogg, in his white paper […] argues that in astrophysics, people are always the ends, never the means. When we hire a graduate student to work on a project, it should not be because we need that specific result. It should be because the student will benefit from doing that work.

This assumes that the training can be decoupled from need to produce output. While it is conceptually possible for a training program to work without the export discipline of publishing; it is economically very difficult to do so, especially at scale. I don’t mean financially difficult, I mean that the incentives do not work out.

Export discipline is the idea that good state-led industrial policy should reward companies that are able to export (and not just sell on an internal market which can be warped by state subsidies) because the global export market provides a check on quality. If graduate schools were simply about training without the need to produce high-quality science, then how do we avoid them degenerating into inward-focused school, just a continuation of the students’ previous training? Only with the export discipline of needing to produce materials that will pass external peer review, can we reliably teach students how to do it. If we want students to produce writing of enough quality to be read (and not just graded), it is very helpful that they produce writing that is going to be read.

If the current model depends on conditions that are weakening, what replaces it? How should PhDs evolve? I don’t know. This is not school, we have problems that we cannot solve (yet).

Links

• Research has customers: “Researchers (and generally people marinating in mainstream culture) are awash with the idea that good research ‘deserves’ funding.” I think this often comes from the Busy Town1 model of the world, where there is a role called researcher (and butcher and taxi driver and …) and a good society will have someone filling each of those roles.

• This World of Ours: “Security people need to get their priorities straight. The “threat model” section of a security paper resembles the script for a telenovela that was written by a paranoid schizophrenic. In the real world, threat models are much simpler. Basically, you’re either dealing with Mossad or not-Mossad.”

Tweets

^I fear this may be very true. Average agency is over

^The culture loves the quirky accidental discovery, but most discoveries are the product of an industrial process that produces them.

Subscribe now

Photos

School (before PhD-level) is almost solipsistic: everything that matters is what happens inside the student’s mind. When my kid’s high school history teachers asks for an essay on the Australian Gold Rush, they do not do this because they have concluded that the world is missing essays on the Australian Gold Rush and that my kid is in a good position to help. They want to inculcate skills. If the essay itself is lost before anyone else reads it, there is no loss to the world.1

At the end of a PhD, though, a student should be an actual researcher. If they are learning, that is a perk of the job (and jobs should have perks); but the point of their work is the output: new knowledge for the world that they communicate to others. Thus, a PhD program takes a person who generally has a lot of technical skills, but has been brought up to act in a school-like manner (inward-focused), and turns them into a researcher who is contributing to science (outward-focused).

The inward- vs. outward-focused mentality subtly changes everything. For example, when a student is presenting to their supervisor, the supervisor knows the material and is evaluating whether the student knows it too. At the end of the PhD, the student should be presenting to audiences who don’t know the material but need to learn it. Several of the formal skills for good presentations (e.g., good pacing, clear graphics) are the same between these two types of presentation, but inward- vs. outward-focused are completely different orientations.

In school, receiving feedback is generally bad. The ideal homework gets A+ and no corrections. When working on a real paper for publication, the first draft is just a draft and feedback and iteration are part of the process. Having been through 20 years of “feedback is bad”2, it’s not easy to immediately switch to “feedback is good.”

On group projects, John D. Cook put it well:3 “The best teams have people with complementary skills, but similar work ethic. Academic assignments are the opposite.” As researchers, academics often work exactly in such complementarily skilled teams. But as teachers, group projects are rarely designed that way, because such designs do not fit the schema of classes and grades.

In case it was not obvious, this is partly a preamble for a discussion of LLMs and the AI Becker problem.

Related previous Rabbit Thought:

There is a tremendous amount of information regarding evolutionary history and biochemical function implicit in each sequence and the number of known sequences is growing explosively.

— Margaret O. Dayhoff (1967)1

Margaret Dayhoff’s Atlas of Protein Sequence and Structure started as a book of 100 pages in 1965, it had exploded to 400 pages by 1969.2

1983: “The need for efficient methods… has become evident”

“With the development of large data banks of protein and nucleic acid sequences, the need for efficient methods of searching such banks for sequences similar to a given sequence has become evident.”

– Wilbur & Lipman, 1983

1985: A 200-residue search in under two minutes!

“Comparison of a 200-amino-acid sequence to the 500,000 residues in the National Biomedical Research Foundation library would take less than 2 minutes on a minicomputer, and less than 10 minutes on a microcomputer (IBM PC).”

– Lipman & Pearson, 1985

The entire protein universe was half a million residues. A two-minute search felt fast.

1991: “Biologists must become computer-literate”

“To use this flood of knowledge, which will pour across the computer networks of the world, biologists not only must become computer-literate, but also change their approach to the problem of understanding life.”

– Walter Gilbert, 1991

1994: “An information explosion that continues unabated”

“The advent of rapid DNA sequencing technology in the mid-1970s led to an information explosion that continues unabated today.”

– Altschul et al., 1994

1997: “Exponential growth… a challenge for both end users and database curators”

“The exponential growth of DNA sequence data has become a challenge for both end users and database curators alike. When one of us (M.S.B.) was finishing graduate school, GenBank (release 42) contained a mere 6.7 Mb in 9700 sequences. However, as we write this, GenBank has topped 1000 Mb in >1.6 million sequences.”

– Ouellette & Boguski, 1997

1998: “Databases are growing rapidly”

“The available sequence databases are growing rapidly and are partially redundant in content. This leads to increasing strain on CPU resources and decreasing density of first-hand annotation.”

– Holm & Sander, 1998

1999: “GenBank continues to grow at an exponential rate”

“GenBank continues to grow at an exponential rate. Over the past 12 months 770 000 sequences were added… GenBank now contains approximately 1.8 genome equivalents of finished sequence and approximately 1.8 genome equivalents of unfinished sequence.”

– Benson et al., 1999

2007: “Vastly greater than the 96 obtained with modern sequencers”

“In the last few years methods have emerged, which for the first time challenge the supremacy of the dideoxy method. The common feature of these methods is that they are ‘massively parallel’, meaning that the number of sequence reads from a single experiment is vastly greater than the 96 obtained with modern capillary electrophoresis-based Sanger sequencers.”

– Clyde Hutchison, 2007

2025: “Growth and accumulation… have outpaced efficient analysis”

“The breadth of life’s diversity is unfathomable, but public nucleic acid sequencing data offers a window into the dispersion and evolution of genetic diversity across Earth. However the rapid growth and accumulation of sequence data have outpaced efficient analysis capabilities.”

– Chikhi et al., 2025, bioRxiv

(This is from a talk I gave at BrisJAMS last month and I will probably be posting a bit more from my notes on it)

• Television and live sports. I liked these paragraphs:

  • Bars were critical to television’s early success. ‘More people see television sets in bars than would ordinarily see them in showrooms’, said the director of a Chicago station. ‘The bars are helping sell sets to private homes’. Although some bartenders complained that sports fans contented themselves with a single beer rather than multiple cocktails, most reported that television brought in more customers, ultimately raising sales. Sometimes a bar would place a large magnifying glass in front of the screen to make the broadcast more visible at a distance.

  • Amid widespread fears of ‘juvenile delinquency’, television promised to keep kids out of trouble. ‘There’s no juvenile delinquency here’, proclaimed an ad for TV sets, portraying a family seated around their living room console. [it seems kids were touching too much grass back then — Luis]

• Update on the new eLife model. Submissions seem to be down by a lot: “We received a total of 2,853 submissions in 2025 (compared with 6,396 in 2024 and 5,590 for Feb–Dec 2023).” I will eventually write my thoughts on the current publication system, but I just do not see the point of eLife’s model at all.

• Why is the sky blue?

• Day job link: We put out our quarterly update newsletter for March 2026. Focus was on Yiqian’s new preprint on urban soils.

Tweets

Spain blocking docker when there is football being played it too Spanish of a concept:

One of the best things you can do right now is train your intuition for what are LLM-shaped tasks. That is mostly just trying to throw more LLMs at more things

Photo

Links

• This reads like a reductio ad absurdum of the naturalistic fallacy: most teens seem to think that cigarettes are healthier than vaping because of “chemicals”. /ht on Twitter

• Very interesting on bank tellers and automation. Like JD Vance, I only knew the first half of the story.

Tweets

Very interesting math fact:

And this is a nice addendum

• A mushroom that reliably makes people hallucinate little people.

• The mundanity of excellence. I read this every few years, very good!

• School is worse than social media. Note that this is mostly about high school being bad. My general opinion is that primary school is good, grad school is good; in between, it’s dubious. Also, the social media moral panic is completely unjustified by the evidence.

• On that classical cookbook, Microwave Cooking for One: “[ T]he true frontier for microwave-only cooking [is] the Maillard Reaction.Around 150°C, amino acids and sugars combine to form dark-colored tasty compounds, also known as browning. For a good browning, you must rapidly reach temperatures well above the boiling point of water. This is particularly difficult to do in a microwave – which is why people tend to use the microwave specifically for things that don’t require the Maillard reaction. But this is because people are weak. True radicals, like Marie T. Smith and myself, are able to obtain a perfectly fine Maillard reaction in their microwave ovens.”

Tweets

Photos

Happy Year of the Horse, a story in three parts!

Links

• The uncertain origins of aspirin. I myself thought that willow bark tea was a traditional remedy in Europe and then someone at Bayer had figured out how to synthesize the same molecule. Turns out this is a very simplified view of what happened.

• Pessimistic results on involuntary hospitalization (including a very understated conclusion sentence). Mental health disabilities are one really hard problem in policy.

• Day job link: We posted our quarterly update newsletter earlier this month!

Tweets

Photos

I will be in Luxembourg through year’s end and in Lisbon in early January before returning to Brisbane. Feel free to reach out if you’ll be at any of these locations!

Links

• “Even in the medically well-equipped United States, Sin Nombre virus has maintained an astonishing 40% mortality rate.” [Eukaryote writes blog] — One can hope that this is an over-estimate from milder infections not being diagnosed

• If you hear fake watches, you probably think “fake thousand-dollar Swiss watches”, but apparently, there are also fake 15 dollar watches [H/T Tom Whitwell]. Related, see this history of the Swiss watch post-Japanese competition (interesting how new it all is).

• From the Psychology has always been fake archives: Oliver Sacks mostly made it up (h/t literal banana).

• Canada lost its measles elimination status. A big reason is that Mennonite communities have low vaccination rates. Part may be general religious distrust of secular authorities, but a big part seems to be a genuine language barrier: not enough medical professionals speak Low German.1

• I will fail on my 2025 goal of publishing 30 substacks! I started a market for 2026.

Photos

Since my last post here, we also posted on our group substack, so check that for professional news:

Tweets

★

★

★

There is damning with faint praise and there is this:

★

Especially in Europe, the abrupt end of every winter/xmas celebration the day of always seems completely absurd. That even secular things like the city open air ice rinks close on the 25th of December rather than stay open through January makes no sense. Where is that post-Christianity when you need it?

★

★

Photos

I will be in Barcelona for the Decoding Microproteins Across Evolution and Disease (Gordon Research Conference) until the 22th. I will visit beautiful Valencia, Luxembourg, Cork (Ireland), Lisbon, and finish my mini-Europe tour in Heidelberg for the EMBL Human Microbiome Symposium (Sept 16-19). Feel free to reach out if you want to meet at any of these locations.

Links

1. Numbers that are surprisingly large: the fraction of men that are convicted of a serious felony: “In Sweden, a study found that 1% of the population was responsible for 63% of violent crime convictions. But the same study also showed that ~7% of men were convicted of a violent crime in their lifetime (Falk et al., 2013). In Denmark, depending on birth cohort, about 6–7% of men receive an unsuspended prison sentence at some point in their lifetime (StatBank, STRAFFO2). In the same birth cohorts, about 7–9% of men are eventually convicted of a violent crime (StatBank, STRAFFO1).”

2. Delete your old emails to save water is a message that you receive from British institutions that do not think themselves as intellectually clownish. (As usual, Andy Masley did the maths)

3. How fentanyl gets smuggled into the US (NY Times)

Tweets

Photos

I will be in London from Tuesday (15 July) through the 19th1, and then in Liverpool for ISMB/ECCB2025 and stop at University of Warwick on the 25th before heading back home. Please come say Hi if you

Links

1. Movies with the biggest gap when ranked by women vs. men

2. The Medical Evidence Project. For background, “The DECREASE were a family of trials published by a university in the Netherlands [Erasmus University]. They reported that giving patients undergoing major surgery died a lot less (note: a lot less) if they were given beta-blockers. […] Unfortunately, this was also ‘negligent’ and ‘scientifically incorrect’. As the DECREASE trials, but particularly the first one - DECREASE I - were included in a meta-analysis of all of the studies available, the European Society of Cardiology did not explicitly recommend against their use for several years. This killed a lot of people.”

3. The anti-memorization drive is another classical example of 2nd generation literalism: too much schooling was based on memorization, but we have since taken the critiques of that too far in the other direction. This is a good article on why memorization is important.

4. A randomised trial shows that using AI makes programmers about 20% slower even while they think they are about 20% faster! A lot of quibbles in the discussion, but it is important that the authors included some pre-registered predictions of this result and neither economists not AI experts got it right. My current views is that LLMs are great, but it’s vital to learn to recognise the tasks where they will perform very well and those where they won’t.2

Tweets

Photo

Earlier this month, I saw that there will be a new edition of the conference in 2025 and there is an application process, so I decided to re-listen to the series and write down a few notes to add to my application.1 These are the notes, in no particular order.

Some of the discussions suffered from Seeing Like a State where the formal systems are assumed to be the real systems. As I've noted before, the formal system and the real system are not always the same thing.2

In particular, I rarely find that the terms of my grant funding restricts what I do beyond some very broad topics. If I have a grant to work on antimicrobial resistance, I can normally work on whatever I think are the major problems in antimicrobial resistance because I wrote the grant where I described what I think are the major problems in the area. There is a certain amount of grantsmanship in not over-committing to particular paths, but I have also written to programme officers at funding agencies saying effectively hey, we were thinking of doing X and that is what we wrote, but because of Y, we now think that the overall goal of the grant is now best served by Z. POs are reasonable people who respond to reasonable arguments and the general response is sounds reasonable, thanks for letting me know. Sticking to the grant is very low on the list of reasons why I do not pursue what I think is the most important science.

The view from Stanford. This is a view which sees young people as incredibly agentic and motivated with great ideas who are then forced to jump through several hoops before being able to truly pursue their visions. With no disrespect to students, in the life sciences, this is not really what happens. It actually takes a long time before young people are even able to formulate a good problem. This is not due to lack of intelligence. It is just very difficult to get to the frontier of knowledge from the outside.

I will regularly meet with a student who will show me a recent paper (say, from 2024) and say that he is modeling their project on this published paper (for example, wanting to use a similar technique in a different context). Then, the following dialogue ensues. Me: "we could do better by moving to newer methods/technologies". Student: "but this is less than a year old, at a high-impact journal, from a leading group in the world, how could we be more up to date?" Me: "it was published in 2024, which means they submitted it in 2023, which means that the work was done in 2021. I know the group, had coffee with the senior author in Paris, they aren't doing it this way anymore." Papers are like the light from the stars, a few years out of date and subject to red shift.

I wish that people were applying for PhDs with well-formed ideas and telling me to get out of their way. Instead, it is a part of my job to bring them up to the state-of-the-art and help them design good projects. It takes years. In fact, several people will tell you that the expectation is that students can design a project at the end of their doctorate. Maybe, we should try to aim for a world in which the state-of-the-art is more legible, not hidden in private meetings and emails, but that would be a radical goal, not the world as it is now.

The guests are often pulling in opposite directions in unacknowledged ways. While some guests bemoaned the fact that PhD and postdocs spend years working on someone else's vision, other guests bemoaned the fact that it is difficult to assemble the large teams that are necessary to tackle large problems. While both of these views are critiques of the current system, they are opposite critiques!

I tend to side with the view that we would benefit from larger teams of stably-employed professionals instead of the current system where we have to firegraduate anyone who gets too productive, while at the same time minimizing the risk that we spend too much time contributing to larger goals, lest we get a bad case of middle author disease. However, this means that I think we should have more people who are working in science at the behest of other people's vision, not fewer!

Peer review: in the grant system vs. the publication process There are two points where peer review comes into the picture: grant review and publications. They are very different and have different costs and benefits. In general, I think in the modern publication system (which includes preprints), publication peer review works quite okay, although we could improve it on the margin. Peer review in the funding system seems much more random. In the discussions, the two meanings of peer review were often used partly interchangeably.

The lack of good metrics To be fair to the guests, this was often acknowledged, but still a few guests discussed the number of papers published as a relevant metric, but this would be like arguing that all companies are equally efficient because they all publish the same number of annual reports. Papers have grown in the content they pack and keep growing (see the point above about students modelling themselves on recent papers still failing to see that the bar is already higher).

In Australia and several other countries, a PhD is 3-4 years, compared to the US standard of around 5. Does this mean that Australian PhD students are 25-40% more efficient? Of course not, but different countries have different standards for what they call a PhD (which some people will use as arbitrage).

★

A few problems that I do encounter regularly, could benefit from some theorising and were not mentioned:

(1) Non-fungibility of contributions and unfavorable cross-discipline status exchange rates. If you’ll forgive the dense terminology, this is a huge problem. Academia is first and foremost a status economy, but except at the very very top (Nobel Prize), status is not fungible and has terrible exchange rates between disciplines. For someone who works in one area to contribute to another area is rarely prized. For example, if I were to spend some of my time, working on an important study on Progress Studies3 and this was a success in all fronts (paper in respected economics journal, cited in legislation that changed things for the better), it would still have few direct career benefits at it would not be legible to grant panels in the life sciences. Regularly, we have projects which would benefit from computer scientists, but would bring the computer scientists few benefits in their economy. Even between different subfields of computational biology there are no good exchange rates. Papers I publish in the microbiome field will get some readership based on my name and some recognition that I have accrued,4 but this will bring me no benefit if I publish in even a closely-related topic.

(2) As I discussed in another post before, the up or out system. It shapes everything, from project design to collaborations, to careers.

(3) The slowness of decisions and processes at all levels: grants regularly take up to a year between submission and decision, hiring decisions take 6 months, paper reviews take months... This was mentioned in the context of Fast Grants, but I think it was still underrated how it makes everything worse. Delays do not just slow things down, so that the same result comes later. People change their behaviour in response to the delays. Talent identification is incredibly hard if projects take several years and success has a huge stochastic component and leads people to focus on lower risk. Career switches can take up to a decade. To make a Ruxandrian point, long career limbos impact women the most.

★

The problem with lived experience testimonials is a tendency to miss the forest for the trees. It is also easy to get into the mode of complaining about the traffic which can sound like one is engaging in policy discussions.

Still, let's say I come into the office on a random Thursday and look around our Centre asking myself Why are the people here not producing better science faster? How could the Australian taxpayer be getting more bang for their buck? If instead of just a group leader, if I was a Dean, for example with some ability to steer the activities of faculty but without a huge budget, how could I make things better? I feel that Progress Studies should have some sub area that is trying to provide answers to these questions.

★

In retrospect, there is major foreboding of the current situation. Several guests, including Tyler Cowen, discuss the loss of confidence in scientific institutions, institutional blindness to this fact and vaguely gesture towards a coming backlash. The original series was recorded before Trump was elected and the backlash is now here.

Your society calls this blue. One day, a maverick comes along and says "Gross's square is green"!

Everyone looks at it and marvels at this insight! Indeed, it was never blue, it was partly green. They understand it was not purely green, but in the cultural background of having called Gross's painting blue, saying "Gross's square is green" captures something deep. The folks who still resist, going around saying "no, it's blue" are missing the point.

The next generation grows up hearing it being called green. They don't necessarily know it used to be called blue, though. They think it means that people see the square as green. Eventually some new maverick comes along and writes "Come On, Obviously Gross's Square is not Green!"

★

I have been calling this phenomonon second-generation literalism.

One generation believes A, which is obviously not correct. Someones says the opposite of A is true and this generation sees it as a helpful correction and arrives at a happy middle ground, but the next generation grows up only hearing the correction and takes it literally. In vaguely Bayesian terms, if your prior is blue, calling teal green, will probably land you in the right ballpark. But if you make green your prior, you will be wrong.

★

Scott Alexander recently wrote Come On, Obviously The Purpose Of A System Is Not What It Does. Originally the phrase was coined in a more naïve age and served as a corrective by highlighting that systems were sometimes captured by interested parties, that you needed to dig beyond the official mission statement. Like the original behaviorism, it served a purpose and later became limiting. Nowadays, we are—if anything—too cynical about institutions.

Other example of second generation literalism include rejection of the Great Man Theory of History. The original Great Man Theory was almost cartoonishly simplistic: "Universal History, the history of what man has accomplished in this world, is at bottom the History of the Great Men who have worked here." However, we have now come to the opposite extreme where people deny that individuals ever matter.

One interesting example is Noah Smith's assertion that "China and the U.S. are very culturally similar, rather than representing distinct, alien poles of Eastern and Western civilization." It is an interesting corrective to the alternative view. As a green to the current consensus of blue, we should welcome it, but it is very obviously false.1