Citations in Math Academia

Many Internet commenters have criticized MIRI for not producing enough research, relative to their budget – not writing enough papers, or not getting those papers peer-reviewed, or not getting enough citations. However, MIRI’s specialty is math and computer science, which might have lower citation counts than experiment-heavy fields like chemistry or biology. For a quick sanity check, I looked up a few non-MIRI mathematicians as points of comparison.

Grigory Margulis is probably the most accomplished mathematician I’ve personally met. He’s a Fields Medalist, won the Wolf Prize in 2005, and is doing pure math as a Yale professor full-time, so it seems reasonable to assume he’s in the upper quantiles of productivity. A Google Scholar search for the last five years turned up eight papers that he’s co-authored; by my count, those eight papers (combined) have 35 citations, of which nine are self-citations. All of those papers had multiple authors, so it took well over five person-years of total effort to produce them.

But of course, a Fields Medalist isn’t a representative math researcher. One friend of mine recently got a math Ph.D. from an elite university; as a grad student, they spent years doing math research full-time, and they also did a lot of part-time research in undergrad. They wrote several papers while in grad school, plus (of course) a dissertation, but these don’t appear to be on Google Scholar; presumably they’re still awaiting publication, or they weren’t published in a place Google indexes. They also published two papers before grad school, of which only one was peer-reviewed; these two papers have 13 total citations, of which five are self-citations.

Another friend of mine got a math Ph.D. some years ago, from a less elite university. They wrote four papers which appear in a Google Scholar search. Of those four, one wasn’t a math paper, and was published long after they graduated; one was their dissertation; one was posted on arXiv, but doesn’t seem to have been formally published; and one was published as a conference paper. Excluding the non-math paper, the remaining three papers have eight total citations.

Another friend of mine just got a math Ph.D. from an elite school, and is taking an academic job after graduating. They’ve written a number of papers, given talks, etc.; but again, a lot of these don’t appear on Google Scholar. Three of their papers are on Google Scholar, but all three appear to be arXiv papers that haven’t been formally published, and the three papers have three total citations.

But all that might just be selection bias in who I know. Using to pick two math postdocs – one from Stanford, one from Berkeley – their CVs list a combined total of eighteen papers, of which seven have been formally published, four are listed as “accepted” but not yet published, and the remaining seven are on arXiv or self-hosted. Of these 18 papers, the most cited one had a total of 13 citations, of which four were self-citations.

(Disclaimer: I’m not a math academic; comments/corrections from people who are appreciated.)

Universities vs. Scientology

A handy guide:


New York Times Makes Up Facts About SF Housing

Last Saturday, the New York Times published an article titled “High Rents Elbow Latinos From San Francisco’s Mission District”. (They later changed the title to “Gentrification Spreads an Upheaval in San Francisco’s Mission District”, without acknowledging the change.) The article opened with this sentence:

“SAN FRANCISCO — Luxury condominiums, organic ice cream stores, cafes that serve soy lattes and chocolate shops that offer samples from Ecuador and Madagascar are rapidly replacing 99-cent stores, bodegas and rent-controlled apartments in the Mission District, this city’s working-class Latino neighborhood.”

The statement is clear: “luxury condos are rapidly replacing rent-controlled apartments”. The problem with this “fact” is that it’s not true. The New York Times’ writer, Carol Pogash, made it up out of thin air to sell her story. As any journalist will tell you, nothing sells like conflict… and if that doesn’t mesh with the truth, well, the truth be damned.

How do we know that? Because San Francisco Planning keeps very detailed records of all city housing. We can just look up every Mission development since 2010, and see how many older apartments they replaced. The final total is… zero. Yes, zero. In the last five years, since rents started rising after the 2008 crash, not a single Mission apartment has been torn down for “luxury condos” (or any other new housing). But the NYT wants people to think rent-controlled apartments are being “rapidly replaced” for new development, so that’s what they’ll print, even though this has never once actually happened.

Here’s the full list of Mission housing developments built since 2010:

480 Potrero Ave., 84 units, under construction as of 2014; replaced a vacant lot
35 Dolores St., 37 units, under construction as of 2014; replaced abandoned auto shop
899 Valencia St., 18 units, under construction as of 2014; replaced a gas station
685 Florida St., 4 units, under construction as of 2014; replaced a vacant lot
39 San Carlos St., 3 units, under construction as of 2014; units over ground-floor garage
85 Brosnan St., 3 units, under construction as of 2014; replaced an office building
930 Shotwell St., 2 units, under construction as of 2014; replaced a vacant lot
1076 Hampshire St., 2 units, under construction as of 2014; replaced a storage building
1515 15th St., 40 units (7 affordable), built 2014; replaced an abandoned gas station
2421 16th St., 12 units (1 affordable), built 2014; replaced a vacant lot
3418 26th St., 11 units, built 2014; replaced a vacant lot
1600 15th St., 202 units (40 affordable), built 2013; replaced an abandoned factory
3500 19th St., 17 units, built 2013; replaced a surface parking lot
200 Dolores St., 13 units (2 affordable), built 2013; replaced a vacant lot
141 Albion St., 3 units, built 2013; replaced a vacant lot
857 Alabama St., 2 units, built 2013; replaced an abandoned storage building
3135 24th St., 9 units, under construction as of 2013; replaced a parking garage
1280 Hampshire St., 3 units, under construction as of 2013; replaced a garage
299 Valencia St., 36 units (4 affordable), built 2012; replaced a vacant lot
411 Valencia St., 16 units (2 affordable), built 2012; replaced a garage/auto repair shop
179 San Carlos St., 3 units, under construction as of 2012; replaced a vacant lot
555 Bartlett St., 60 units (9 affordable), built 2010; replaced a paint store
2101 Bryant St., 26 units, built 2010; replaced a hole in the ground
736 Valencia St., 8 units, built 2010; replaced a vacant parking lot

Theranos interview (2005)

An interesting May 2005 interview with Elizabeth Holmes, founder of the mysterious Theranos. Original audio here.

Dr. Moira Gunn: This is the Tech Nation Podcast from IT Conversations. The world we know is changing. This is BioTech Nation.

Dr. Moira Gunn: The Center for Drug Evaluation and Research at the FDA has just reported that in 2004, over 400,000 Americans reported adverse drug reactions, and 100,000 Americans died. On the benign side, other research shows that 40% to 60% of all patients don’t benefit from the drugs they’re prescribed. I asked Elizabeth Holmes, the President and CEO of Theranos: why can’t we figure out who will have an adverse drug reaction, and why certain drugs aren’t going to work?

Elizabeth Holmes: I think that part of it has to do with the fact that there is no mechanism in place to deal with monitoring patients on an individualized basis today. When we began Theranos, what we focused on was creating a customized medicine tool that could be used in the home by every patient. So that every day, a patient can get real time analysis of their blood samples, and look at not only how drugs are reacting in their body, but how other metabolic or physical factors contributed to how well a given drug works in them as an individual.

This is different from the traditional process of sending a patient into a clinic at random time points, trying to get a sample of their blood and then analyzing, at that second in time, what the drug is doing. Because it gives you a much better and much more complete understanding of all of the factors that contribute to how well a drug works or does not work. Like if the patient’s taking other drugs, which happen to cross interact. So, the ability to begin bringing monitoring, as we call it, into the home, we believe could fundamentally change the way that both patients are treated as well as drugs are developed.

Dr. Moira Gunn: I guess right now it’s called the RDX Metabolic Profiler?

Elizabeth Holmes: That’s absolutely right.

Dr. Moira Gunn: Well, we’ll get those marketing people on that. I think it may be called something else by the end, something handy by the end. Now tell us, exactly how big is it? What does it do? What do you got to do if you’re using it?

Elizabeth Holmes: So, it’s a handheld device and it’s fully integrated. The only thing you have to do is hold your finger, or you could actually use any part of your hand or your arm, up toward the device and it takes a very small sample of blood. So small that you can barely feel it. Thanks to the art of glucose monitoring, small blood sampling has really …

Dr. Moira Gunn: So it extracts a little from your hand?

Elizabeth Holmes: Exactly. It’s a little tiny needle that pulls a little tiny drop of blood, and when it gets the drop of blood, basically it runs it through, what we call, a biochip which separates out all the cells and other types of analytes in your blood which could traditionally clog a biosensor. And then, in real time it runs many different chemistries. Looking for different, in this case, targeted markers. Like the drug concentration, or concentrations of other proteins that may be in your blood that are indicative of either risk…

Dr. Moira Gunn: Adverse drug reaction.

Elizabeth Holmes: Exactly.

Dr. Moira Gunn: Okay, so once it … identifies that? What does it do? Just tell you?

Elizabeth Holmes: So, when you do that-

Dr. Moira Gunn: It has a big screen that says, “Sit down”?

Elizabeth Holmes: No. The patient doesn’t see anything. It’s a very small handheld device. So once the device begins working, it’s a real time event in which the blood sample is analyzed. When it separates all the cells out and it identifies the markers it’s looking for, the first that happens is you get a signal. It’s basically reflective of a concentration, or the presence or absence of certain cells you may be looking for.

And when that happens, the electronic aspect of the device takes hold and transmits that data to our website. Where we’ve written what is basically biostatistics algorithms to correlate that information, and profile it. So, we’re actually in the process of redesigning our websites, so that patients and physicians can log in, and a nurse can monitor this data and then say to the patient, “You know, you’re fine.”

Again, the backdrop to all of this is, when a drug is prescribed, we are coupling the system with the drug. So if you know, when you go to get a drug, that you have risk of an adverse event, or you’re not sure, or you’re nervous about it, you can monitor yourself for a month, and then evaluate whether or not that drug is the best drug for you.

Dr. Moira Gunn: Okay. I got to ask it, does it hurt to have the drop of blood extracted?

Elizabeth Holmes: I can tell you, personally, I hate needles. They make me want to faint, and I am fine doing with doing this drop of blood. We’re talking really really small. You can barely see it. If you poke yourself in the arm or on the palm of your hand, it doesn’t hurt as much as if you do it on the fingertip, because there’s many nerves in your fingertip. So it’s more of a pain site, whereas your arm has thicker skin. So you actually get an even smaller drop of blood out, but you don’t hit the nerves that make you feel pain.

Dr. Moira Gunn: And you’ve got plenty of blood to do your assay.

Elizabeth Holmes: Oh yeah. Absolutely. That’s the beauty of the technology, is that we’re really talking about miniaturization.

Dr. Moira Gunn: Now a lot of people are saying, “Well she’s President and CEO. Where are all the engineers that built this?” This is built around your patent.

Elizabeth Holmes: Yeah. That’s true. Part of the culture of our company is to make sure that we are fully integrated, so people who are working on business development and people who are working on marketing, everything, revolves around the engineering aspects and the technology aspects. And continually striving to be, really, the leader in creating an industry around these personalized monitoring systems.

So, yes, I am actively involved in the technology and the technology did come from… sort of an integration of work I’ve done in different technical fields. And the concept that if you could bring different technologies together, you could maximize the power behind them. I think it’s very clear that this is a wonderful time for the convergence of the electronic and IT infrastructure with biosystems. In our case, it’s to create biosensors.

Dr. Moira Gunn: If this is a job interview I couldn’t ask you, but I can ask you. And it’s the radio, so people are surprised. How old are you Elizabeth?

Elizabeth Holmes: I’m 21.

Dr. Moira Gunn: You’re 21.

Elizabeth Holmes: Yes.

Dr. Moira Gunn: So, you were at Stanford. What were you studying there?

Elizabeth Holmes: I was studying Chemical Engineering, but I was also involved with Electrical Engineering, and with some biosensing projects.

Dr. Moira Gunn: Okay, so you were doing all this. Did you actually build a new technology while you were there? Or did you drop out to do that?

Elizabeth Holmes: I actually did build other new technologies while I was there. I was working on a project for a major pharmaceutical company, a wireless biosensor. And I was working on another microfluidic project, basically dealing with fluids in very small volumes, which is relevant to what we’re doing now.

I actually left Stanford to go work in Singapore. Background on, I guess, story of my life. I have been studying Mandarin for a long time and have spend some time studying in China. I wanted to go back to Asia, but was interested in biotech. And so I went to Singapore, because there’s a tremendous amount of resources that are being poured into research there. I got the opportunity to help develop a novel approach in microarray, and was looking at that technology and thinking about the type of sensors I’ve built at Stanford. And realizing that if you could integrate the ability to do high-throughput screening, meaning the detection of many different types of markers into a little titanium chip like the ones that we had expertise building here… you would really have a powerful sensor, and truly a platform with respect to the ability to say, “Look, we’re going to work toward monitoring anything, anytime.”

Dr. Moira Gunn: What’s the status of the device today? Is it still just a prototype? Where are you?

Elizabeth Holmes: Okay. Our first applications are actually in monitoring acute pain killers. And that device is going into the production phase. We hope to release it, actually, to a pharmaceutical partner around mid to late this year.

Dr. Moira Gunn: So you’re almost there. You’re almost in manufacture.

Elizabeth Holmes: Oh, absolutely. I think it’s a iterative process, because what we look at is the ability to monitor different things just based on changing this little cartridge that slides into your handheld readers. We’ve got the reader, now we’re developing a series of different cartridges for different purposes.

Dr. Moira Gunn: Now, how much money have you raised thus far in venture capital funds?

Elizabeth Holmes: So, in venture capital funds we raised just over 6 million, and then we’ve also raised money from private investors.

Dr. Moira Gunn: Okay, and you’re 21?

Elizabeth Holmes: Yes.

Dr. Moira Gunn: Okay. I’m going to tell my 2 children they better get off their duffs, Elizabeth. I have one more question for left for you.

Elizabeth Holmes: Absolutely.

Dr. Moira Gunn: What are you going to do when you’re 30?

Elizabeth Holmes: This. We have ideas, and actually the way that we structured our company is to build what we call an innovation division. And already, we have next generations of this product in prototype form in-house. And that’s with respect to miniaturizing the system to make it faster. To make it more high-throughput. To put it into all sorts of different types of devices that can take use to the point where this is automated. and you don’t even have to touch your finger on the device.

Dr. Moira Gunn: No pain. Elizabeth this-

Elizabeth Holmes: No pain.

Dr. Moira Gunn: No pain.

Elizabeth Holmes: That’s the objective.

Dr. Moira Gunn: This has been terrific Elizabeth. Come back and see us anytime, and we really look forward to seeing you.

Elizabeth Holmes: All right. Thank you so much.

Dr. Moira Gunn: Elizabeth Holmes is the President and CEO of Theranos. For BioTech Nation I’m Moira Gunn. You’ve been listening to TechNation podcast from IT Conversations. For more information visit our websites at and I’m Moira Gunn. Thanks for listening.

The Chinese Robber Fallacy

The Chinese Robber Fallacy is where you use a generic problem to attack a specific person or group, even though other groups have the problem just as much (or even more so).

Suppose you’re racist against Chinese people. You can go on the Internet and say:

“Man, screw the Chinese. The Chinese are thieves.”

And when someone replies: “Hey, is that really true?”

“Yeah! Just look at <example> and <example> and <example> of these robberies by evil Chinese criminals.”

“Sure, but that’s just anecdotal evidence.”

“Our statistics say that Chinese people commit an average of <big number> thefts a year. That’s a lot! How could you trust a Chinese person?”

“But don’t non-Chinese rob people too?”

“Maybe, but if so, that doesn’t make the Chinese any less guilty, does it? First we should deal with the Chinese criminal problem, and then if we’re successful, maybe we can move on to other types of theft.”

“Are the Chinese really the first group we should target in our anti-theft campaign?”

“Hey, quit trying to change the subject. Are you trying to deny the immorality of stealing people’s hard-earned property? Why, just go into Chinatown and walk around for a while, you’ll see a Chinese mugger soon enough, it’s right there in front of our eyes… “

Checking Out The Necronomicon

(Song by Raymond Arnold. To the tune of ‘Winter Wonderland’.)

Dusty tome, lies forgotten
Cover worn, pages rotten
A curious book
I’ll just take a look
Checking out the Necronomicon

Creepy words, pages turnin’
As your brain, is a churnin’
Insidious memes
Infecting your dreams
Haunted by the Necronomicon

In the graveyard I can make a promise.
That is not dead which eternal lies,
Soon I’ll reunite with brother Thomas!
(For with) strange aeons even death may die…

More I’ve read, the more I’m listenin’
In my head, voices whisperin’…
‘Tonight is the night’,
‘The stars are all right’,
Time to use the Necronomicon

In the graveyard we could raise an army,
Send it out to ravage all the land…
Sure, the thought may seem a bit alarming,
(But if you) read the book, I swear you’ll understand!

Later on, we’ll conspire
As we dream by the fire
To face unafraid
The plans that we made
Studying the Necronomicon

Robin Hanson on Cold Fusion

(Quotes taken from a Facebook thread)

Robin: “This report on a test of the E-Cat cold-fusion device is disturbing. Our theoretical prior is against it, but this does look competent, if not elegant or overwhelming. I think it has earned a replication attempt; someone should try to replicate it.”

Me: “This is fraud every six ways from Sunday.”

Matthew Bailey: “This “independent verification” is biased just like the previous one was. Rossi chose someone who was heavily invested in the E-Cat to “review” it (making them not so independent). Remember, about a year ago, he did this EXACT same thing.

Yes… It has earned a Replication Attempt. BUT…………………….. Rossi won’t tell anyone what is in the actual Reaction Vessel so that it CAN be replicated. Remember, this is the guy who has been convicted of Fraud about half a dozen times, nearly every one of them involving some form of “Free Energy.”

As someone else said: The priors seem to indicate that this too is fraud.”

Robin: “Matthew, I agree Rossi’s track record is a big cause for concern, and justifies a very careful test watching for that. But I don’t think one should need to see Reaction Vessel details to attempt replication.”

Me: “Robin: what are your odds that Rossi’s discovered anything which might plausibly be useful for fusion? Want to make a bet?”

Robin: “Alyssa, I’d give at least 2%, so 50-1 odds. That tempt you?”

Me: “Robin: You’re on. $10,000 against $200? Am at work now but can draw up formal terms tomorrow.”

Robin: “Alyssa, can you wait 25 years for settlement? It has already been 25 years since the first cold fusion results were claimed, so clearly this subject takes a very long time to sort out.”

Me: “The claim is not for cold fusion generally but for one specific device. Fleischmann’s results from 1989 were thoroughly debunked within a year or two (for that specific experiment).” [link for context]

Robin: “Conditional on this new device working as claimed, it seems not at all clear the ’89 results were debunked. I could give $200 to one of my sons, and make him responsible for paying you, so you could be assured the responsible party will probably be alive. It could be $200 then, or what $200 today becomes using a standard investment vehicle.”

Me: “What evidence would you accept, then, as it having been debunked? The 1989 results were given dozens of replication attempts, all of which failed.”

Robin: “Showing that experiments didn’t rise to a reasonable standard of proof to convince an audience is not at all the same as a showing that its claim was false. Just as a lack of a statistically significant estimate of a parameter is not proof that the parameter equals zero.”

Me: “Robin: Can you name one single case, in the history of the hard sciences (physics, chemistry, biology, etc.), where:

a) a specific experiment was conducted, and it produced results which were both implausible under contemporary science and commercially valuable;

b) at least a dozen other people tried the same experiment, with the exact same setup and materials and tools and equipment, and got different data which were in accordance with science as then understood and indicated no commercial value;

c) the results from a) were later shown to be real after all?”

Robin: “Alyssa, I don’t have time or stats to debate all possible experiment scenarios. Experiments are almost never exactly the same, that is why replication is hard. See Collins’ book Changing Order, for example.”

Me: “Collins spent a lot of time investigating the paranormal, and was apparently convinced by proponents that it was at least plausible. That certainly doesn’t mean he’s wrong about everything (Newton wasted years on theology), but as a source of information on the scientific method it makes him highly suspect.”

Robin: “Collins is an excellent sociologist of science.”

[context: Collins’ paper “The construction of the paranormal: Nothing unscientific is happening” sadly does not seem to be online. However, the book Robin recommends (Changing Order) is on Google Books in snippet form, and in it Collins says (for example) “It is particularly unfair to expose fragile new sciences to the rigours of critiques based on a canonical model which even established sciences cannot attain. The ‘Committee for the Scientific Investigation of the Claims of the Paranormal’ and other such vigilante organizations are suspect in this regard.”]


Get every new post delivered to your Inbox.

Join 63 other followers