Law Profs. with PhDs & Data Collection

A guest post by Adam Chilton:

UCLA law professor Lynn LoPucki recently posted an article examining 120 empirical papers that were published in 15 leading law reviews and the Journal of Empirical Legal Studies. LoPucki coded these articles to see who was writing them, and what kind of data they used. The paper’s primary finding is that articles in the dataset written by law professors without a PhD were more likely than those with a PhD to code original data instead of using existing dataset sources. From this fact, LoPucki concludes that there is a “reduction in coding resulting from the hiring of more J.D.-Ph.D.s” that is undesirable for the legal academy. 

I have two thoughts about this interesting finding. First, I don’t think that we can infer from LoPucki’s paper that law professors with PhDs actually code less data. The reason is that LoPucki’s units of analysis are papers and not professors. It could be the case that law professors with PhDs code new data for a lower percent of their empirical articles, but still collect new data for a much higher absolute number of papers. LoPucki’s claim could be true–and I don’t have data to suggest otherwise–but I don’t think that it’s demonstrated in his current paper.

Second, I’m not persuaded by the claim that the measure of a good empirical legal studies paper is whether it codes original data. To be fair, LoPucki’s argument is not that coding original data is necessarily important for all disciplines. Instead, his argument is that  coding original data is important for legal scholarship because it typically requires direct engagement with legal materials like cases or statutes.

I think that for all empirical papers–whether written by law professors or economics professors–the question of whether to collect original data or use existing data sources should depend entirely on what data are already available. For some topics it would be a waste of resources to reinvent the wheel, and for other topics collecting new data may be absolutely necessary. I think that an empirical legal paper is valuable if it credibly brings new evidence to an important topic, not if the dataset was coded specifically for the paper.