How an Analysis Becomes Official

In my last post I alluded to the fact that it is no trivial matter to take an analysis from a bunch of plots that I have made to an official statement of scientific fact put out by the collaboration. The road to approval is so long because in high energy physics, all members of the collaboration publish a result, not individuals. In fact, from outside of the collaboration you have no way of knowing who performed which analysis by reading the documentation. Most other scientists find this arrangement very strange since generally papers are very explicit about the level of involvement each name on the paper had in its production. High energy physicists take the egalitarian view that since our experiments are so large and complex, they cannot function without all of the members, and that the contribution of the person running the pixel detector is no less important that the person who is searching for the Higgs. Therefore, every name (all 3000 of them!) gets put on all papers in alphabetical order. The approval process is designed so that all collaborators can have their say about each analysis, which generally means that any result coming out of a HEP collaboration has gone through much stricter peer review than the average journal article. It also results in a lot of headaches for the person trying to get the analysis approved.


To explain the process, I'm going to employ School House Rock's awesome and educational, "How a a bill becomes a law". Rewatch it if you haven't seen it in a while. It's pretty great.



The Bill in question is my analysis and I play the part of the congresman who proposes it. My advisor is represented by the people who ask for the law, because she came up with the analysis concept. The drafting of the law, i.e. the dirty work of the analysis, kept me at CERN until obscene hours of the night for days on end in May and June. The committee fits best into the role of the Analysis Reviewers and Editorial Board, to which each analysis is assigned. These people look at the analysis in detail and typically ask you good question which drive you back to your computer to investigate, and in the end produce a better paper.

Once the board and reviewers deem the analysis mature enough, the paper (which, in this case, is not destined for a journal, but will be made public) is sent out to the collaboration and they are given a week to comment on it. This stage is equivalent to the Bill going to the House and Senate (imagine we have a single legislative body). At the end of the comment period the analyzer gives a talk about it, which would be equivalent to a floor debate. This part of the process involved me flying to Copenhagen to give a talk at our Collaboration meeting to a room of about 300 people. It was a little intimidating, but also fun because I got to use a nifty nearly invisible microphone that went over my ear and an enormous screen (12 ft tall, maybe?). My talk went smoothly, with no interjections or grumbling when the Physics convenor spoke the magic words, "Does anyone object to this continuing to the next stage?". Some of the analyses were contentious and forced to revise before coming up for a approval again. In most cases I agreed that the analysis wasn't quite ready and felt that the approval procedure had done its job.

Sadly, once ATLAS approves my analysis it does not go to Obama's desk. It instead goes to two of the three heads of management of the experiment: the Publication Committee chair, the Physics convenor and the Spokesperson. The first two read my analysis, and this is where it stalled a bit. I won't go into the details, but it was very confusing and frustrating. However, once those two "sign-off" on the analysis it is ready to go to the public and the long, arduous journey is over! The analysis now has the ATLAS stamp of approval.

I'm going to two conferences this August to present my work (and the work of others). Its quite exciting to know that for almost the first time I'll be showing plots that were made by me!