In December 2024, fellow engineer Alice Rum and I organised an event titled “Pull Request Awards” in Miro. The overall style was a light-hearted take on “Academy Awards but for Code”. Below is an outline of the things we did.

Tasks

  • Do an open call for volunteers to help run the event
  • Make a blog post announcing the event and rules
  • Create a public form to receive submissions for good PRs, alongside a reasoning
  • Spread the blog post and form across all engineering channels
  • Contact some high level engineering leaders to get them to record “and the award goes to…” segments
  • Create a list of eligible repositories
  • Assign the task of reviewing PRs and filtering down into a shortlist
  • Label each shortlisted PR, then add a comment explaining the reasoning
  • Announce each shortlist on the engineering channels and congratulate authors on making it this far
  • Plan a ceremony event, invite all of engineering
  • Figure out some interesting facts per repository/category to introduce it during the event
  • Decide on prizes
  • Get the prizes built/ordered
  • Announce the winners in text after the event for those who couldn’t attend
  • Contact winners to arrange the handoff
  • Plan a retrospective with all the volunteers to find areas for improvement

The following things happened for us when people learned about our project:

  • We were contacted by engineering managers with 3d-printers, who helped build the awards
  • The engineering leadership advertised our project and ceremony in all-hands
  • Some folks advertised our project further and submitted PRs for our consideration

We wanted to also have the following, but didn’t manage to:

  • Public voting
  • A committee per award making the decision

Awards

For the prizes, we decided to award to each winner a 3d-printed contribution graph for the year 2024, which looks kinda like this:

If the video above doesn’t work here’s another link

Links:

Selection Process

We had tens of thousands of PRs to get through, in about 32 working hours. We accepted the following facts:

  • We are biased, imperfect judges, but we were the only ones available in that timeframe
  • We’d openly discuss what criteria we used, while accepting the subjectivity of their interpretation
  • We would exclude all our volunteer’s PRs to avoid conflicts of interest
  • We wouldn’t be able to thoroughly review or even understand every last PR
  • We would be able to assess some repositories/categories better than others
  • We wouldn’t use any kind of AI/LLM assistance - all biases would start and end with the human volunteers

In the end, it was more important to us to run this event in a “good enough” way, than to fail to run it in a “perfect” way.

See also: Thoughts from reading 1000 iac Pull Requests in one day