August 7, 2019
(reposted from Better Scientific Software)
I’ve been participating in open source software projects since around 1994; and when asked what I’ve learned, I always say,
“It’s all about people.” So while I could be writing about all the technical things that are going on in my scientific software projects,
let me instead write about people.
Emacs’s CC mode
My first involvement with open source was when I was a freshman at the University of Stuttgart, in Germany, in 1994. I got an account on
one of the computer pools. Most students didn’t back then, and probably also didn’t want to deal with the IBM version of Unix (AIX)
that the machines were running, but I enjoyed the challenge and did some software development for fun on these machines. I used emacs
(that time’s Integrated Development Environment) for C++ programming and became annoyed that the code indenter got lots of things wrong.
Now, C++ is a difficult language to parse, but the regular expressions used to identify what is what just weren’t very good either. So I
dug into the source code of CC Mode, the emacs mode for C++. Emacs packages are programmed in Lisp, which I didn’t know, but I figured out
improvements to the regular expressions and sent them to the person who was listed as the author of the file.
I don’t recall the name of that person (I think it was Barry Warszaw), where he worked and lived at the time, how old he was, what he
is doing now, or basically anything else of our interactions. But I do know that here I was, a young student with essentially no knowledge
of how things worked, and here was someone taking the time to mentor me through his perspective of how things are best done, going
through iterations of the code, and eventually putting my contributions into CC Mode. It was an awesome introduction to how I think that
open source software should work: a patient mentoring relationship, with an eye toward the fact that to bind contributors to a project,
they need to feel like their contributions are appreciated.
Since that time, I’ve worked with quite a number of other open source projects. It turns out that the people didn’t all have the
same style as that maintainer of CC Mode. For example, emacs as a whole has a leadership structure in which a single person can
dictate what is supposed to happen (and who has done so in other projects they lead). I also
participated in the GCC project for many years, which during that time dropped the ball on many patches submitted by newbies for
small technical reasons – such as a malformatted changelog entry where the maintainer would simply say, “You’ve got to fix the
changelog entry” but would not take the time to mentor people about how such an entry should look, why, and what to do. This
leadership style meant that lots of actual and potential contributors walked away. Most did so quietly; some others first forked GCC
to found the EGCS project (which later supplanted GCC again when nobody was left in GCC); and yet others started the LLVM project.
Plenty of scientific software projects work in similar ways: The people who started a project a long time ago either don’t care
about new contributors joining the project or just don’t know how to make it work.
My point is that there are a lot of packages out there and every one seems to have its own style of leadership. Much of this style
probably comes down to the personalities of key individuals or the group as a whole, but surely also how much thought the maintainers
of a software put into what they want their style to be.
My own path: the deal.II finite element library
In 1997, I was working on my master’s thesis in Germany and was too stubborn to use the finite element package (called DEAL) that was
used in the lab at the time. So I wrote my own and called it deal.II. I stayed on for a Ph.D. in that lab and continued with my
software. Two others then joined in the effort, and in 2000 we put everything on a website – not because we really knew what we were
doing but because we could: that’s what one did at the time, with the World Wide Web as we know it today just finding its shape at the
time. We had no expectations for the whole endeavor; but to our surprise, people started using the package. And they started asking
questions. And then they also started contributing functionality.
All of this led to some sort of reckoning: If we spend time answering questions, we won’t have time to write code ourselves; if we
start taking other people’s contributions, then we have to accept their coding styles and quality standards; or if we walk them through
how we want things done, then again we won’t have time to write code ourselves. With now twenty years of thinking about these issues,
I can say that these are not false dichotomies. They’re real issues, and one needs to make a choice: How much do I value other people
using or contributing to the project (with their style), or how much do I think “I can do this better!” and value my control over the
project? What is a fallacy is to think that the project will see less progress if I spend time on things other than writing software:
That’s because if I mentor half a dozen people on how to write good software, they might actually produce more functionality than I
could myself; and if I do a particularly good job, they will eventually be good enough to review my code as well - and make me write
My own approach to software has been to follow this latter path. I like working with others and mentoring newer members of our community.
And this has paid off. What started as a project for myself is now one that is used around the world, has been the basis for more than
1,200 publications, and has 11 co-equal principal developers.
And substantially more than 200 people have contributed over the years - including 50-60
contributors to each of the past few releases. deal.II is now a real community project. Decisions are made among all principal
developers, all of whom are responsible for reviewing patches and helping the community, and whose patches are reviewed to the
same standard that we hold everyone else to. I may have founded the project, but I don’t get a pass on writing tests or documentation
or just cleaning up a piece of code I want to commit: I have to have my patches approved by another maintainer, just like everyone else.
All of this didn’t come by accident. It was a choice made every day by those running the project. We want to be a project with a large
community based on principles of equality, openness, and merit. We all think that we serve the project best by supporting our
community and mentoring the next generation of contributors into leadership positions where they work alongside us and, as we move
along in our careers and have less time for writing software, relieve some of the workload from us. It turns out that if you take
the time to teach people and then give them a piece of responsibility of their own, they step up and want to contribute to the
greater project. Given how widely used deal.II is, that seems to have worked well for the project.
But, it also seems to have been good for most of us who run the project. Most of the current principal developers were recruited
into this role while they were graduate students; they later became productive postdocs, and 8 of the 11 now have permanent positions
in a variety of places. All of us share the credit that comes with leading a well-regarded software project, opening doors for
publication opportunities and grant funding. Surely that is reflected also in the trajectory of our careers. For me personally,
giving up control of “my” project and inviting others has been nothing but good professionally: it has led to thousands of citations
of my papers, millions of grant dollars, opportunities to collaborate with people in many interesting fields who wanted to use deal.II,
and eventual promotion to full professor. It has also brought amazing talent through my lab, in the form of graduate students, postdocs,
and short- and long-term visitors - people with whom it has been truly fun to work. I don’t think this would have been as easy if I
had kept deal.II under closer control. Personally, it has also been a fantastic ride: I got to work closely with so many, often younger,
people whom I now call my friends.
What kind of community develops around a software package is not a coincidence but instead a consequence of the leadership style.
Where we are today as a project is a consequence of many hours of patiently reviewing patches and not just saying “This is wrong,
rejected!” but instead saying “I think you could do this better if you restructured this code in the following way” and then
patiently explaining how and why. This takes time, without a doubt. It also takes time to respond to questions on the mailing list,
explaining how to implement something in user code and then adding “I think that would also make an excellent addition to the library
itself. Would you like to give that a try? For this, you’d have to look at A, B, and C, and when you’re there, get back to us and we’ll
talk you through the next few steps to make this into a pull request!” Nine out of ten times you never get to see any code, but the
remainder of the time you do get a half-baked pull request out of it that requires some more work on your behalf; the second time
around, this person’s code is already better; and occasionally after feeling encouraged a number of times to contribute their work,
that person will become a future principal developer who will write the kind of sentence above in a few years when answering someone
on the mailing list.