Not as nice this morning. First of all the temperature on the Brindlee mountain shul eBoard was almost 10 degF higher than yesterday, and no wind. And then the gym was crawling with educationalists, all seemingly oblivious to being rendered bankrupt by the revelation that learning styles are bunk. And then the podcast episodes were highly unmemorable today. And the only thing that resonated from the electromagnetic receivers was that the repubgnants could do better running Theodore Roosevelt (or Abraham Lincoln) for chief executive than any of the current gaggle of candidate wannabes.
So lacking any inspiration to yammer this morning I am going to turn to the literature. The latest issue (V13, #5) of Computing in Science and Engineering has a rather nice article entitled “Software Engineering for Scientists” by Kelly, Smith, and Meng. This journal is a joint IEEE and AIP publication and is available freebie on the internet one issue late. So wait a couple of weeks if you don;t subscribe and then you can read online.
The article tries to bridge the gap between scientist software and software engineering, which really isn’t about engineering but about management. The basic problem is that software is a relatively new thing, maybe a half-century od, and we still don;t have a good empirical basis for knowing how to manage it. So we have lots of brain flatulence and mind whack about how to do so. Happily with the speed of today the bad ideas get identifie pretty quickly and get weeded out of the small organizations quite quickly. The big organizations are something else entirely.
The difficulty is that the best way to build and maintain software is to have one programmer do it. (And probably nothing else.) Sadly, this approach is not deemed economically viable so the basic data are ignored and all sorts of wasteful and counter-productive approaches are tried to managing software.
These approaches, the better ones, at least, can be made to work poorly in the cases of software no one really cares about, like operating systems (Windows is the classic example here) and organization process software. It basically doesn’t work at all on software that anyone cares about like games and science software.
Since software engineering is all about management it is big on plans and schedules and instrumentality of operation. So what it ends up doing is managing the folks who work on software more than it manages the software itself. And it assumes that the software is some sort of all pervasive goop that has neither beginning nor end.
The problem with science software is that too often it has no such good behavior. Back when I was an undergraduate student, I would come across some need for a program, such as analyzing some laboratory data, write the program, use it, and then put the program away until I got distanced enough from the effort and discarded it. In graduate shul things were a bit more structured in that I had long term research plans and more stability. But the programs were not stable as I kept rewriting them to reflect the direction the research was going. After graduation most of the programs I have written have been for my research and they very much fitted into the paradigm of write and rewrite programs along the lines of the work but dispose of them once their usefulness has demonstrably (?) been established.
This is not how software engineering doctrine sees software. It is a product, an appliance, perhaps a foundry tool. Something that is not put to work until it is finished. After that it may be refined but there is still a distancing between development and production. My experience with science software is rather the opposite. Development and use are one with the research. Coding is not some independent activity, it is part of the mechanics of research just like tuning equipment or arguing with colleagues over interpretation.