I’ve never viewed software as a controversial idea any more than I’ve considered wheels to be controversial. Wheels are simple things. They have an axle and a rim, and are useful for many things. Software is a little more complicated but is little more than a list of instructions. Yet it doesn’t take much searching to discover engineers who don’t view software the same way that software developers view it. Indeed, there are many mechanical and electrical engineers who will freely claim that software should not be left in the hands of software developers. They argue that the field of software engineering does not properly engineer software. If we ever want truly good software, we should take it away from the people who claim the title of “software engineers” and put it in the hands of people who are properly trained as engineers, such as electrical or mechanical or even civil engineers.
This argument, which I have come to call the “Engineer’s Complaint,” comes in part from the competition of engineering disciplines. Members of every engineering field can feel that their specialty should have dominance over the others. Electrical engineers point to their creation of power and communications grids and claim that they have been the innovative force of the last 70 years. Mechanical engineers claim that electrical engineers could not create or utilize power without mechanical devices. Civil engineers note that they are the oldest branch of the field and have made their mark on the actual landscape of the earth. Members of all three fields view software engineering as an upstart and have made it a target for their criticism.
Also, the origins of the Engineer’s Complaint can be traced to how software has infiltrated and altered all forms of engineering. Software has replaced the analog control circuits of electrical engineering with programs. It has destroyed the field of linkage design in mechanical engineering and encouraged the designers in that field to make devices out of independent motors and digital controllers. By forcing design programs into the tasks of creating buildings, bridges, and roads, it has taken power away from the civil engineer and put it into the hands of architects and designers.
However, one cause of the Engineer’s Complaint might be the fact that software is not the same kind of artifact that is found in the other engineering disciplines. Software is best considered as a part of work. Traditionally, work is defined as the combination of capital and labor. That definition, however, misses a key point. It fails to account for the knowledge that work requires. If we are creating work, we need to know how to do something, we need to train workers to use tools, we need to organize workers so that they work well together. Prior to the Industrial Revolution, this knowledge was generally held by the workers. They knew how to make things. They knew how to use tools and would teach these skills to their children. They organized themselves and decided who was to do what task.
Before the Industrial Revolution, workers tended to hold knowledge in an informal way. They described processes in stories or poems. They used work-songs to coordinate labor. This early literature does have some examples of more formal ways of planning work. We can find early plans for buildings and lists of instructions for creating weapons. However, few people started thinking about work knowledge in a systematic way, at least in Europe, before about 1755.
With the start of the Industrial Revolution, people started preparing lists of instructions for industry in about 1800. One of the more famous examples of this work was done at the Springfield Armory in the US. This factory manufactured weapons for the US Army. In about 1798, it began to develop a process to create identical versions of guns—guns with replaceable parts. They wanted to be able to take a part from one gun, exchange it with the same part in another, and have both weapons work. This was not possible with the existing manufacturing methods.
When people started thinking about calculation, they naturally turned to the kinds of instructions that had been produced for factories. In 1831, Charles Babbage, for example, described his first computing machine in a book called On the Economy of Machines and Manufactures. In the 1940s, the pioneers of electronic computing also drew from the manufacturing literature to describe the work of programming. Indeed, they drew the word “programming” from industrial engineering. To industrial engineers, a program was the plan and instructions for a day of work.
The Industrial Revolution not only created a formal way of recording the knowledge we needed to run a plant, it also gave that knowledge a new owner. These formal instructions for creating objects were owned not by the individual workers but by the factory owners. These owners decided what should be made and how. In much the same way, we now have people who own programs, decide which programs they want to use, and get their computers to run those programs.
At this point, I could stop this article. I could say that programs derived from factory instructions, that software engineering is similar to industrial engineering, and that we have learned a little history. However, this is not a history column. This is a column about how we work with modern technology. Manufacturing instructions are not sufficient to run a factory. We need other forms of knowledge, including how to operate tools and how to organize workers. Often, this kind of knowledge is held informally and is transmitted by the worker. Hence, the worker still has some say in how factories operate. Indeed, since the 1980s, we have seen more and more plants involve workers in the daily management of factories and have looked for ways to connect the knowledge of workers with the knowledge of industrial engineers—informal knowledge with systematic knowledge.
As an artifact, software is highly systematic. It has to be organized in a certain way or it won’t work properly. It has to be used as designed or it will give erroneous answers. Yet, if we look beyond the surface, we find that software needs at least three different kinds of knowledge. Each kind of knowledge is unique. Most important, each kind is developed by a different group of people.
The first kind of knowledge includes algorithms—the formal, organized way of doing things. This kind of knowledge is created by researchers in universities and senior engineers in companies. It is transmitted through classes and books and papers. When we try to identify who owns it, we generally look to company owners.
The second kind of knowledge might be described as the knowledge of languages or of programming tools. We have been able to create a great number of tools that help us organize, structure, debug, and test software. Although these tools are often created by companies, they often represent knowledge that is passed informally from worker to worker. After our first effort at programming, we generally learn programming languages on our own. We learn how to debug by working with senior programmers. We learn how to organize a system by listening to experienced developers.
The final kind of knowledge concerns how we organize people to develop software. Increasingly, this knowledge is moving from company owners to the programmers themselves. This movement follows a similar movement in manufacturing that began in the 1980s with the growing interest in quality control. It started to become popular in the US following the 1989 publication of the book The Machine That Changed the World. It moved into software with such ideas as Xtreme programming, Scrum development, Agile development, and open source.
These three different kinds of knowledge help explain some of the Engineer’s Complaint and some of the challenges faced by software development. Of the three groups of knowledge, only one—algorithmic knowledge—is formally in the hands of senior engineers and researchers. Tool knowledge is in the hands of workers, and organizational knowledge belongs to a combination of workers and management. This split is also found in traditional engineering, but those fields have one substantial difference. In those fields, the engineers do not work alongside their laborers. Engineers do not share desks with shop stewards, and shop stewards stand above common workers.
In a software team, engineers share space with programmers and code clerks. Indeed, by participating in a software project, code clerks can become programmers, and programmers can rise to the level of software engineers. In such an environment, ownership of knowledge can become muddled. Programmers know something about algorithms. Engineers understand development tools. Coding clerks understand how the project is organized.
At base, the Engineer’s Complaint is not a criticism of software engineering. It is an observation about the fluid nature of software development and how different kinds of knowledge are claimed both by people who are formally trained engineers and by those who are not.
About David Alan Grier
David Alan Grier is a writer and scholar on computing technologies and was President of the IEEE Computer Society in 2013. He writes for Computer magazine. You can find videos of his writings at video.dagrier.net. He has served as editor in chief of IEEE Annals of the History of Computing, as chair of the Magazine Operations Committee and as an editorial board member of Computer. Grier formerly wrote the monthly column “The Known World.” He is an associate professor of science and technology policy at George Washington University in Washington, DC, with a particular interest in policy regarding digital technology and professional societies. He can be reached at grier@computer.org.