Following a recent conversation with Richard Burian, I realized that both of us had assumed that a necessary if not sufficient condition for a new scientific variable to represent a genuinely novel concept is for it to allow a finer partitioning of possible states of the world than was previously possible. The idea is intuitively plausible. If I posit the existence of a new variable property of material bodies, then I can discriminate more possible states of the world. If, for instance, I posit the existence of an internal resistance for wires, then states that were previously indistinguishable when described in terms of current and voltage in a wire, are now potentially distinguishable on the basis of resistance. If I posit the existence of new kind of particle, then it seems I have recognized a greater variety of possible worlds. Corresponding to what were previously unique states of the world are now many possible states in which the new particles assume various positions, velocities, and so on. Recognizing a genuinely novel property (or class of properties) seems to entail admitting a finer-grained view of the world. But I'm no longer convinced that's the case.

Before I explain why I'm unconvinced, let me back up and explain the question at issue and where it came from. Since the heyday of logical positivism, the consensus in mainstream philosophy of science is that there does not exist a "logic of discovery", a method for mechanically generating significant scientific hypotheses. The only serious argument to this effect turns on the notion of conceptual novelty. The key premise is that no algorithmic process can introduce variables (or associated concepts) that were not already present in the presentation of the data or observations for which we are seeking an explanatory hypothesis. So, for instance, Hempel (1966, p14) claimed that one cannot "...provide a mechanical routine for constructing, on the basis of the given data, a hypothesis or theory stated in terms of some quite novel concepts, which are nowhere used in the description of the data themselves." Laudan echoed the sentiment a couple of decades later. He conceded that, while machines can certainly carry out algebra and curve-fitting, the essence of scientific discovery is the introduction of explanatory theories "...some of whose central concepts have no observable analogue" (Laudan, 1981, p186). Though he makes no explicit argument to this effect, he takes it as obvious that no effective procedure could introduce the sorts of concepts far removed from observation that are at the heart of modern theories.

How much of a stumbling block for automated discovery is the required sort of novelty? That's rather difficult to answer without a more substantive account of conceptual novelty. However, Hempel's syntactic characterization suggests a plausible necessary condition that Laudan would presumably endorse: a novel class of variables represents a novel concept just if the values of that variable are not functions of preexisting variables. Thus, if you already have concepts of mass and velocity, adding momentum or kinetic energy (both of which are defined as simple functions of mass and velocity) doesn't really introduce conceptual novelty. However, introducing a new variable m to represent a heretofore unacknowledged property of inertial mass into a theory involving only position and velocity is a sort of conceptual novelty.

Interestingly, introducing properties like inertial mass into theories previously lacking them is the sort of conceptual invention that automated discovery algorithms were capable of by the end of the decade in which Laudan wrote. I'm thinking specifically of third program in the BACON lineage developed by Herb Simon, Pat Langley, Gary Bradshaw, and Jan Zytow (1987). If we take the above condition as genuinely necessary for conceptual novelty, then BACON.3 is at least a counterexample to the claim that the condition cannot be met by an algorithm. It does in fact introduce an inertial mass when given data from experiments with springs, and it introduces a variable for resistance when examining currents in various circuits. Of course, you might just take this as an indication that the proposed condition for conceptual novelty is not sufficient. That's not an argument I want to take up this time.

What I do want to do is scrutinize the notion that positing a novel concept must somehow increase the number of possible worlds we recognize. In the sense of logical possibility, the new variables allow a finer partitioning of the world. Equivalently, they are not functions of existing variables. But if their introduction is well-motivated, it seems that enough of the additional logical possibilities are nomologically precluded that the number of ways the world might be remains the same. To see what I mean, it will help to consider in a little detail how BACON.3 introduces a variable. Consider the following table of data (adapted from figure 4.1 in (Langley, et al, 1987)):

BACON begins with the first three columns of data. Letters label distinct wires and batteries. The only variable measured is current, which is represented by a real number. Upon examining the first three rows of the table (corresponding to the same battery but different wires), BACON notes that current varies from wire to wire. The next step of the algorithm is, practically speaking, driven by the fact that BACON cannot relate non-numerical variables (e.g., the identifiers for distinct wires) to numerical variables. But we might give it a rather plausible methodological interpretation: if a variable changes from one circumstance to the next -- in this case, from one wire to the next -- it is reasonable to suppose that there exists a hidden, causally salient property which varies from wire to wire. Let's call that property conductance, and assume that it can be represented by a real number as well.

Following this maxim, BACON introduces a new variable whose values are shown in the third column. How were these values determined? As is clear from the table, BACON assigns a conductance equal to the values of the previously known variable, current. The authors don't discuss this procedure much, but it is a simple way to ensure that the new variable explains the old in the sense that there is a unique conductance value for each resulting current.

So far, it's not clear that the "new" variable is very informative or novel. But things get interesting when we get to the next three rows of the table. Since each wire was already assigned a value for conductance, BACON uses those values again, and notes that for battery B, the conductance and the current are proportional to one another. Unlike the case for battery A, however, the constant of proportionality is now 1.1444. Similarly, for the last three rows (corresponding to battery C), BACON finds that conductance and current are related by a slope of 1.6003. How to explain this variation? Posit a new variable! This time, we suppose there is a property of batteries (the voltage) that explains the variation, and we assign values identical to the slopes in question. If we note that conductance is the reciprocal of resistance, we can see that BACON has just 'discovered' Ohm's law of resistance: I = v / r. Of course, that relation is tautological if we consider only the data on hand. But treated as a generalization, it is quite powerful and most definitely falsifiable. We might, for instance, find that a new wire, D, has a conductance of c as determined using battery B. But when connected to battery A, the new wire could show a current not equal in value to c. This would violate Ohm's law.

There are two lessons to draw from the procedure described above. First, it sure seems like positing previously unconsidered intrinsic properties like conductance and voltage amount to producing novel theoretical concepts. Thus, it looks as though there is no real barrier to the algorithmic production of novelty, and the objections of Hempel, Laudan, and others are simply misguided. Second, the introduction of a novel concept does not entail recognizing a greater diversity of possible worlds, at least not in every sense. It is certainly the case that if we assume that a newly introduced variable can take on any value consistent with its representation (e.g., any real number), then as a matter of logical possibility, we have considered a finer partitioning of states of the world -- there are more ways the world might be for which we can provide mutually exclusive descriptions. But these logical possibilities are, as a rule, moot. The whole reason for introducing a novel variable is to explain previously unexplained variation. That means that a variable is likely to enter scientific consideration already bound up in a nomic relation with other variables. That law-like relationship precludes many logical possibilities. In fact, in cases like Ohm's law, those relationships will be such as to permit all only those states of the world we already recognized as possible in terms of known variables.

Note that I am not suggesting there is no way to introduce new variables that allow for a finer discrimination of states of the world. It seems obvious that such a thing is possible. My point is just that it is not necessary. In fact, it seems like in most cases of scientific relevance, the new variables do not provide finer discrimination.

To sum up, variables are introduced to do a job: they are supposed to represent whatever hidden properties vary from one circumstance to the next and so explain a previously unexplained variation. But that means that they are generally introduced along with law-like relations to other variables. These relations generally (or at least often) restrict the values in such a way that no finer partitioning of the states of the world is achieved.

Works cited

Hempel, Carl G. 1966. Philosophy of Natural Science. Prentice-Hall Foundations of Philosophy Series. Englewood Cliffs, N.J: Prentice-Hall.

Langley, Pat, Herbert A. Simon, Gary Bradshaw, and Jan M. Zytkow. 1987. Scientific Discovery: Computational Explorations of the Creative Processes. Cambridge, Mass: MIT Press.

Laudan, Larry. 1981. Science and Hypothesis. Dordrecht, Holland: D. Reidel Publishing Company.