Closed-form formulas are overrated

tl;dr A closed-form formula is a means of expressing a variable in terms of functions that we have got names for. The set of functions that we have got names for is a pure accident of human history. Thus having a closed-form formula for an object of study is also merely an accident of human history and doesn’t say anything fundamental about the object.

The essence of scientific investigation

Scientists like understanding things. A good test of understanding is the ability to predict. For example, we can claim that we have understood gravity because we can predict with amazing accuracy where the moon, for example, is going to be at any given time in the future.

In the next few paragraphs, I am going to want to make extremely general claims and that will require me to talk about some very abstract concepts. So let me talk about those abstract concepts first.

Most of the things that we have tried to understand in the history of scientific investigation can be thought of as an abstract number crunching device. The moon, for example, is something that we see in the sky at a particular angle at a given time. So we can think of the moon as a device that takes time as input and turns it into a particular position in the sky. We can denote time by a number t and the position by two numbers x and y. Thus moon converts t into x and y.

The number crunching that the moon does is not arbitrary. If one observes the moon for a while, it is easy to start seeing some patterns. Some obvious patterns are immediately visible. For example, there is a certain continuity in the way it moves, i.e., its position in the sky does not change too much in a short period of time. There are other very non-obvious patterns too. These patterns, in fact, required centuries of scientific investigation to uncover.

When we are trying to understand the moon, we are trying to understand this pattern. More precisely, we want to write down a set of rules that perform the same number crunching as the moon does, i.e., if we start with a t and apply those rules on t one by one, we get an x and a y whose values match exactly with the values that the moon’s number crunching would have given us. Now, I am not claiming that understanding the relationship between x, y and t tells us everything about the moon. Of course, it doesn’t say anything about whether there is oil on the moon’s surface. But let me just use “understanding the moon” as a metaphor in the rest of the article for understanding this specific aspect of the moon’s motion.

This is not specific to the moon, by the way. Consider some other subject of investigation. For example, the flu virus. One crude way of modelling the flu virus as a number crunching device is to say that it converts time into the expected number of people infected. That’s a very high level picture and we can make the model more informative by adding some more parameters to the input. For example, say, the average temperature that year, the humidity etc. The output can also be modified. We can, for example, make the output a vector of probabilities, where probability number i tells us how likely is it that person number i will get infected by the flu virus. There could be many ways of understanding the flu virus, but once we have asked one specific question about it, we have essentially modelled it as a number crunching device that converts some set of numbers into another set of numbers.

The main challenge of scientific investigation is that we do not usually have access to the inner workings of the number crunching device under investigation. In this sense, it is a black box. We only get to see the numbers that go in and the numbers that come out. Just by observing a large number of these input-output pairs, we take up the task of figuring out what’s going on inside the black box. We know that we have figured it out if we can replicate it, i.e., once we have constructed a set of our own rules that have the same behavior as the black box.

Things get interesting once we try to understand what kinds of rules we are allowed to write. For example, do we really have to write those rules? Is it fine if I hire a person who knows the rules and when given a time t, always outputs the correct x and y, the x and y that the moon itself would have churned out? Is it still fine if the person I have hired only understands the rules and can replicate the correct input-output behavior but can not explain the rules to me? If that is fine, then how about creating a machine, instead of hiring a person, that manifests the same input-output behavior in some way? For example, may be, the machine is simply a screen with a pointer and a dial so that when you set a specific t on the dial, the pointer moves to the correct x and y coordinates on the screen? Is that fine? Or may be, the machine is just a giant rock revolving around a bigger rock so that when a person standing on the bigger rock looks up at time t, he can see the smaller rock exactly at coordinates described by the corresponding numbers x and y?

I don’t know which of the scenarios above should be considered a “valid” understanding of the moon and which ones should not. But it seems clear that there can be several different ways of “writing” the set of rules. The primitive way of doing this was to write the set of rules as a closed-form formula.

What is a closed-form formula?

x = 2t + 1 is a closed-form formula. So is x = sin(t) + cos(t).

Until high school, I was under the impression that in order to understand the moon, one was required to present some such closed-form formula, i.e., express both x and y as functions of t. But that’s an unnatural constraint.

For example, what if x was a slightly weirder function? Say, x was 2t+1 for t < 1000 and sin(t) + cos(t) for t > 1000? May be we would still accept that, mainly because there exists a conventional way of writing such piecewise functions in math. But what if x was something even more weird? For example, say x was equal to the smallest prime factor of t? Or may be x was something that just cannot be written in one sentence? May be x was just given by a sequence of instructions based on the value of t, so that if you started with a value of t and followed those instructions one by one, you would end up with the value of x?

The punchline of this argument is that sin(t) (or even 2t+1, for that matter) is already such a set of instructions. Just because human beings, at some point, decided to give it a name doesn’t mean it is more fundamental than any other set of instructions for converting t into x. Thus in the process of understanding the moon, one should not worry about coming up with a closed-form formula.

At the same time, it is clear that some ways of writing the rules are better than others. For example, having a moon’s life-size replica revolve around the earth’s life-size replica as your set of rules is a bit inconvenient from the point of view of making predictions.

What, then, is the “correct” way of writing the rules? I want to claim that the answer to this question can be found by understanding computation and, specifically, the area of computational complexity. But I will not make this article any longer.

Advertisements

4 thoughts on “Closed-form formulas are overrated

  1. Maybe you are being too harsh with those poor closed-form formulas. True, they’re just a label, i.e., sin(t). On the other hand, we usually label things that we know how to manipulate, that is, differentiate, integrate, etc.

    • vinayakpathak says:

      May be I haven’t made the main message very clear. I am definitely not against labeling things. What I’m saying is that asking, “So did you get a closed-form formula for it?” is not meaningful. This question should not be a part of scientific inquiry. A more reasonable question would be something like, “So did you get a polynomial time algorithm for it?” I thought it was worth writing an article about it because I have seen many people even today worrying about getting closed-form formula for things.

      • Or maybe I have not made my comment clear.
        What I meant is that in the first case I can answer: “Yes, and *therefore* now I can differentiate it and do other awesome things!!!”.
        If you have a polynomial time algorithm that, say, predicts the position of an object in future time, you are still not able to answer (at least not in any immediate way) questions like: “when will it reach the minimum?”

  2. vinayakpathak says:

    It seems WordPress does not allow threaded comments beyond depth 3, so I am making this a depth 1 comment.

    Your point about being able to differentiate and integrate things is a good point, but I think the requirement of having a closed-form formula is still ad-hoc and doesn’t even guarantee that you will be able to do useful things such as finding the minimum quickly. For example, when you say, “Yes, and therefore I can differentiate,” I am going to say, “Ok, find me the minimum, then,” and you will say, “Ah crap, the derivative turns out to be a function whose root seems to be difficult to calculate and I can’t even find a numerical method that provides any guarantees on convergence.” In other words, just because the derivative can be written in terms of functions that happen to have names, there is no guarantee that you will be able to do useful things with it. At the same time, there exist functions that do not have a closed-form formula, but their derivatives do. For example, take any function whose integral does not have a closed-form formula. So having a closed-form formula is neither necessary nor sufficient for being able to do cool things with the function. All it shows is that there exists this arbitrary set of properties that we have found some functions to satisfy and now it seems this new function also satisfies that fairly arbitrary set of properties.

    Complexity theory, however, can provide some insight here too, as long as you have figured out what you want to do with the function. The first thing that you could want to do with F(x) is to compute it for different values of x. In this case, it’s clear that wanting F(x) to be a poly-time computable function is more natural than wanting it to have a closed-form formula. If you want to be able to find the minimum, or may be do some more cool things with it, you may want to be able to solve the following stronger problem in polynomial time: “Does there exist an x such that F(x) < k?" Thus even in this case, it seems that instead of asking "Does it have a closed-form formula?" you should be asking "Does F(x) belong in a complexity class such that answering questions such as the one mentioned above can be done in polynomial time?" This makes me wonder: does this class have a name?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s