Basic Principles of Mathematical Programming

Creating new mathematical algorithms can be fun and very rewarding. When all goes well, that is. Anyone who has taken a shot at mathematical programming knows how frustrating it is to debug programs crashing with obscure error messages or producing numerical garbage.

Here are a few tips for reducing the chance of introducing errors in mathematical code.

Basic Principle 0. Always, that is ALWAYS, write a comment before you write a line of code. Think about what you are about to do, formulate it in words, and write those words as a comment. Only then start writing the code module intended to implement your thought.

If you do not follow Basic Principle 0, you will not be able to figure out what your code is about after roughly two weeks of writing it. Then all that coding needs to be done again, wasting time.

Basic Principle 1. Start with the simplest possible situation, preferably one with known analytic solution. Make sure it works by testing it thoroughly; the best test is comparison between numerical and analytical values. In case there is no analytical solution, run the code with various resolution/accuracy parameters (such as N and N+1 and N+2). You should see convergence in the numerical results.

Basic Principle 2. Once you have something that solves a simplified sub-problem correctly, move towards the final complicated goal with as small verified steps as possible. In other words, modify the code the smallest amount possible that gives a testable result and then test it thoroughly.

Basic Principle 3. Doubt your code at all times. Test it repeatedly in different ways. Think of the code as a devious enemy trying to look OK most of the time but producing incorrect results when least expected.

With good luck, following Basic Principles 1, 2 and 3 will take to the goal of a correct algorithmic solution to a complicated problem. However, more often than not, you are in a situation where the code crashes or produces garbage and you have no idea why. In that case you should follow Basic Principle 4, which I actually learned from a Finnish athlete competing in orienteering at the national level.

Basic Principle 4. When you’re lost, go back to where you still knew where you were and go again from there. In less metaphorical terms: start over with a tested and trusted code for a simpler sub-problem.

I know that Basic Principle 4 sounds like a statement from the Master of the Obvious (a character in Dilbert comic strips). However, let me assure you that it is surprisingly difficult to remember and follow Basic Principle 4.

To enable Basic Principle 4, you should be able to easily go back to a previous phase in your project. This leads us to

Basic Principle 5. Keep old versions of your code.

For those of you using an automatic version control software, this is a no-brainer. However, I don’t use one and I do not know a single mathematician who does. So here is a practical advice for non-version-control-users. Develop the first generation of your code in a folder called Projectname/Matlab/v1/. Once the code is tested and trusted and it’s time to take the next step, create a copy of the above folder and name it Projectname/Matlab/v2/. Develop the next version there. In case you need to resort to Basic Principle 4, you just create a fresh copy of Projectname/Matlab/v1/ and go from there.

The last Basic Principle is related to mathematical modelling and problem solving involving real-world data.

Basic Principle 6. Start with simulated data and make sure that everything works in the simulated setting. Only after that move on to work with real data.

The rationale behind basic Principle 6 is this. Suppose you start directly working with real data. If all just works perfectly, you’re the luckiest scientist on Earth and all is fine. In the more probable scenario something does not work right away and the code needs to be debugged. How to find the problem?

Following Basic Principle 6 enables you to first find and root out bugs within the sealed world of computational models. Real data always poses surprising and difficult problems, and this way you can deal with them separately.