Alon points out that, once you start describing four dimensions in terms of four co-ordinates, the justification for calling your extra co-ordinate an extra dimension follows largely from the fact that you get some sort of four-dimensional geometry thereby. Most geometrical notions in two and three dimensions generalise in obvious ways to the case with four or more dimensions.
Because of what I am about to say, I'm going to note here that "linear transformations" include particularly nice geometrical notions like rotations and reflections as well as things which bend, stretch and squash shapes in well-defined ways.
Length is defined using Pythagoras’s theorem, and angle is defined using inner products. Linear transformations are defined by the more easily generalized property that T(v1 + v2) = T(v1) + T(v2) and T(kv) = kT(v), which coincides with the more concrete definition in two dimensions.
Now, The Question.
What's with relativity and that whole "time is the fourth dimension" thing?
Well, there's one obvious answer. If a particle is situated in a particular place at a particular time, you can write the particle's position in space and time using four co-ordinates (x, y, z, t). Four co-ordinates, four dimensions - but that's not all there is to it. After all, you could have written the particle's position using four co-ordinates in that fashion well before the special theory of relativity had ever been dreamed of. What is it about relativity that makes this viewpoint suddenly relevant?
The word "relativity" refers to the notion that the way we define our co-ordinates is relative. For example, I might say that the point (0, 0, 0) in the usual three-dimensional space refers to the lower left hand corner of my computer screen. You might, instead, choose to say that the point (0, 0, 0) refers to the lower left hand corner of your computer screen. It shouldn't matter whether we number our points with my system or your system - the geometry (or physics) is the same. Where we put the zero point is just a matter of how you describe it. Similarly, you might say that the x axis points directly in front of you - or you might say that the x axis points directly to your right. It doesn't matter which way you define it (ignore the fact that the rotation of the Earth is going to make your co-ordinate system rotate - in other circumstances we might want to avoid that, but this is just an example).
There are well-defined mathematical ways of getting from one co-ordinate system to another. Not coincidentally, they often involve linear transformations - for example, going from the system where the x axis points straight in front of you to the system where it points directly to your right involves a rotation.
Before special relativity, time was basically exempt from these sorts of co-ordinate changes. Oh, you could define the moment at which t = 0 in different ways, but that was about it. Rotating some of your space so that it points in the time direction would just be stupid - or so you would think. In special relativity, though, that is almost exactly what you do. Depending on how fast you are travelling, and in which direction, you get a bit of space in your time direction. The linear transformation involved isn't a rotation, though, so time is not just the same as all the other co-ordinates. Thus, special relativity does not allow you to turn your time all the way around so that it goes backwards, in the same way that you can rotate your x axis so that it points in the opposite direction. However, the similarity to co-ordinate changes in three-dimensional geometry is visible enough that the idea of time as the fourth dimension as a consequence of relativity has stuck.