Hmmm, you make a good point. But it is the ending of a particular stream of time ... depending on your time travel model. If there is only one stream of time and it flips between different alternatives each time a decision is made, then it has changed (and thus the prior direction may in fact have ended).
If on the other hand, every decision spins off a new time stream and there are an infinity of them, then you've chosen your personal direction, but alternate you in other timestream alternatives could well be carrying on the orthodox (as the books were written to describe) timeline.
Fortunately, other than breaking their brains over metaphysics, it has little impact in play. And if the questions are disturbing... OMG! THE STORM IS UPON US!....