I understand what you’re saying because I was thinking a similar thing. If you only have a finite number of years then your premature death deprives you of what few years you had. Say you day at 40 when you would have lived to 80. That’s half your life!
But then I thought: if you were never going to die, it’s a completely different kettle of fish! If you try to measure it on the same scale your math breaks. You die at 40 when you would have lived forever! You missed out on an infinite number of years which adds up to a lot more than half your life.
Furthermore, the premature death of an immortal is arguably more tragic because they would never have died otherwise. For a mortal death is inevitable, we have to accept it. But for an immortal it is fundamentally something that should not be!
In the end the argument is somewhat academic though, as we are rarely in the company of immortals.