What is so bad about Autumn Mr. Shakespeare?

In "That time of year", the speaker has reached a new place in his life. He's reached Autumn, and is using that as a metaphor for bad things in his life. How Autumn brings back horrible memories ( I would have bad memories if I had hair like you Mr. Shakespeare). how Autumn also brings a death of his joy (line 8) and youth. Perhaps he should go walk among the vibrant leaves to see Autumn for it's true colors. Sure with fall comes winter and death of nature, but it only ends in winter. Autumn is not a 2nd death dear sir. It's a complex sadness. Like he's almost resentful that it has to be this way, but doesn't wish to change it.