Thich Nhat Hanh rightly said - It is not impermanence that makes us suffer. What makes us suffer is wanting things to be permanent when they are not.  

Listen to this episode on the philosophy of impermanence.