The universe is 14 billion years old. I’m not sure how long it would take for the first truly habitable planets to form. You need to cycle through enough stars to form heavy elements and AFAIK you need the building blocks of carbon based matter so it might take 5 billion years just to get to the point of life being possible, let alone complex life, we’re talking just barely enough to support E. coli. Life on earth took 5 billion years to go from E. Coli to Elon Musk and being able to go to the moon and hopefully Mars. Being optimistic, I think you might have a window of opportunity from 10 billion years ago to a couple billion years in the future.

It might be that you need more than just a marginally habitable planet. We might be lucky because we have outer planets to stop asteroids from hitting the earth, we have a moon to create seasons, we aren’t tidally locked, and we have lots of liquid water. If you get all life on the planet destroyed by asteroids and comets, or the seasons are too short, or only the twilight zones between the super hot tidally locked zone and the frozen night zone— all of this might make it hard to start life and might make colonization difficult.