Our road has been rough, but the worst is over - Kemi Adeosun reveals