In fact, Harmon was very close to walking away from the show a mere four seasons into its run. According to TV Guide's ...