Actually the declaration of love, Lois rejection is the main problem. Lois asks him to go to the Kerth Awards banquet with her, and does not in any way say "it's not a date" like she did back in the pilot. Lois has not withdrawn, Clark has just misread her initial shock at his declaration of love.
Plus without the pain of the "I would love you even if you were an ordinary man"/"under the cercumstances I can not believe that" there is just not as much hurt and fear on Clark's part.
