I think that line sold this song for me.
I mean I love Sufjan Stevens, we all know this.
One day I wanted to have a Stevens marathon so I jumped onto the tube and zoned out. Then this song came on and I hadn't heard it before. It had the usual lovable qualities that his stuff normally does: his voice and that zoning ability that is a must in music for me... I was minding my own business, listening to him, feeling a bit sad like I wanted to and then he throws that line at me: "but you love someone else" and my heart hurts for him, "but you want to be with someone else" and I was SOLD. See the thing is I remember things differently. In my brain one word becomes a whole paragraph and that line became a tragic novel.
So when you're listening to it and thinking "why did this move her so much?" remember that to me a whole love affair is ending, one where he wooed her, loved her and then moved away for some unavoidable reason and she met someone else but he still loved her and they have this huge dramatic heart wrenching break up. This is what's playing as she is walking away from him.
At night. And it's raining. And he doesn't have an umbrella.
"If you once knew how I love you"