The word "le" in "I went to the library" is the modal particle "le", which means that things have changed, that "narrative" is dynamic and describes the whole process of "go".

I went to the library.

"Le" in I Go to the Library is a tense auxiliary word "le", which means the completion of the action, the "referred to" is static and refers to an objective event.