Generally speaking, going to the library to read books does not require any formalities and fees. Unless you check, you can get a card at the museum with your ID card and a deposit of 100 yuan. If you don't want to do it, the deposit can be refunded to you.

I remember that in Quyang Library (next to Carrefour in Quyang), you can definitely read without a card. As for other libraries, you can go in and ask the librarian.