No, it doesn't simply translate to andare; the idea of going to the library is implicit, but the sentence conveys the sense of having gone to the library, staying there for a while. However, the speaker is no longer in the library.
How would you say this in English? I think you can use to stay as well:
As soon as I left the gym, I stayed at the library for a while.
Of course, in order to stay at the library, you have to go there, but you don't need to say that explicitly.
The Italian sentence could also mean
As soon as I left the gym, I went to the library.
if the context makes clear that I went there for a short time, for instance in order to return a book I borrowed. However, for this case, I'd use Appena uscito dalla palestra, sono stato alla biblioteca, that is, I didn't really stay there long. Alternatively, sono passato dalla biblioteca.