I haven't watched the series (I just got the first disk of Game of Thrones from Netflix and will be watching it soon), but I've read all of the books - some of them several times.
Books 1-3 are without equal, in my opinion. 4 and 5 slow down things considerably.
There is a Chapter in Book 3 (Anyone who has read the book probably knows what I am referring to) that just knocks your socks off. I literally had to put the book down, walk around outside, and then come back in and re-read the chapter to make sure I had read it correctly.