I have seen the first four films without ever reading one of the books; and admittedly doing so on purpose because of the movies.
While I did not love the first two movies, I did not hate them either. I did however love Cuaron's vision for the third, and thought the fourth film was well made and began to bring more darkness into the series, which I enjoyed.

I hope I like the fifth.

Haven't read any of the books. I thought "Phoenix" was a wonderful film, and I disagree with your assessment of the acting - the three of them have been getting better each film.

As for being confused, not really. There are things here and there that I don't know the whole backstory on, but for the most part the movie makes perfect sense.

Up till now I thought Cuaron's movie was the best iof the lot, but I'd have to revise that opinion and go for the 5th now! Quite a bit is missing indeed (perhaps 1/3 of the book?), but we've come to admit the impossibility of including the whole material! I think the story flies by and should be perfectly comprehensible to those who haven't read the book. I read it when it came out 2 years ago, and don't remember all the details, so wasn't too disturbed by the missing bits and pieces.

And yes the adult actors are all great, my only note of sorrow is we don't see enough of the regular ones! I'd like to see more David Thewlis and Maggie Smith! Thankfully we should get a good dose of ALan Rickman in the 6th movie when it comes out!

And the three leads have been improving with each film! :o)