The most DIFFICULT book I've ever read was El Ingenioso Hidalgo don Quijote de la Mancha (more popularly known as Dox Quixote). Imagine reading a Shakespeare play when your command of English is moderate at best. That's what reading that book was like for me, as my Spanish is good but not perfect, and this was written hundreds of years ago. The idioms, usage, and general flow of the book were almost impossible. I had to keep a dictionary next to me, and I swear I read it as much as the book itself. I stuck with it because a teacher I greatly admired told me that my education was incomplete unless I had read this, and to truly appreciate the genius I needed to read it in the original Spanish. Turns out he was wrong, the translations I've read were much more worthwhile.
A Clockwork Orange is another difficult read, simply becuase it's written in the slang of the time. There are some sentences that you simply never really understand, even with the glossary at the end of the book. That one's easy to get through, however, because it's a good, fast moving story.
If you mean the book that's the toughest to get through I'd say Crime and Punishment. Talk about tedious. Nothing ever happens! Okay, Raskalnikov feels guilty, I get it! Feeling guilty sucks! It's not that hard for a Catholic to empathize with! I managed to stick through that book because the literature course I was taking had a weekly quiz on what was happening in the book. But I never took a course from that professor again.
Edit-
Looks like one person's pain is another's pleasure! I know people that enjoyed Thomas Hardy, and one person right here enjoyed Crime and Punishment, and that just boggles my mind. But I read the Fountainhead in less than a week and was captivated the entire time. Same with Atlas Shrugged. There's no accounting for taste, I guess.