Surprise in Fiction

So it's National Novel Writing Month, and I thought about giving it go on a new project but instead decided to stick with working on my current project, with a goal of having a complete revised draft by the end of the year.  This work mostly involves writing "glue" scenes, that get things from point A to point B, and working on bolstering the plot and characters.

This got me thinking about my favorite plots and characters, and something that I've realized in the last few months about storytelling - it's terrific when you you're reading/watching a story and the characters do something surprising.  It's extremely annoying when your reading/watching something and the characters do something randomly out of character for the sake of surprise.  So what's the difference?   My two easiest examples of this come from tv show Castle.

The example of something delightfully surprising was in a early episode of Castle, when the two main characters Kate Beckett and Rick Castle end up on separate blind dates in the same restaurant.   You expect the average TV formula that they will get jealous of each others dates and end up sabotaging their own dates.  Instead, the writers did something kinda brilliant.  Instead of caring about each others dates, they both become obsessed with talking about the case they're on, and as they are talking to their dates come to the same conclusion about who the potential killer is.   They then abandon their dates to go and solve the mystery together.  The scene hits all the same emotional notes of the expected scene, and yet manages to reinforce the partnership and the crime drama aspect of the show.

My example of annoying out-of-character surprise also comes from Castle.  In the end of the last season, Kate Beckett got an offer a job in Washington, D.C.  and she took it.  Sounds like a reasonable plot complication right?  Uh-uh.  The show has made an absolute point of the fact that Kate Beckett is ultra-committed to her job as a homicide detective, even at the expense of other career ambitions, making the D.C. job feel like a convenient manufactured drama.   Could there have been a way for this same plot point to be in character?  Oh, I'm sure.  Give Kate a couple really sucky cases, where she feels like she doesn't have enough power to get justice and such a change is at least explainable.

So why am I talking about this?  (Other than the fact that I like talking about Castle.)  Of course, when I'm writing my book I want there to be surprises and things that come unexpectedly to the reader.  I hope my novels aren't predictable, but I do think I could use some work on the whole surprising thing.  It's hard, at least for me, to figure out how to do this.   I've really tried, in my own work to look for those little moments where something amazing happens that even I didn't quite plan for.  As far as I know, I've only pulled it off only once in the current novel.   But I'm hoping I that as I revise this draft I can come up with more ways to build pleasant surprises into my story.