The novels of Haruki Murakami make no sense. Better, they make no sense to a Japan obsessed with work and duty and on the perpetual edge, as John le Carré once quipped, of a collective nervous breakdown. Murakami's surreal tales have minimal contact with this country. Likewise, his brainy drop-out characters do their best to avoid cultural stereotypes, right down to showing little interest in gainful employment.

Millions of Japanese revere his fiction precisely for this stance, a rejection of conformity coupled with an earnest search for meaning. Many critics aren't sure how to assess – or perhaps simply access – his confabulations. The stories seem so spacey. They don't seem very Japanese.

Kafka on the Shore is Murakami's first "big" book since 1998's The Wind-Up Bird Chronicle earned him the kind of acclaim in the West that he had previously enjoyed in Japan. As such, expectations for it are high, especially given the attention paid to his short fiction in recent years by magazines such as The New Yorker and Harper's. Whatever else there is about the new novel, it cannot be said to lack the courage of its convictions, or its quirks.

The plot eludes easy summary. Kafka Tamura is a 15-year-old Tokyo runaway. He is alienated from his artist father and desperate to understand why his mother abandoned him when he was a child, taking his sister along with her. Kafka lifts weights and reads voraciously and listens to John Coltrane. He looks older than he is, and moves among adults with uncommon ease.

His vague quest is to find his missing parent. Equally, the teenager is haunted by the decidedly oedipal prophesy that he will one day kill his father and sleep with both his mother and sister. Riding a bus to a provincial city, Kafka is drawn first to a girl the approximate age of his sister, and then to a private library, where he falls under the spell of a cross-dressing librarian named Oshima and a certain Miss Saeki. She is a former pop sensation whose one composition was an enigmatic ballad called Kafka on the Shore.

Meanwhile, there is Mr. Nakata, an elder from the same Tokyo neighbourhood that Kafka fled. As a boy during the war, Nakata was among a group of schoolchildren who passed out for a period atop a hill, shortly after a plane – or was it a UFO? – flew overhead. Unlike the others, however, the boy remained unconscious for weeks. When he finally awoke, his mind had been erased and his capacities diminished. As consolation, the old man can communicate with cats. Cats, in turn, can talk to him, allowing for a career as a finder of missing felines. (A missing cat also features in The Wind-Up Bird Chronicle.)

That skill leads Nakata to the house of one Johnnie Walker, a phantom who eviscerates cats and devours their hearts. The harmless simpleton is goaded into murdering the phantom, who may, in fact, have been some manifestation of Kafka Tamura's father. It seems so to Kafka, at least, when he awakens that same night hundreds of miles away but still soaked in someone's blood. Thus, the dream of a runaway teen and the apparent reality of a damaged old man intersect for the first time.

The rest of Kafka on the Shore is a racing philosophical whodunit equipped with neither an obvious philosophy nor a conventional resolution to help steer its course. The plot grows ever more hallucinatory as the characters approach convergence, including a cameo by another phantom, this one disguised as Colonel Sanders.

How Murakami renders such a bewildering story so compelling, and so touching, is a testament to his genius. As always with his fiction, part of the exhilaration comes from the feeling that the author had no more idea of where he might be headed when he was writing the book than readers have while reading it. The work of the "real" Franz Kafka often elicits a similar response, and Murakami is openly in debt to the great Czech author both for his narrative labyrinths and his stylistic freefalls.

Such daring brings certain risks. This is especially true when it is executed with the haste evident in sections of Kafka on the Shore, especially the final 100 pages. In the past, Murakami has lowered the risk factor by writing gracefully laconic prose that served to regulate his unbridled imagination. There is no such mechanism in the new novel, with the inevitable results. Those passages of clunky, repetitive writing certainly do jar.

But even this flaw will probably further endear the book to his admirers, in particular those from his native land. Murakami's novels often include walk-on parts for ordinary folk who interrupt their routines to assist his out-of-the-ordinary characters in their odd quests. In Kafka on the Shore, a truck driver named Hoshimo assumes the role, and his delight at blowing off work, and in hanging around waiting for something even stranger to happen, may be an analogue for the connection between this writer and his audience.

Critics who accuse Murakami of propagating a muddled hippie philosophy simply aren't on the same page as those who devour his books. Aside from displaying a detachment consistent with Buddhism, his fiction isn't overly attracted to abstract thinking about life or death. His guiding lights, rather, are a yearning for connection and a persistent intimation of the loss that accompanies all journeys from cradle to grave. As he moves deeper into his career, his books seem to be seeking even more profound resonances of this instinctive nature.

Here is true freedom, Murakami suggests: freedom to dream, to meander, to be alert to the sheer imponderability of existence. Here it is, if only people are willing, in effect, to stop making sense. One needn't be Japanese to find the assertion uplifting and even heroic.