Yeah, I know. We all hate labels. But if our ultimate goal is space on a bookstore shelf, we have to be able to suggest to an agent or editor what shelf that might be.
The best place to start is an actual bookstore. Find books like yours and see where they’re shelved (speaking as a former bookstore shelver, I can tell you how subjective this is, so don’t consider these hard and fast rules.) Some categories are traditionally paired, like Mystery/Crime and SF/Fantasy.
Or try Amazon. Look for books similar to yours and scroll down to "Look for Similar Items by Category."
Here are some basic fiction genres. You’ll notice how many overlap or can be combined. It's OK to combine up to three (but not more) in your query. Also, if you don't get nibbles with one category, it's OK to call your work something else. Agents say they do that all the time, depending on what an editor is looking for.
Chick Lit: People in the industry still use this term, but you’re not allowed to put it in your query or they'll make fun of you in a meeting, according to agent Barbara Poelle. It means light, funny women’s dating stories with a distinctive, can-we-talk voice. If it has a happy ending, try to shoehorn it into Romance as a “romantic comedy.” Otherwise, try Women’s Fiction.
Christian/Inspirational: any work that supports a Fundamentalist Christian world view. No explicit language, sex or content. Violence is OK.
Commercial: Traditionally, any plot-driven fiction, but now, according to AgentQuery, this means "high concept" projects with a unique subject and potential audience of zillions: stories that can be summarized in one wow-inducing sentence.
Crime Fiction: Stories centering on the physical aspects of a crime or the workings of the criminal mind.
Detective Fiction: Just-the-facts-ma’am details of bringing a criminal to justice. If the detective has a badge, it’s a Police Procedural.
Erotica: A major market in e-book publishing—especially erotica for women that has romantic and/or paranormal elements.
GLBT: This category sort of annoys me, since it segregates 10% of the population, but if your MC has a minority sexual orientation and this label helps you get published, go for it.
Fantasy: Not just about elves, dragons, and talking badgers any more. Dark Fantasy (vampires, were-persons) Urban Fantasy (spawn of Buffy) and Erotic Fantasy (vamps and were-persons hooking up) are big. Epic Fantasy (Tolkein-inspired) not so much. Try rewriting epics for Middle Grade.
Historical Fiction: A story set fifty or more years in the past that uses the time period as an element of the story.
Horror: Scare the pants off your reader — á la King. Not selling so well right now, according to agent Laurie McLean, except for Vampire Horror. And Zombies. Anything zombified is big. Splatterpunk (ick) has a small but steady readership. I guess it keeps them from torturing small animals.
Literary: Language and character trump story. Get a story published in The New Yorker first.
Mainstream: This once-basic category is on the wane. As Patrick Anderson detailed in his book, The Triumph of the Thriller, former mainstream staples like family chronicles, historical epics, and sweeping Micheneresque sagas are no longer big sellers.
Middle Grade—Novels in all genres for Tweens. Increasingly sophisticated these days.
Multi-Cultural: Anything NOT about middle-class characters of northern European heritage. Generally family sagas. Big plus if it’s set in a current war zone.
Mystery: Crime-solving puzzles. Classic Whodunits, Cozies, Private Eye, and Noir are still going strong, and Historical, Supernatural, and Literary mysteries are hot. Cozy series with craft and hobby themes are steady sellers. And a recent article by author Fleur Bradley for Sisters in Crime mentioned the emergence of “Geezerlit”—mysteries set in nursing homes and retirement communities with octogenarian sleuths.
Romance: Must follow specific publisher guidelines and provide happy endings. (One agent blogs bitterly about love stories submitted as romance.) Hot subgenres are Paranormal, Urban Fantasy, and Time-Travel. Also, Regency, Elizabethan and Scottish Medieval are perennial favorites. Western romance still sells. Plus there’s a growing market for explicitly erotic romance.
Romantic Suspense: Combines elements of romance and mystery with a fast-paced, protagonist-in-constant-jeopardy plot.
Satire: If you’re in the US and write for a sophisticated audience that gets irony, emigrate. Or sneak it in as another genre.
Science Fiction: The plot should be based on science rather than myth or make-believe. Subgenres include Social, Cyberpunk, New Wave, Alternate History, Military, and Apocalyptic. “Hard” science fiction—the kind where you have to know a whole bunch of physics—still sells, but not in the quantities it once did.
Speculative Fiction: Very hot right now. Any fiction that plays with reality. Popular subgenres: Steampunk (set in a faux-Victorian/Edwardian alternate universe) Time-Travel, and Slipstream (surreal literary-fantasy.)
Suspense: Fast-paced adventure with a protagonist in constant peril.
Thriller: Save-the-world, fast-paced adventure. The stakes must be high—not just one person in jeopardy, but civilization itself. Flavors include: Spy, Political, Military, Conspiracy, Techno-, Eco-, Legal, Medical, and Futuristic.
Westerns: Horses, guns, and stoic agricultural workers in the late 19th/early 20th century American West. Kind of a dead horse.
Women’s Fiction: Women struggling against adversity. Can be literary, gritty, weepy, or funny. If you’ve seen similar storylines on Lifetime or Oxygen, chances are it will fit in here. It usually, but not always, includes a realistic love story. Endings can be sad.
Young Adult: Any of the above categories written for teens. Literary novels with teen protagonists sometimes sneak into print as YA to avoid the hasn’t-published-in-The New Yorker police.
Mix and match as you hone your query, and with luck, you’ll find a genre label to reach your potential readers.