The Vile Village

In this book, in order to investigate the mysterious "V.F.D.", the Baudelaires decide to live with a village named V.F.D. which has offered to care for them. However, they soon discover the villagers are insane in their cultish hive mind mentality, obsessed with worshiping crows, and governed by power-hungry seniors who enact thousands of ridiculous rules and burn citizens to death at the stake for absurd reasons like putting more than fifteen pieces of nuts on a hot fudge sundae.
This book is considered to be the "plot twist" of the series because the Baudelaires can no longer call on Mr. Poe for assistance after the events of this book (although he was barely any help to begin with), and the children are deemed "criminals." Also, after this point, the Baudelaires are not assigned any more legal guardians.