Are genetically engineered foods safe?

Grocery shoppers are finding it more and more difficult to avoid food that has been genetically modified. Many processed foods contain quantities of GM foods buried in the list of ingredients. Should people be worried about GM food, or is the media making a fuss about nothing?
Here is a site which contains general, balanced information on the topic:
http://www.ornl.gov/sci/techresources/Human_Genome/elsi/gmfood.shtml
Genetic Roulette, by Jeffrey M. Smith, an investigative reporter, is an exceedingly scary book on the genetic modification of food. After briefly skimming through this book at my local book store, I’ve drawn up a list of foods to avoid. Margarine is at the top of my list. I’ll substitute olive oil for GM oils like canola and corn. More information about the book may be found here:
http://www.seedsofdeception.com/Public/GeneticRoulette/index.cfm