How is GPU Different from CPU?

What is the main difference between a GPU and a CPU? Can you explain in technical terms how one differs from the other? It is expected that GPU, which is a new type of processor, will make processing faster than using only CPU. I would like to know from someone who uses machines having GPU and CPU if this assertion is true. Does the GPU perform faster and better than using CPU alone or there is not much difference in performance.
