The complexity of processing video signals arises from the large number of techniques and considerations involved. This paper explores some of the key concepts involved in the representation and processing of video signals and explores how modern video systems have attempted to maximize the efficacy of each of these factors.