It involves everything. Thinking about the elevation of different types of terrain, the reason for the placement of cities or even why a certain group of people wear shoes on the wrong feet.
Everything that we can create to backup the decisions, actions or reason to be of our world's inhabitants (when I say world it can go from a house all the way to a galaxy or universe) should be included.
Some of the points that the authors, including myself, focus on are: Geography, History, Inhabitants and their Races, Religions, Fauna and Flora , Magic and Technology, Architecture and Languages.