In 2000, wages of full time full year workers were more than 30 percent higher in metropolitan areas of over 1.5 million people than rural areas. The monotonic relationship between wages and city size is robust to controls for age, schooling and labor market experience. In this paper, we decompose the city size wage gap into various components. We propose an on-the-job search model that incorporates latent ability, search frictions, firm-worker match quality, human capital accumulation and endogenous migration between large, medium and small cities. Counterfactual simulations of the model indicate that variation in returns to experience and differences in wage intercepts across location type are the most important mechanisms contributing to the overall city size wage premium. Steeper returns to experience in larger cities is more important for college graduates while differences in wage intercepts is more important for high school graduates. Sorting on unobserved ability within education group and differences in labor market search frictions and distributions of firm-worker match quality contribute little or slightly negatively to observed city size wage premia in both samples.