Climate change and variability continue to adversely impact the livelihoods of many agriculture-dependent households in dryland sub-Saharan Africa. Climate vulnerability is shaped by institutions and socioeconomic processes including land tenure arrangements and infrastructural development. This paper employs a participatory mixed-method approach including household questionnaire surveys, key informant interviews, oral narratives and focus group discussions to understand the dynamics of livelihood challenges in 6 dryland farming communities of different vulnerability status in Ghana. Acknowledging the importance of agronomic practices as well as economic and environmental processes in influencing climate vulnerability in dryland farming systems, this paper demonstrates how the complex land tenure system is implicitly involved in shaping the vulnerability of two groups of farmers (migrant farmers in the Ejura Sekyedumase district and female farmers in the Bongo district) by limiting the adaptation options available to these groups. Our results suggest that women's rights regarding land ownership should be formalized in land policy in order to reduce cultural discriminations against them. Opportunities for women to own and formalize land registration titles should be pursued by the Government of Ghana. This will help women to secure property rights over land and land use via mechanisms such as collateral to access credit, which could be used to implement climate adaptation practices. The rights of migrant workers should also be recognized in Ghana’s land policy to provide them with opportunities for adaptation in a similar manner to non-migrants. Climate change adaptation policies should also consider the broader socioeconomic and environmental factors that hinder smallholder farmers' ability to implement adaptation measures.