There have been numerous recent calls for research on the human side of software engineering and its impact on project success. An analysis of which challenges in software engineering teams are most relevant and frequent is still missing. As teams are more international, it is more frequent that their members have different personal values as well as different communication habits. Additionally, virtual team setups (working geographically separated, remote communication using digital tools and frequently changing team members) are increasingly prevalent. We designed a survey instrument and asked respondents to assess the frequency and criticality of a set of challenges, both within teams as well as between teams and clients. For the team challenges, we asked if mitigation measures were already in place to tackle the challenge. Respondents were also asked to provide information about their team setup. The survey included an instrument to measure Schwartz human values. The survey was first piloted and then distributed to professionals working in software engineering teams. In this article, we report on the results obtained from 192 survey respondents. We present a set of challenges that takes the survey feedback into account and introduce two categories of challenges; inter-personal and intra-personal. We found no evidence for links between personality values and challenges. We found some significant links between the number of distinct nationalities in a team and certain challenges. We found evidence that a higher degree of virtualization leads to an increase of the frequency of some human challenges. We present a set of human challenges in software engineering that can be used for further research on causes and mitigation measures, which serves as our starting point for a theory about causes of contemporary human challenges in software engineering teams. Our findings warrants further research on human challenges in software engineering and gather more evidence and test countermeasures, such as whether the employment of virtual reality software incorporating facial expressions and movements can help establish a less detached way of communication. IEEE