The Social Web provides comprehensive and publicly available information about software developers: they can be identified as contributors to open source projects, as experts at maintaining weak ties on social network sites, or as active participants to knowledge sharing sites. These signals, when aggregated and summarized, could be used to define individual profiles of potential candidates: job seekers, even if lacking a formal degree or changing their career path, could be qualitatively evaluated by potential employers through their online contributions. At the same time, developers are aware of the Web's public nature and the possible uses of published information when they determine what to share with the world. Some might even try to manipulate public signals of technical qualifications, soft skills, and reputation in their favor. Assessing candidates on the Web for technical positions presents challenges to recruiters and traditional selection procedures; the most serious being the interpretation of the provided signals. Through an in-depth discussion, we propose guidelines for software engineers and recruiters to help them interpret the value and trouble with the signals and metrics they use to assess a candidate's characteristics and skills.