Background. Although selection favors exploitative competition within groups, a group of hypercompetitive individuals may be less productive than a cooperative group. When competition is costly for group fitness, among-group selection can favor groups with 'policing' individuals who reduce within-group competition at a cost to their own fitness, or groups of individuals who restrain their competitive intensity ('self policing'). We examine these possibilities in a series of explicit population-genetic models. Results. By comparing results from models of half and full sib structured populations, we find that increased relatedness increases the strength of among-group selection against competition genotypes, and increases the strength of among group selection favoring policing genotypes. However, the strength of selection favoring costly policing behavior also increases with increased levels of competition. When levels of competition and policing feedback on one another, groups with lower levels of relatedness can favor higher levels of costly policing. Conclusion. The result of the joint selection on policing and competition leads to results different from those based on the evolution of policing alone. Our model makes 'long term' predictions equivalent to those of optimization models, but we also show the existence of protected polymorphisms of police and civilians, as well as competitors and non-competitors.