We systematically overview the literature in keyword search and propose several promising research directions. The paper is organized by each agent in the search process; searchers, advertisers and the search engine and reviews the key research issues for each. For each group, we outline the decision process involved in keyword search. For searchers, this involves what to search, where to search, which results to click, and when to exit search. For advertisers, this involves where to bid and which word or words to bid on, and how searchers and auction mechanisms moderate these behaviors. The search engine faces choices on mechanism design, website design and how much information to share with its advertisers and searchers. These choices have implications for customer lifetime value and the nature of competition between advertisers. Overall, we list a number of potential areas of future research arising from the decision processes of these various agents.