Video event detection is one of the most important research topics in computer science. The recognition of complex events, e.g. “birthday party”, “wedding ceremony” or “attempting a bike trick”, is even more difficult since complex events consist of various human interactions with different objects in diverse environments with variable time intervals. In this book, we propose two learning methods. The first proposed method, named maximal evidence learning (MEL), is based on a large-margin formulation that treats instance labels as hidden latent variables, and infers the instance labels and the instance-level classification model simultaneously. MEL can infer optimal solutions by learning as many positive instances as possible from positive videos, and negative instances from negative videos. The second proposed method is called evidence selective ranking (ESR). ESR is based on static-dynamic instance embedding, and employs infinite push ranking to select the most distinctive evidence. Extensive analysis on large-scale video event datasets shows significant performance gains by both methods.In addition, we also demonstrate key selected evidence is meaningful to humans and can be used to locate video segments that signify an event.