Hypervideo is an instance of hypermedia. A video with hyperlinks in designated areas of certain frames is just like hypertext in the WWW environment. A video can be viewed as a main document the producer wants to present and it can include many hyperlinks. Thus, it can be presented with more auxiliary materials. For effective browsing and navigation, semantic annotation is also essential for the visual content of movies. It is very important to show any particular object that the movie producer wants to highlight in educational videos or commercials. Synchronization problem needs to be solved when the different objects are rendered in a multimedia player. We propose an interactive and structure-based video authoring system, in which we can describe the video object and make multistory movies. The system provides a manual object-based interface for the film producer to select any meaningful object and annotate it. We also provide a video story construction function to interact with the video viewer. A hypervideo player is also developed to play the video file made by the authoring system.