This manuscript proposes a real-time system to detect, recognize, and track the multiple vehicles on the roadway images. The system uses an image capture system to snap a sequence of images and a moving object segmentation method to separate the moving vehicles from the image sequences. After the segmentation of the objects, the objects can be classified and counted by the proposed detection and tracking methods respectively. In this system, the occlusive problems are solved by the proposed occlusive segmentation method and then each segmented vehicle is recognized according to their outlines and tracked by a tracking algorithm at the same time. Even though the occlusive vehicles appearing in the images have been keeping merging or having more than two vehicles to merge together all the time, the system can still segment the vehicles. The proposed recognition method uses the visual length, visual width, and roof of vehicles to classify the vehicles to vans, utility vehicles, sedans, mini trucks, or large vehicles. Experiments obtained by using complex road scenes are reported, which demonstrate the validity of the method in terms of robustness, accuracy, and time responses.