Time line thumbnails plugin for Flowplayer Flash generates and displays preview images on the players' seek bar when the user however over it. The great thing about it you do not need to prepare and store the thumbnails, the plugin takes care of it by extracting and decoding the key frames from H264 mp4 videos. It downloads meta data and the necessary key frames only using byte-range http requests. Your H264 mp4s should contain enough key frames which could be arranged though settings during the encoding process but happens naturally each time a scene changes. The plugin is one in serious of plugins based on http mulitstreaming and mp4 parsing using javascript/actionscript.
Download and use it now: Time line thumbnails plugin Flowplayer