Re: video to SVG

[John Swensen]

On Jun 30, 2010, at 9:07 AM, narendra sisodiya wrote:

> 
> 
> On Wed, Jun 30, 2010 at 6:26 PM, Marc Weber <address@hidden> wrote:
> Excerpts from narendra sisodiya's message of Wed Jun 30 14:24:58 +0200 2010:
> > Hi, I am trying to work on a project for Video to SVG conversion.
> > Basically, the classroom videos which has only one white-board and lines and
> > curves can be compressed by converting into vector markup like SVG. All i
> > need to find the curves and lines inside video/imaegs(frames)
> >
> > Any starting points if I use Octave to build this project. I have done some
> > basic matrix manipulation on octave but I have not done any video or image
> > processing using octave or any other tools
> 
> Are you sure that Octave is the best tool for the task (maybe it is. I
> don't know it very well). I'd just use ffmpeg (or mencoder or such) to
> create many .jpg files from viedeo.
> 
> video to image is not a problem.
> I want to manipulate at pixel level in an image to detect lines and curves 
> inside that image and their increments
> 
>  
> Then I'd try using an existing ocr
> solution which outputs .svg files.. (Inkscape has a built in OCR
> command. This could be a starting point.)
> 
> I do not think I need OCR. on whiteboard, there can be 1000 type of 
> handwriting. All i need to convert lines and curves into their markups.
> 

There are still some holes in your specification.  Are you going to want to be 
able to take a completely filled whiteboard with no occlusions and convert it 
to SVG, or are you planning on taking each frame of the video individually and 
converting each frame to SVG and re-concatenate to a movie of SVG files?  Will 
the professor be standing in front of the board and occluding portions of the 
board and casting a shadow on the board?  Did you want to take existing 
lectures and convert them, or will this be new lecture and you can in some 
manner control how they are presented?  I know you probably don't want a high 
cost solution, but if you can constrain how the lecture are being presented, 
the best solution would be to have person sit at a tablet PC or WACOM tablet 
and talk into a microphone as they present their lecture.  In this manner, pen 
strokes and pressures can easily be recorded and the SVG is easily 
reconstructed from the recorded information.

That being said, if you really want to just convert video of a whiteboard to 
SVG then some pretty basic image processing can probably get you pretty far 
(depending on the quality of the video).  If I were doing this, my first stab 
would be as follows:
1) If the camera isn't moving, crop out everything outside the actual 
whiteboard area.  This can be accomplished by taking an initial frame with no 
writing and subtract this background image from each subsequent image.  Make 
sure you normalize the brightness of each image to account for lighting 
changes.  Also make sure that when the lecturer decides to erase the whiteboard 
that they erase it well.  This will have a big effect on the next step.
2) Perform simple thresholding to determine what is writing and what is not.  
This may be an exercise in "knob-turning" based on the color and quality of the 
markers being used.  You might recommend the lecturer use only dark colored 
markers and be liberal in throwing away markers that are starting to dim.
3) Use a connected components algorithm to identify what are real writing and 
what are stray marks on the board.
4) Use an edge detection algorithm on each connected component to define an SVG 
polygon.  You can be smart about decimating the points on the boundary of the 
polygon and even converting the connected components to strokes, but making a 
polygon of the boundary is definitely the easiest.

Under "ideal" conditions, this should do an OK job.  Whenever I have a computer 
vision/image processing problem, my first step is to search SIGGRAPH for an 
existing solution.  If you have access to scholarly publications, you would be 
surprised how easy it is to find solutions to fairly common problems.

John Swensen