In order to make it work simply change following options.

Enable all needed modules, in this case i am configuring it for streaming.

"mod_flv_streaming",
"mod_h264_streaming",

Specify file extension

flv-streaming.extensions = ( ".flv" )
h264-streaming.extensions = ( ".mp4" )

Now enable document root from where you want to serve content.

server.document-root        = "/srv/www/html/tube.xyz.com/video/"

Setup Log / PID file

server.errorlog             = "/var/log/lighttpd.error.log"
accesslog.filename          = "/var/log/lighttpd.access.log"
server.pid-file            = "/var/run/lighttpd.pid"

Setup server port/IP

server.port                = 81
server.bind                = "127.0.0.1"

Setup server user

server.username            = "ketan"
server.groupname           = "ketan"

Finally setup virtual host.

$HTTP["host"] =~ "tube.xyz\.com" {
            server.document-root = "/srv/www/html/tube.xyz.com/video"
}