§ ¶Documentation sucks
If you ever wonder why so many programs come with nonexistent or lousy documentation, it's because programmers hate to write it, and that's because it takes forever to do so. I decided to start working on an updated help file for VirtualDub today, because the help file I'm currently shipping is quite old and incomplete, and the new capture system is basically undocumented. I started at around 2pm today, it's now 8pm, and I think I'm only about a third through the capture system documentation, to say nothing of processing mode. Bleah.
I also just noticed that I've been listening to nothing but Minami Kurabayashi songs for the past six hours, so maybe it's time to take a break.
Hint regarding VirtualDub's new help file: To display help, VirtualDub has to decompress the VirtualDub.vdhelp file and launch a web browser. This can fail for a variety of reasons, such as a read-only program directory. If you run into problems, you can read the help files by changing the file extension to .zip and extracting the double-layered archive.
The help files are processed through a utility I wrote called Lina, the source of which is in the VirtualDub source archive. It takes source files in psuedo-XML format and transforms them into the output HTML files. The design is somewhat similar to XSLT, with the primary differences being that the stylesheet is the data source, and that result trees are automatically reprocessed for templates. I played around with using XSLT 2.0 stylesheets with Saxon, but got stuck on the latter point — it is surprisingly non-intuitive and difficult in XSLT to get it to reprocess result trees and not have it strip out all container nodes. Lina also strips or compresses whitespace as appropriate, which also turned out to be annoying in XSLT.
I then looked into writing a new version of the preprocessor in C#, called Amelia, in the hope of being able to use cool XPath expressions. I immediately got lost in the maze that is System.Xml when I tried to get it to process HTML entities properly. Seems that to get entity processing you have to use XmlValidatingReader instead of XmlReader, and while passing it a DTD is the fastest way to get it to processing entities, it's a pain to get it to use one without having it download the stupid thing every time from w3.org. Then I tried to stick the DTDs in a resource, where I discovered (a) that the VS.NET 2003 managed resource editor is just the XML editor, and (b) the documentation tells you to compile one of the sample applications for an actual resource editor. At which point I just closed the Solution in disgust.
Many of you are probably wondering why I switched to the goofy double-zip format for help files. The reason is that I couldn't stand using Word and WinHelp anymore. Word itself is pretty aggravating — Microsoft still hasn't solved the backspace-screws-the-whole-paragraph problem — but to compile a WinHelp file you have to export RTF, and RTF transfer between Word and the help compiler isn't that reliable (What You See Is Maybe What You'll Get). Bulleted lists, in particular, tend to get messed up. Eventually I want to switch over to using HTML Help, but I don't have the process set up for that yet and I have a budget for how many things I can break each release.
I pretty nice help file tool I stubled upon is HelpMaker, at http://www.vizacc.com
It is capable of generating both WinHelp and Html Help files, from a single source. There are more of these tools of course, but this one is free (as in 'free to use', not open source). Might be too late for you now, but maybe for future projects?
robert - 19 09 05 - 05:14
Windows XP Program Files directory has read-only security permissions for limited useraccounts by default. This is by design. Programs should write back their information to the user's application data directory to avoid problems when there are multiple users on one pc.
hp38guser - 20 09 05 - 05:21
hp38guser: to work well in a limited user account, vdub would need to move its 'filters' directory there too. Will just a 640kb exe left behind, why not just install the whole software in a user's account? Due to the binary being packed, it can't even share memory in case of multiple instanes... So keep it simple, copy it to each user's directory... Anyway I wonder who would have more than 2-3 limited user accounts on a single Windows machine, it's so limited as to be almost unusable...
Now, about documentation: why not create it using OpenOffice, and offer it as a separate download in rtf, OASIS or pdf download? I mean, although contextual help is nice, one has to wonder how comes man still has so much use under Linux. You could also just see what system they use in OpenOffice, and use it yourself...
Mitch 74 - 20 09 05 - 17:54
That's great, except that VirtualDub does not have and has never had an installer, so it wouldn't normally end up there. As long as the normal install vector is unzip and the uninstall vector is the delete key, I do not intend to follow the "spew files to user profile" model. When I have an uninstaller, then I'll reconsider.
I'd rather stick to HTML-based documentation as it is most universal. OASIS/OpenOffice is not that prevalent on Windows, and I don't like PDF.
Phaeron - 20 09 05 - 23:53
For the "no spew" method, I thank you :-)
I have a folder sitting on my desktop with all the self contained apps in it and a link on the desktop to the exe. Various virtualdub versions fill up the majority of that folder.
SonOfAdam - 21 09 05 - 19:26
Writing manuals is hell.
You have to be allways correct and are payed less
than if you would write novels.
Tom Coschwitz (link) - 22 09 05 - 17:18
yeah, pdf is the suxxor!
meanie - 22 09 05 - 22:33
Well, I guess you could just re-use the structure of the documentation offered with avisynth... Thing is, I was thinking about using OOo files to create the documentation because once there you can export it to the format you want: DOC, ODF (which is nice because it's already zip-ed, but you're right, there are no small viewers existing in Windows), PDF, RTF and... HTML :p
Mitch 74 - 23 09 05 - 03:17
Comments from a LUA mode freak (Least-privilege User Access – MS term),
Just want to make a small case for "spewing files to the user profile".
First I want to say thank you for all your efforts for the video community.
Second I want to stress that your program runs fine on a limited user account ( I haven’t had any problems yet) as long as it is not placed in "Program Files". I unzipped it as an admin and ran it 1st time as admin to get the help files created, AFAICR.
Security on the windows platform is getting a more and more urgent matter. Security conscious home and workstation users are (or should be) switching over to running windows in LUA mode or NonAdmin mode like it always have been on unix. At least executable files should never be writeable by anybody but the admin. There simply are no reasons for it and think of all the viruses this would stop if every body ran as nonadmin. (@Mitch 74 - Memory use is not the issue)
Anyway, I could be rambling on about the necessity of making the switch to LUA mode, but it won’t happen for the most users as long as programmers don’t make their programs LUA mode ( ie. WinXP Logo Program) conforming. For the average user the complications coping with LUA mode at the current time is probably to big a hurdle.
As I implied your program seems well-behaved in LUA mode, except ofcourse for the help file thing, but for a fairly technically knowledgeable user that shouldn’t be a problem, which I suppose most users of VirtualDub are.
Sorry for the lengthy rambling, it was just that you kind of hit a nerve on a LUA mode freak with the "spewing files to the user profile" comment ;)
Henrik - 03 10 05 - 02:24
Running everything as LUA isn't a panacea. Programs that are running LUA still have user-level permissions, and as long as that includes network and user filesystem access, trojans or malicious code still has the ability to trash a user's files and infect other machines. Most modern worms and viruses don't need admin access to spread and cause damage. Unless you have an OS that allows you to restrict execution to known executables — and I mean actual known executables, not just filenames — then I don't see how you are any safer having your applications in a read-only Program Files rather than in a local home directory. Without this, it is still possible to generate and run an arbitrary executable with user permissions in the background. AFAIK, Windows can't do it, and the default setup of most Unix systems doesn't do this either (although I have heard of systems with patched Solaris kernels that can).
The real problem with Windows is that the filesystem and Registry are both messes. This makes it difficult to (a) figure out what should and shouldn't be there, and (b) to separate user data and configuration from installed files. Otherwise, it would be a lot easier for users to notice a problem, back up their data, and reinstall part or all of the system from a known good source. It also doesn't help that every major program wants to install its own autorun, own task manager, and a billion common COM components, which makes it hard to distinguish malicious behavior from merely distasteful practices.
I should also note that another reason I don't support Microsoft's file layout plans is that they are a HUGE PAIN IN THE BUTT for those of us who use the command line. I am perfectly fine with /home/foo/x or ~x. There is no way in #$&*( I am going to constantly put up with typing paths to all my data in "C:Documents and SettingsfooMy Documents..."!
By the way, VirtualDub's performance can be degraded if you run it in a LUA environment. For instance, attempting to preallocate a capture file without the SeManageVolume privilege will force VirtualDub to halt while the entire capture file is cleared.
Phaeron - 03 10 05 - 03:11
I'll agree that it isn't not the end solution, an anyway offen you have to run a setup executeable as an admin, which could be infected and the your hole machine is compromised. Thats why I actually would prefer programs distributed in a zip like yours and only run by the user never the admin so that this account never compromissed. I suppose MS MSI install-format is an attempt to overcome that dilema.
And I'll agreed that the C:\Documents and Settings mess is a mess, the GROUP/USER privilege schemes are confusing and that the reg. db is the most idiotic, unsafe, data endangering, user unfriendly bulls**t MS have come up with, but sadly these are the security schemes you have to work with because MS controls the lower level rutines they are based on. Well I agree alot don't I :).
Yes, lowering the priviliges for users isn't a panacea, it's more a strike in the ever ongoing battle agains malicius software. And it makes the pole of blackhats and virus writers capable of doing there dirty deeds smaller. But then again,- they just release an updated root/virus-kit and we have all the script-kiddies on our back again. :(
SeManageVolume ? Is there away to allow limited user these priv. without compromissing security to much?
Henrik - 03 10 05 - 04:04
Regarding #$&* typing C/:Documents and Settings/looooooongPath to accss home. I agree. Love $~ on linux and cygwin.
Isn't it possible to change the "C/:Documents and Settings/" paths with PowerToys. Anyway I think that NFTS have some hardlink capabilities, though I've never digged deeper into it, but it would be nice if you could use them in the same way as symbolic links on efs2. Actually I've made symbolic links to C/:Documents and Settings/user/documents in cygwin but they are of course only accessible there.
Henrik - 03 10 05 - 04:55
"but sadly these are the security schemes you have to work with because MS controls the lower level rutines they are based on"
Sorry,- that was nonsens.
What I think I meant was
"but those are inherited annoyance designed by MS and sadly we just have to live with it."
And now I promise you that I won't spam your blog with futher LUA rambling. :)
Henrik - 03 10 05 - 05:21
Why not just decompress into %temp%/virtualdub, then delete the folder and contents on close? Even if it crashes and doesn't delete until the next close, it's not like it'd be noticable in between the masses of installer junk, winamp junk, and that horrifying adobe junk. Ugh.
Windows can do whitelist-only, but it's through group policy, which rules out XP Home without a lot of registry hacking.
foxyshadis (link) - 03 10 05 - 22:28
It trashed my percent signs around temp, but you know what I mean.
foxyshadis (link) - 03 10 05 - 22:30
Windows XP supports blocking execution of EXE-files that are not approved by the admin. I think it’s called SAFER. It is under the security policies.
You can say: Run everything from c:Programme* and C:Windows*, but no files with the MD5-sum of solitaire and so on. Also run only software signed by admin.
Christian - 22 08 06 - 06:58
i need a password to unzip the download file what is the password
Greg - 14 03 11 - 17:31