Current version

v1.10.4 (stable)

Navigation

Main page
Archived news
Downloads
Documentation
   Capture
   Compiling
   Processing
   Crashes
Features
Filters
Plugin SDK
Knowledge base
Contact info
Forum
 
Other projects
   Altirra

Search

Archives

01 Dec - 31 Dec 2013
01 Oct - 31 Oct 2013
01 Aug - 31 Aug 2013
01 May - 31 May 2013
01 Mar - 31 Mar 2013
01 Feb - 29 Feb 2013
01 Dec - 31 Dec 2012
01 Nov - 30 Nov 2012
01 Oct - 31 Oct 2012
01 Sep - 30 Sep 2012
01 Aug - 31 Aug 2012
01 June - 30 June 2012
01 May - 31 May 2012
01 Apr - 30 Apr 2012
01 Dec - 31 Dec 2011
01 Nov - 30 Nov 2011
01 Oct - 31 Oct 2011
01 Sep - 30 Sep 2011
01 Aug - 31 Aug 2011
01 Jul - 31 Jul 2011
01 June - 30 June 2011
01 May - 31 May 2011
01 Apr - 30 Apr 2011
01 Mar - 31 Mar 2011
01 Feb - 29 Feb 2011
01 Jan - 31 Jan 2011
01 Dec - 31 Dec 2010
01 Nov - 30 Nov 2010
01 Oct - 31 Oct 2010
01 Sep - 30 Sep 2010
01 Aug - 31 Aug 2010
01 Jul - 31 Jul 2010
01 June - 30 June 2010
01 May - 31 May 2010
01 Apr - 30 Apr 2010
01 Mar - 31 Mar 2010
01 Feb - 29 Feb 2010
01 Jan - 31 Jan 2010
01 Dec - 31 Dec 2009
01 Nov - 30 Nov 2009
01 Oct - 31 Oct 2009
01 Sep - 30 Sep 2009
01 Aug - 31 Aug 2009
01 Jul - 31 Jul 2009
01 June - 30 June 2009
01 May - 31 May 2009
01 Apr - 30 Apr 2009
01 Mar - 31 Mar 2009
01 Feb - 29 Feb 2009
01 Jan - 31 Jan 2009
01 Dec - 31 Dec 2008
01 Nov - 30 Nov 2008
01 Oct - 31 Oct 2008
01 Sep - 30 Sep 2008
01 Aug - 31 Aug 2008
01 Jul - 31 Jul 2008
01 June - 30 June 2008
01 May - 31 May 2008
01 Apr - 30 Apr 2008
01 Mar - 31 Mar 2008
01 Feb - 29 Feb 2008
01 Jan - 31 Jan 2008
01 Dec - 31 Dec 2007
01 Nov - 30 Nov 2007
01 Oct - 31 Oct 2007
01 Sep - 30 Sep 2007
01 Aug - 31 Aug 2007
01 Jul - 31 Jul 2007
01 June - 30 June 2007
01 May - 31 May 2007
01 Apr - 30 Apr 2007
01 Mar - 31 Mar 2007
01 Feb - 29 Feb 2007
01 Jan - 31 Jan 2007
01 Dec - 31 Dec 2006
01 Nov - 30 Nov 2006
01 Oct - 31 Oct 2006
01 Sep - 30 Sep 2006
01 Aug - 31 Aug 2006
01 Jul - 31 Jul 2006
01 June - 30 June 2006
01 May - 31 May 2006
01 Apr - 30 Apr 2006
01 Mar - 31 Mar 2006
01 Feb - 29 Feb 2006
01 Jan - 31 Jan 2006
01 Dec - 31 Dec 2005
01 Nov - 30 Nov 2005
01 Oct - 31 Oct 2005
01 Sep - 30 Sep 2005
01 Aug - 31 Aug 2005
01 Jul - 31 Jul 2005
01 June - 30 June 2005
01 May - 31 May 2005
01 Apr - 30 Apr 2005
01 Mar - 31 Mar 2005
01 Feb - 29 Feb 2005
01 Jan - 31 Jan 2005
01 Dec - 31 Dec 2004
01 Nov - 30 Nov 2004
01 Oct - 31 Oct 2004
01 Sep - 30 Sep 2004
01 Aug - 31 Aug 2004

Stuff

Powered by Pivot  
XML: RSS feed 
XML: Atom feed 

§ The ARC file format

Finding comprehensive specs for a file format is often hard.

There are a few file formats that have really well thought out documentation, such as PNG. Then there's the archive format ARC, which I was trying to write a decoder for. The great thing about search engines on the Internet is their ability to magnify the popularity of a single document called "The ARC archive file format" that doesn't actually tell you most of what you need to know about the format, such as the bitstream ordering for bit packing.

Now, there is source code available for ARC extraction, but using source code isn't always the best idea. First, there can be legal issues if the licensing or origin of the code turns out to be unfavorable, but another problem is that source code often isn't very good documentation and often still needs to be reverse engineered a bit to extract the essential bits... assuming it's even comprehensive. As such, I often avoid using source code as a reference if I can. Fortunately, the compression methods used here aren't crazy like arithmetic encoding and can be eyeballed in a hex editor for verification, so with some elbow grease the rest can be determined from existing archives and decoders.

Here's what I've determined:

Header fields

The ARC file format doesn't have a main file header, only individual headers for each file or block. The best doc I've found for the header is one off of Simtel:

http://cdn.simtel.net/pub/simtel/00/03/99/43/arc_file.inf

For the most part the header is documented pretty well, but this file has clarifications in several files that are documented incorrectly or omitted elsewhere:

Compression mode 3 (packed)

A form of run-length encoding, where the byte 0x90 is used to denote a run. This means that like PCX encoding, it has the dubious honor of favoring some byte values over others. 0x90 0x00 is the escape for the 0x90 literal, while any other follow byte is the number of times to repeat the previously emitted byte, plus one. The wiki for The Unarchiver also documents this format as Rle90:

http://code.google.com/p/theunarchiver/wiki/Rle90Algorithm

The algorithm is also used in a pre-pass on many of the later compression algorithms.

Compression mode 4 (squeezed)

A combination of run-length encoding (RLE) and Huffman coding. The run-length encoding is the same as Packed (mode 3) and is decoded after the Huffman decoder. The compressed format is as follows:

This form of tree encoding has the advantage that it's easy to read and can be used directly in decoding, but takes quite a bit of space, 1K for a full tree. Canonical Huffman tree encoding is a lot more efficient -- just using 4-bit lengths drops this to 128 bytes.

Compression mode 8 (crunched)

This is a combination of run-length encoding (RLE) and Lempel-Ziv-Welch (LZW) encoding. The run-length encoding is the same as Packed (mode 3) and is decoded after the LZW decoder. As usual, there are crucial details missing from most documents:

Comments

Comments posted:


ARC? Where are you getting stuff in this format from?

ender - 04 08 11 - 04:00


My guess would be that it's for the Altirra Atari emulator project. Not VirtualDub.

Scott - 05 08 11 - 09:00

Comment form


Please keep comments on-topic for this entry. If you have unrelated comments about VirtualDub, the forum is a better place to post them.
Name:  
Remember personal info?

Email (Optional):
Your email address is only revealed to the blog owner and is not shown to the public.
URL (Optional):
Comment: /

An authentication dialog may appear when you click Post Comment. Simply type in "post" as the user and "now" as the password. I have had to do this to stop automated comment spam.



Small print: All html tags except <b> and <i> will be removed from your comment. You can make links by just typing the url or mail-address.