Manifest Format

0. Archive signing

This proposal does not specify the format of the archive file. Any format supporting hierarchical paths is acceptable. Files added to the archive for this proposal will be kept in a directory as ordinary files in the archive.

1. Reasons for this scheme.

No modification to HTTP or HTTP servers is necessary.
Archive formats which are already widely deployed may be used.
Users do not need to have cryptographic tools to use files contained within these archives.
Multiple signers.
Multiple signing algorithms.
Allows different portions of archives to be signed by different parties, as well as overlap, and unsigned files.
Supports needs for Java archives and installable software distributions.
Adheres as much as possible to existing standards (both file format and PKCS7 signing.)
One signature verification per archive per signer.
Progressive verification.

2. Filenames in the archive

Archives are signed by creating a "META-INF/" directory at the top level of the path tree. A leading "/" is acceptable if other pathnames contain a leading "/" (ie, "/META-INF/").

There is exactly one manifest file in the archive, with pathname

  META-INF/MANIFEST.MF

Signature files have pathnames of the form

  META-INF/x.SF

for some string x containing only the characterse A-Z 0-9 and dash or underscore. x must not be more than 8 characters.

The encoding of non-ASCII characters in filenames (if they are supported) is defined by the archive format or some other convention, not by this specification. But filenames in the META-INF directory are restricted as above.

The names "META-INF", "MANIFEST.MF", and the filetype ".SF" should be generated as uppercase, but recognized in any case.

3. Name-Value pairs and sections

In most cases, information contained within the manifest file or signature files is represented as so-called "name: value" pairs inspired by the RFC822 standard.

Groups of name-value pairs are known as a "section". Sections are separated from other sections by empty lines.

Binary data of any form is represented as base64. Continuations are required for binary data which causes line length to exceed 72 bytes. Examples of binary data are hashes and signatures.

Implementations shall support header values of up to 65535 bytes.

Specification:

  section: *header +newline
  nonempty-section: +header +newline
  newline: CR LF
         | LF
         | CR (not followed by LF)

; That 'not followed by LF' probably requires some minor
; ugliness in the parser. Sorry.

  header: alphanum *headerchar ":" SPACE *otherchar newline
          *continuation

  continuation: SPACE *otherchar newline

; RFC822 has +(SPACE | TAB), but this way continuation lines 
; never get mangled.

  alphanum: {"A"-"Z"} | {"a"-"z"} | {"0"-"9"}

  headerchar: alphanum | "-" | "_"

  otherchar: any Unicode character except NUL, CR and LF

; Also: To prevent mangling of files sent via straight e-mail, no 
; header will start with the four letters "From".

; When version numbering is used:

  number: {"0"-"9"}+

; The number needn't be, e.g. 1.11 is considered to be later
; than 1.9. Both major and minor versions must be 3 digits or less.

4. Manifest file.

The manifest file consists of a list of files present within the archive itself. Not all files in the archive need to be listed in the manifest, but all files which are to be signed must be listed. The manifest file itself must not be listed.

A preliminary section appears at the top of the file containing, at minimum, this standard's version number.

  Manifest-Version: 1.0

Optionally, a version required for use may be specified:

  Required-Version: 1.0

If Version: is higher than Required-Version, extensions may be used.

The manifest file consists of sections which are entries for various files in the archive.

Many of the headers themselves are not covered by this specification. The following are required:

  Name: dirpath/whatever.class
  Hash-Algorithms: (list of algorithms)
  (algorithm)-Hash: (base-64 representation of hash)

The "Name:" header shall contain a relative path to the file being signed within the archive, OR an absolute URL referencing data outside the archive.

Multiple hash algorithms may be listed (and corresponding (*)-Hash: lines must exist for each one listed); at least one must be present.
Two entries for the same file may not appear in the manifest file. When duplicate entries do occur, only the first is recognized.
Headers which are not understood are ignored. Such headers may include "policy": information used by applications.

Example manifest file:

  Manifest-Version: 1.0

  Name: common/class1.class
  Hash-Algorithms: MD5
  MD5-Hash: (base64 representation of MD5 hash)

  Name: common/class2.class
  Hash-Algorithms: MD5, SHA
  MD5-Hash: (base64 representation of MD5 hash)
  SHA-Hash: (base64 representation of SHA hash)

Specification:

  manifest-file: "Manifest-Version: 1.0" newline
                 manifest-start
                 *manifest-entry

  manifest-start: section

  ; Optional header is
  ;   Required-Version: number "." number
  ;
  ; Required-Version indicates that only tools of the given version
  ; or later can be used to manipulate the file.

; The value of Hash-Algorithms is a comma-separated-list:

  comma-separated-list: +headerchar "," *whitespace comma-separated-list
                      | +headerchar

5. Signature instructions

Each signer is represented by a signature file.

A preliminary section appears at the top of the file containing, at minimum, this standard's version number.

  Signature-Version: 1.0

Information supplied by the signer but not specific to any particular pathname should be included in this preliminary header.

The major part of the file is similar form to the manifest file. The purpose here is to allow "policy" to be embedded in headers supplied by the signer rather than the manifest owner (who is normally the person who generated the archive.)

Signature files consist of sections which are essentially lists of names, all of which must be present in the manifest file. Extra headers may be included here. A hash is also present, but this is a hash of the entry in the manifest file.

Paths or URL's appearing in the manifest file but not in the signature file are not used in calculation. This allows subsets of the archive to be signed.

Only the Name: is required.

Validating a file:

The signature is first verified when the manifest is first parsed. This verification can be remembered, for efficiency. This only validates the signature directions themselves, not the actual archive files.

To validate a file, a hash value in the signature file is compared against a hash calculated against the corresponding entry in the manifest file. Then, a hash value in the manifest file is compared against a hash calculated against the actual data referenced in the "Name:" header -- relative path or URL.

 Example signature file:

  Signature-Version: 1.0

  Name: common/class1.class
  Hash-Algorithms: MD5
  MD5-Hash: (base64 representation of MD5 hash)

  Name: common/class2.class
  Hash-Algorithms: MD5
  MD5-Hash: (base64 representation of MD5 hash)

Specification:

  signature-file: "Signature-Version: 1.0" newline
                  signature-start
                  *signature-entry
                  signature-end

  signature-start: section

; Optional header is
;   Required-Version: number "." number
; This has the same meaning as in manifest-start.
; The only required header is Name. Its format is the same
; as in a manifest-entry.

6. Digital Signatures

A digital signature in the specification is a signed version of the ".SF" (signature instructions) file. These are binary files not intended to be interpreted by humans.

Digital signature files have the same filename as the .SF file but different extension. The extension varies depending on the type of digital signature.

  .RSA      (PKCS7 signature, MD5 + RSA)
  .DSA      (PKCS7 signature, DSA)
  .PGP      (Pretty Good Privacy Signature)

For those formats that do not support external signed data, the file shall consist of a signed copy of the .SF file. Thus some data may be duplicated and a verifier ought to compare the two files.

Formats that support external data either reference the ".SF" file, or perform calculations on it with implicit reference.

Each .SF file may have multiple digital signatures, but those signatures ought to be generated by the same legal entity.

Filetypes may be 1 to 3 "alphanum" characters. Filetypes unrecognized must obviously be ignored.

7. Notes

Before parsing:

if the last character of the file is an EOF character (code 26), the EOF is treated as whitespace.
two newlines are appended (one for editors that don't put a newline at the end of the last line, and one so that the grammar doesn't have to special-case the last entry, which may not have a blank line after it).

Headers:

In all cases for all sections, headers which are not understood are ignored.
Header names are case insensitive. Programs which generate manifest and signature files should use the cases shown in this specification however.
Header names cannot be repeated within a section.

Versions:

Manifest-Version and Signature-Version must be first, and in exactly that case (so that they can be recognized easily as magic strings). Other than that, the order of headers within a section is not significant.

Ordering:

The order of manifest-entries is not significant.
The order of signature-entries is not significant, except that the hashes that get signed are in the that order.

Line length:

No line may be longer than 72 bytes (not characters), in its UTF8-encoded form. If a value would make the initial line longer than this, it should be continued on extra lines (each starting with a single SPACE).

Errors:

If a file cannot be parsed according to this spec, a warning should be output, and none of the signatures should be trusted.

Limitations:

because header names cannot be continued, the maximum length of a header name is 70 bytes (there must be a colon and a SPACE after the name).
NUL, CR, and LF can't be embedded in header values, and NUL, CR, LF and ":" can't be embedded in header names. We could add quoting for these, but it probably isn't worth the extra complication. Another possibility would be to require them to be encoded using two bytes.
It's more difficult to test conformance to the spec without limits on the complexity of a file. So, implementations should support 65535-byte (not character) header values, and 65535 headers per file. They might run out of memory, but there should not be hard-coded limits below these values.

Signers:

It is technically possible that different entities may use different signing algorithms to share a single signature file. This violates the standard, and the extra signature may be ignored.

Algorithms:

No hash algorithm or signature algorithm is mandated by this standard. However, the following is generally expected:
Hash: At least one of MD5 and SHA
Signature: PKCS7

java-security@java.sun.com