Test and fix extensibility of current schemas

[ibboard]

ticket:150 requires an extension to the schema. Ideally, WarFoundry should be both backwards and forwards compatible (i.e. newer apps can use older files but not take advantage of new features because the files don't define it, but older apps can also use newer files and just ignore the newer features that they don't understand). This hasn't been properly tested, though, and I suspect the schemas might not be correctly defined to support this as they stand.

[ibboard]

Extending schemas causes file load to file. Need to investigate how to get C# to recognise the anyAttribute and any tags in the schema (​details)

[ibboard]

Can't seem to get rid of

XmlSchema error: Attribute declaration was not found for http://ibboard.co.uk/warfoundry/attrib:extraAttrib XML

even when schema has <anyAttribute> and the schema for the extra attribute is loaded in the namespace manager and the schema cache.

[ibboard]

Ooops - helps if the extra schema is actually copied to the output folder (or if Mono would say "can't find file" rather than ignoring the fact you've added a non-existant schema).

[ibboard]

(In r277) Re #152: Test and fix extensibility of current schemas

• Add extensibility to Race schema

Also:

• Code tidy for WarFoundryXmlFactoryUtils

[ibboard]

Backwards compatibility is looking possible through careful definition of new attributes with default values or new elements that are optional, but forwards compatibility needs more exploration as it will currently fail validation because the old app won't understand the new namespace.

[ibboard]

Remove #175 because that level of backward/forward compatibility is too much work for an initial beta version.

Solution for forward compatibility is ​XSLTs to ​remove things we don't understand and ​post-cleansing validation instead of load-time validation.

[ibboard]

(In r304) Re #152: Test and fix extensibility of current schemas

• Add "any" elements and "anyAttributes" to schemas where appropriate

Still need to handle cleansing with XSLT

[ibboard]

(In r305) Re #152: Test and fix extensibility of current schemas

• Fix use of <xs:any> by defining <core:extension> tag
• Remove unnecessary min/maxOccurs=1 attributes

[ibboard]

XSLT method suggested in comment:8 may work, but might need extra thought about attributes outside our namespace (all extensions).

[ibboard]

Alternatives to XSD may provide more useful functionality cleanly, but may not be supported by .Net: ​http://www.xfront.com/backward-forward-compatibility/

[ibboard]

Better solution, based on discussion with a work colleague: don't try for backwards and forwards compatibility on a single schema, instead have a schema for each version with later versions adding on to and re-using the earlier ones. No parsing/prep needed, all unrecognised elements from later schemas should be ignored automatically and namespacing means that the XML XPath queries should still work fine.

If it works, dev versions of WarFoundry will be subject to change but we'll have a lock-down at each versioned release.

[ibboard]

Test Empire.race file with attributes from extra namespaces

[ibboard]

The previous attachment adds

xmlns:ext="http://ibboard.co.uk/warfoundry/attrib" ext:extraAttrib="en"

to the standard test race, but the schema for ​http://ibboard.co.uk/warfoundry/attrib isn't defined, so we get

Problem validating against schema for WarFoundry data: XmlSchema error: Attribute declaration was not found for http://ibboard.co.uk/warfoundry/attrib:extraAttrib XML  Line 2, Position 155.

There will be a fix somewhere, but this effectively limits our extensibility until we can handle (ignore) any schemas that we don't know but validate against those that we do know. Method for ​post-validation may help.

[ibboard]

(In r496) Re #152: Test and fix extensibility of current schemas

• Split out some reusable parts of Race schema
• Replace use of "xs:all" with "xs:sequence"
• Replace use of "core:extension" with standard "xs:any"
• Start to use xs:key and xs:keyref to get more built-in validation

[ibboard]

(In r499) Re #152: Test and fix extensibility of current schemas

• Refactor army schema

[ibboard]

(In r500) Re #152: Test and fix extensibility of current schemas

• Remove junk text used to test breaking the schema

[ibboard]

(In r502) Re #152: Test and fix extensibility of current schemas

• Fix complex type for unit - should be named, not IDed

[ibboard]

(In r503) Re #152: Test and fix extensibility of current schemas

• Add unit test for parsing extended document without XSD for namespace

[ibboard]

Pre or post-load validation doesn't make a difference. Validation process seems to need all schemas or none (​Mono source code).

I've not yet worked out how to either do partial validation (only validate against the schemas we have) or use an XSLT or similar to remove all unknown elements/attributes, but retain any elements/attributes that might be used by extensions that supply their own schemas (XSLT is a fixed document, as far as I've seen).

[ibboard]

Recent "fixes" to move from xs:any to xs:sequence seem to work fine in Mono, but not in .Net. Latest WinForms app currently complains about ambiguity of ##any with race:unitAbilities.

[ibboard]

(In r505) Re #152: Test and fix extensibility of current schemas

• Make xs:anyAttribute and xs:any do lax parsing so that we don't get schema errors
• Move Army schema away from core:extension to straight xs:any

Needs more testing

[ibboard]

(In r506) Re #152: Test and fix extensibility of current schemas

• Update to newer racex file for testing that matches our sequence-ordered schema

[ibboard]

(In r510) Re #152: Test and fix extensibility of current schemas

• Disable UPA (Unique Particle Attribution) check to get same behaviour in .Net as Mono and as new XML Schema 1.1 draft
• Unit test with extended schema now passes. More tests needed

[ibboard]

(In r516) Re #152: Test and fix extensibility of current schemas

• Move (optional) categories overrides before unit list so that ordering makes a bit more sense
• Make anyAttribute in Categories schema do lax processing

[ibboard]

(In r517) Re #152: Test and fix extensibility of current schemas

• Allow any element within category (lax processing)

[ibboard]

(In r518) Re #152: Test and fix extensibility of current schemas

• Add lots of extra attributes (from non-existant schema and from existing schema but not defined in schema)
• Add category overrides

[ibboard]

(In r519) Re #152: Test and fix extensibility of current schemas

• Make race and core schemas fully extensible and forward compatible

[ibboard]

(In r520) Re #152: Test and fix extensibility of current schemas

• Simplify test data and test all possible tags

[ibboard]

(In r521) Re #152: Test and fix extensibility of current schemas

• Test system and army files
• Test multiple unknown children in racex file

[ibboard]

(In r522) Fixes #152: Test and fix extensibility of current schemas

• Complete forward and backward compatibility of schemas (according to testing)

[ibboard]

(In r525) Re #152: Test and fix extensibility of current schemas

• Fix typo - "processContents" (plural, not singular)

no-open-ticket

[ibboard]

Release as part of v0.1b7

[ibboard]

Move to new "released" resolution for v0.1b7