XML Schema whiteSpace and the token Data Type

In developing some new XML Schema documents, we came across an unexpected relationship between the XML Schema token datatype and the rules for processing whitespace.

In section 3.3.2 of the XML Schema Part 2: Datatypes Second Edition document, token is defined as follows:

[Definition:] token represents tokenized strings. The •value space• of token is the set of strings that do not contain the carriage return (#xD), line feed (#xA) nor tab (#x9) characters, that have no leading or trailing spaces (#x20) and that have no internal sequences of two or more spaces. The •lexical space• of token is the set of strings that do not contain the carriage return (#xD), line feed (#xA) nor tab (#x9) characters, that have no leading or trailing spaces (#x20) and that have no internal sequences of two or more spaces. The •base type• of token is normalizedString.

Based on this paragraph, we assumed any validator (XSV for example) would indicate an instance error if the value of a token element contained a carriage return, line feed, or tab character. I also assumed a validator would indicate an instance error if the value of a token element contained any leading or trailing spaces or any internal sequences of two or more spaces. These assumptions, however, were not correct as is demonstrated by the following example:

Given the following XML Schema declaration:

<xs:element type="xs:token" name="XmlToken" />

The following are considered valid by validators:

<XmlToken>
  Token
</XmlToken>
<XmlToken>   Token   </XmlToken>
<XmlToken>
  Token1        Token2 
  Token3
  Token4 Token5 Token6
</XmlToken>

The reason these are considered valid token values has to do with the whitespace processing rules. Per section 4.3.6 (whiteSpace) of the XML Schema Part 2: Datatypes Second Edition document, an token actually allows carriage return (#xD), line feed (#xA) and tab (#x9) characters to appear in the value.

[Definition:] whiteSpace constrains the •value space• of types •derived• from string such that the various behaviors specified in Attribute Value Normalization in [XML 1.0 (Second Edition)] are realized. The value of whiteSpace must be one of {preserve, replace, collapse}.

preserve
No normalization is done, the value is not changed (this is the behavior required by [XML 1.0 (Second Edition)] for element content)

replace
All occurrences of #x9 (tab), #xA (line feed) and #xD (carriage return) are replaced with #x20 (space) 

collapse
After the processing implied by replace, contiguous sequences of #x20's are collapsed to a single #x20, and leading and trailing #x20's are removed.

whiteSpace is applicable to all •atomic• and •list• datatypes. For all •atomic• datatypes other than string (and types •derived• by •restriction• from it) the value of whiteSpace is collapse and cannot be changed by a schema author; for string the value of whiteSpace is preserve; for any type •derived• by •restriction• from string the value of whiteSpace can be any of the three legal values. For all datatypes •derived• by •list• the value of whiteSpace is collapse and cannot be changed by a schema author.

Since the whiteSpace value for a token is collapse, all whitespace characters are replaced with a space character, all leading and trailing space characters are removed, and all contiguous space characters are collapsed into a single space character before the XML instance document is validated.

Since all the offensive characters are removed before the document is validated, token values can actually contain carriage return, line feed, and tab character, even though they are forbidden by section 3.2.2 (token) of the XML Schema Part 2: Datatypes Second Edition document.

Enhancements to the Enum.Parse Method

Enum.Parse Method

The .NET Framework has an Enum class that contains two Parse methods (http://msdn.microsoft.com/en-us/library/system.enum.parse.aspx https://docs.microsoft.com/en-us/dotnet/api/system.enum.parse). Both of these methods take a string value and attempt to convert it into an Enum value. This works very well, but it has a few shortcomings.

Shortcoming 1: Enum.Parse Returns Object

The first shortcoming is that the return value from Enum.Parse is an Object. This requires the caller to typecast the result of the Enum.Parse method to the appropriate type. For example:

<FlagsAttribute()> _
Enum Colors
    Red = 1
    Green = 2
    Blue = 4
    Yellow = 8
End Enum

Dim Choice As Colors

Choice = CType([Enum].Parse(GetType(Colors), "Red"), Colors)

To get around this shortcoming I wrote the following method that will return an enumerated value of the appropriate type.

Public Shared Function ParseEnum(Of EnumType As Structure)( _
    ByVal value As String, _
    ByVal defaultValue As EnumType) As EnumType

    Try   

        ' Attempt to convert string to enumeration using the Parse method
        Return CType([Enum].Parse(GetType(EnumType), value, True), EnumType)

    Catch ex As ArgumentException

        ' Return the default value
        Return defaultValue

    End Try

End Function

When this method is used, no typecasting is required. For example:

Dim Choice As Colors

Choice = ParseEnum("Red", Colors.None)

In order to use this method, the enumeration should have some default value that can be used to indicate the string could not be successfully parsed.

Shortcoming 2: Enum.Parse is unaware of the XmlEnum value

The second shortcoming of the Enum.Parse method is it does not take the XmlEnum value into account when trying to parse the string. For example, if the Colors enumeration were defined as follows:

Public Enum Colors
    Invalid
    <Xml.Serialization.XmlEnum("R")> _
    Red
    <Xml.Serialization.XmlEnum("G")> _
    Green
    <Xml.Serialization.XmlEnum("B")> _
    Blue
End Enum

And the following code was executed:

Choice = CType([Enum].Parse(GetType(Colors), "R"), Colors)

A System.ArgumentException("Requested value 'R' was not found.") would be raised to the caller. It would be nice if the Parse method would work for both the ToString and the XmlEnum values, but it only works with the ToString value.

To get around this shortcoming I wrote the following method that will return the correct enumerated value for both the ToString and the XmlEnum values.

Public Shared Function ParseEnum(Of EnumType As Structure)( _
    ByVal value As String, _
    ByVal defaultValue As EnumType) As EnumType

    Dim Result As EnumType
    Dim ValueFound As Boolean = False

    ' Use the standard Parse method
    Try

        ' Attempt to convert string to enumeration value
        Result = (CType([Enum].Parse(GetType(EnumType), value, True), EnumType))
        ValueFound = True

    Catch ex As ArgumentException

        ValueFound = False

    End Try

    ' If that does not work, try the XmlEnum values
    If ValueFound = False Then

        Dim Members() As System.Reflection.FieldInfo
        Dim XmlEnumAttributes() As System.Xml.Serialization.XmlEnumAttribute

        ' Get the list of Enumeration Members
        Members = GetType(EnumType).GetFields()

        For Each Member As System.Reflection.FieldInfo In Members

            ' Only examine Enum Members
            If Member.IsSpecialName = False AndAlso _
                Member.IsLiteral = True Then

                ' Get the XmlEnum Attributes
                XmlEnumAttributes = CType(Member.GetCustomAttributes( _
                    GetType(System.Xml.Serialization.XmlEnumAttribute), True),  _
                   
System.Xml.Serialization.XmlEnumAttribute())

                ' Check the XmlEnum attribute
                If XmlEnumAttributes.Length > 0 AndAlso _
                    String.Compare(XmlEnumAttributes(0).Name, value, True, _
                    System.Globalization.CultureInfo.InvariantCulture) = 0 Then

                    ' Found Value
                    Result = CType(Member.GetValue(Nothing), EnumType)
                    ValueFound = True
                    Exit For

                End If ' If XmlEnumAttributes.Length > 0 AndAlso

            End If ' If Member.IsSpecialName = False AndAlso

        Next Member

    End If ' If ValueFound = False Then

    ' If the value still has not been found, use the default value
    If ValueFound = False Then
        Result = defaultValue
    End If

    ' Return enumeration value
    Return Result

End Function

When this method is used, either the ToString or the XmlEnum value can be used. For example, both of the calls below will result in Choice being set to Colors.Red:

Dim Choice As Colors

Choice = ParseEnum("Red", Colors.Invalid)
Choice = ParseEnum("R", Colors.Invalid)

One possible enhancement that could be made to the ParseEnum function is adding support for parsing comma-separated, XmlEnum values for bit field enumerations (e.g. “R, G” would return the value “Colors.Red Or Colors.Green”). See the FlagsAttribute Class help topic (http://msdn.microsoft.com/en-us/library/system.flagsattribute.aspx https://docs.microsoft.com/en-us/dotnet/api/system.flagsattribute) for more details on bit field enumerations.

Visual Studio 2008/.NET Framework 3.5 Enhancement

Visual Studio 2008/.NET Framework 3.5 adds extension methods, which allow for methods to be added to existing data types without creating new derived types. This allows us to add a TryParse method, like the one shown below, to the enumerations to provide this functionality.

<System.Runtime.CompilerServices.Extension()> _
Public Function TryParse(Of EnumType As Structure)( _ 
    ByVal enumObject As [Enum], _ 
    ByVal value As String, _ 
    ByRef result As EnumType) As Boolean 
 
    Dim ValueFound As Boolean = False 
 
    ' Use the standard Parse method 
    Try 
 
        ' Attempt to convert string to enumeration value 
        result = (CType([Enum].Parse(GetType(EnumType), value, True), EnumType)) 
        ValueFound = True 
 
    Catch ex As ArgumentException 
 
        ValueFound = False 
 
    End Try 
 
    ' If that does not work, try the XmlEnum values 
    If ValueFound = False Then 
 
        Dim Members() As System.Reflection.FieldInfo 
        Dim XmlEnumAttributes() As System.Xml.Serialization.XmlEnumAttribute 
 
        ' Get the list of Enumeration Members 
        Members = GetType(EnumType).GetFields() 
 
        For Each Member As System.Reflection.FieldInfo In Members 
 
            ' Only examine Enum Members 
            If Member.IsSpecialName = False AndAlso _ 
                Member.IsLiteral = True Then 
 
                ' Get the XmlEnum Attributes 
                XmlEnumAttributes = CType(Member.GetCustomAttributes( _ 
                    GetType(System.Xml.Serialization.XmlEnumAttribute), True),  _
                    System.Xml.Serialization.XmlEnumAttribute()) 
 
                ' Check the XmlEnum attribute 
                If XmlEnumAttributes.Length > 0 AndAlso _ 
                    String.Compare(XmlEnumAttributes(0).Name, value, True, _ 
                    System.Globalization.CultureInfo.InvariantCulture) = 0 Then 
 
                    ' Found Value 
                    result = CType(Member.GetValue(Nothing), EnumType) 
                    ValueFound = True 
                    Exit For 
 
                End If ' If XmlEnumAttributes.Length > 0 AndAlso 
 
            End If ' If Member.IsSpecialName = False AndAlso 
 
        Next Member 
 
    End If ' If ValueFound = False Then 
 
    ' Indicate if the value was successfully parsed. 
    Return ValueFound 
 
End Function

I personally like this solution the best. When this method is used, either the ToString or the XmlEnum value can be used. For example, both of the calls below will result in Choice being set to Colors.Red:

Dim Choice As Colors

Choice.TryParse("Red", Choice)
Choice.TryParse("R", Choice)

Ideally the TryParse method would be a Shared (or static) method, but extension methods cannot be Shared (or static).

How to Determine If an Enumeration Is a Bit Field

In .NET you can tell the compiler to treat an enumeration as a bit field by adding a Flags attribute to its declaration. For more information about bit fields in .NET see http://msdn.microsoft.com/en-us/library/system.flagsattribute.aspxhttps://docs.microsoft.com/en-us/dotnet/api/system.flagsattribute. The following function will tell you if an enumeration value is a bit (or flag) field.

Function IsFlagEnum(ByVal value As [Enum]) As Boolean

    If value.GetType().IsDefined( _
        GetType(FlagsAttribute), True) = True Then
        Return True
    Else
        Return False
    End If

End Function

For example, given the following definitions:

<Flags()> _
Public Enum FlagEnum
    None = 0
    ValueA = 1
    ValueB = 2
    ValueC = 4
End Enum

Public Enum RegularEnum
    Invalid = 0
    ValueA = 1
    ValueB = 2
    ValueC = 3
End Enum

IsFlagEnum(FlagEnum.ValueB) will return True, but IsFlagEnum(RegularEnum.ValueB) will return False.