Background and motivation
With a majority of text files nowadays being UTF-8 encoded, it seems it might be beneficial to have a means to compare two ReadOnlySpan<byte> instances (UTF-8 encoded), using culture-aware semantics. This would be consistent with the push towards UTF-8 as a first-class citizen.
I was not able to find a pre-existing mechanism for doing so. If one does exist, and I simply missed it, my apologies.
The closest I did find was the following:
int MemoryExtensions.CompareTo(this ReadOnlySpan<char> span, ReadOnlySpan<char> other, StringComparison comparisonType);
However, this would require first transcoding to UTF-16. In addition to the obvious transcoding overhead, this also potentially explodes memory resource utilization for operations (e.g. sorting) where large numbers of comparisons might be required.
The current options seem to be one of the following: 1) incur transcoding overhead 2) abandon multi-cultural ordering (use ordinal) or 3) abandon platform independence (and P/Invoke something like ucol_strcollUTF8?).
API Proposal
I propose the addition of a new extension method for ReadOnlySpan<byte> similar to the pre-existing one for ReadOnlySpan<char> (perhaps wrapping ucol_strcollUTF8?).
namespace System
{
public static partial class MemoryExtensions
{
public static int CompareTo(this ReadOnlySpan<byte> span, ReadOnlySpan<byte> other, StringComparison comparisonType);
}
}While I think this is consistent with existing method signatures, I am honestly open to any API surface that solves this same problem. I use this only as an example and do not seek to promote any particular implementation.
API Usage
ReadOnlySpan<byte> a = "abc"u8;
ReadOnlySpan<byte> b = "def"u8;
int comparison = a.CompareTo(b, StringComparison.CurrentCulture);
Alternative Designs
No response
Risks
No response
Background and motivation
With a majority of text files nowadays being UTF-8 encoded, it seems it might be beneficial to have a means to compare two
ReadOnlySpan<byte>instances (UTF-8 encoded), using culture-aware semantics. This would be consistent with the push towards UTF-8 as a first-class citizen.I was not able to find a pre-existing mechanism for doing so. If one does exist, and I simply missed it, my apologies.
The closest I did find was the following:
However, this would require first transcoding to UTF-16. In addition to the obvious transcoding overhead, this also potentially explodes memory resource utilization for operations (e.g. sorting) where large numbers of comparisons might be required.
The current options seem to be one of the following: 1) incur transcoding overhead 2) abandon multi-cultural ordering (use ordinal) or 3) abandon platform independence (and P/Invoke something like ucol_strcollUTF8?).
API Proposal
I propose the addition of a new extension method for
ReadOnlySpan<byte>similar to the pre-existing one forReadOnlySpan<char>(perhaps wrapping ucol_strcollUTF8?).While I think this is consistent with existing method signatures, I am honestly open to any API surface that solves this same problem. I use this only as an example and do not seek to promote any particular implementation.
API Usage
Alternative Designs
No response
Risks
No response