I've been playing around with interim builds of XStreamDB 3.1 Beta, and it's coming along REALLY nicely. It's pretty cool when the president of the company is the guy running the beta program. Jim's been awesome, feeding tips and pointers to the new stuff.
They just added/enhanced scoring of fulltext queries, so we can have results sorted by descending relevancy to a query. It's freakin' fast, too. I've added 3734 XML records from CAREO to XStreamDB (which is also running on the CAREO server - a G4/500 Desktop box, not fast by any stretch of the imagination - to keep the playing field level). Scored and sorted queries are returning results in well under a second. Great stuff.
I've even got it doing the processing to pull out just a few elements (title, description, etc...) rather than the whole LOM. It doesn't seem to take any more time to just pull the whole LOM out, either, which is cool. I prefer the mini proxy results for now, because they're easier to read in the results listing, but if the EOAdaptor needs the full record, that's trivial.
The cool thing about the relevancy ranking is that it generates a float value from 0.000000 (completely irrelevant) to 1.000000 (completely relevant). Lots of room in there for subtle variation in relevancy.
Here's the query I'm running right now:
FOR $record IN (Root("Apollo:Metadata"))
LET $score := SCORE $record USING [//* CONTAINS "earth image"]
RETURN
LET $title := $record//*:general/*:title/*:langstring/text()
LET $location := $record//*:technical/*:location/text()
LET $format := $record//*:technical/*:format/text()
LET $description := $record//*:general/*:description/*:langstring/text()
LET $docid := GetDocId( $record )
ORDER BY $score
RETURN
Which returns stuff like this:
Compare that to the smaller statement to just pull the whole LOM:
FOR $record IN (Root("Apollo:Metadata"))
LET $score := SCORE $record USING [//* CONTAINS "earth image"]
ORDER BY $score
RETURN $record
Which returns the whole freakin' LOM document.