Safe Haskell | None |
---|---|
Language | Haskell2010 |
Synopsis
- buildTokenPartitions :: (Hashable uid, Eq uid) => [Entry Text uid] -> HashMap Token (HashSet uid)
- getSearchPartition :: (Hashable uid, Eq uid) => Text -> HashMap Token (HashSet uid) -> HashSet uid
- wordTokenize :: Text -> [Token]
- toTokenizedTuple :: (Hashable uid, Eq uid) => Entry Text uid -> ([Token], uid)
- type Token = Text
- newtype Entry name uid = Entry (name, uid)
- entryName :: Entry name uid -> name
- entryUID :: Entry name uid -> uid
- first :: Bifunctor p => (a -> b) -> p a c -> p b c
Documentation
:: (Hashable uid, Eq uid) | |
=> [Entry Text uid] | List of entries |
-> HashMap Token (HashSet uid) | A map of Token -> [uids] |
Given the list of entries to be held by QuickSearch, return a HashMap keyed on tokens from the strings in the entries, where the associated HashMap value is the list of uids of entries containing the token.
:: (Hashable uid, Eq uid) | |
=> Text | Target string |
-> HashMap Token (HashSet uid) | HashMap associating tokens with sets of uids |
-> HashSet uid | The union of sets of associated uids. |
Given a target string and a Token HashMap, return the union of sets of uids associated with the tokens in the target string
Turn a Data.Text.Text string into a list of casefolded tokens. Turns most non-Alphanum into spaces and deletes all periods and apostrophes.
>>>
wordTokenize ("Jane Smith-Walker, M.D."::T.Text)
["jane", "smith", "walker", "md"]
toTokenizedTuple :: (Hashable uid, Eq uid) => Entry Text uid -> ([Token], uid) Source #
Convert an Entry T.Text uid to a tuple of ([wordTokenize name], uid)
>>>
toTokenizedTuple ("Jane Smith-Walker, M.D.", 1)::Entry
(["jane", "smith", "walker", "md"], 1)