pike.git / lib / modules / Search.pmod / Utils.pmod

version» Context lines:

pike.git/lib/modules/Search.pmod/Utils.pmod:1: + // This file is part of Roxen Search + // Copyright © 2001 Roxen IS. All rights reserved. + // + // $Id: Utils.pmod,v 1.5 2001/06/22 01:28:35 nilsson Exp $ +    public array(string) tokenize_and_normalize( string what )   //! This can be optimized quite significantly when compared to   //! tokenize( normalize( x ) ) in the future, currently it's not all   //! that much faster, but still faster.   {    return Unicode.split_words_and_normalize( lower_case(what) );   }      public array(string) tokenize(string in)   //! Tokenize the input string (Note: You should first call normalize