STRING TOKENIZER CLASS

STRING TOKENIZER CLASS

* the processing to text often comprises of parsing a formatted input string.
* Parsing is the division of text into a set of discrete parts or tokens which in a certain sequence can convey a semantic meaning.
* The string tokenizer class provides the facility of parsing the text. It is often called lexer(lexical analyxer) or scanner the String tokenizer class is present in java.util package
* String tokenizer we can specify an input string and a string that contains delimiters.
* To use string tokenizer we can specify an input string and a string that contains delimiter.
* Delimiter are Character that separate tokens. Each character in the delimiter string is considered a valid delimiter.
* The default set of delimiter consist of white space character such as space tab newline etc.
* In genaral space is used the default delimiter.
* The following are the constructor in string tokenizer

     string tokenizer(string str)
     string tokenizer (string str, string delimiter)
     string tokenizer(string str, string delimiter delimiters, Boolean delim as token)
* In all the tree constructor mentioned above str is the string that be tokenized.
* in the first constructor the default delimiter (space) is used.
* In the second and third constructor delimiter is a string that specifies the delimiters.
* In the third constructor if delim a stoken is true then the delimiters are also returned as token when the string is parsed. Otherwise the delimiter are not returned. Delimiter are not went back as tokens by the first two forms.

*The following are the methods defined in string tokenizer.
      int countTokens()
      Boolean hasMoreElements()
      Boolean hasMoreTokens()
      object nextElement()
      String nextToken()
      String nextToken(String delimiters)

*The following program demonstrates a StringTokenizer
    import java,util.StringTokenizer;
    Class STDemo
     {
        stastic string str:"this-is-my-core-java-notes";
        public static void main(String args[])
        String Tokenizer st= new StringTokenizer(str,"-")
        While (st.hasMoreTokens();
        System.out.println(s1);
      }
}
OUTPUT:
   This is my core java notes (printed vertically)