I have read a file in, and stored it in an array, I now need to Tokenize on a String, that is I want to only split when ")," occur next to each other. Plus I seem to lose a token either side of the delimiter what I am doing wrong

The Input is as below

state(A,B,C,D) :-
  deriv(B,D), deriv(A,C), m_plus(A,C,[]), add(B,A,E,[]),
  m_plus(E,D,[]).
state(A,B,C,D) :-
  deriv(B,D), deriv(A,C), m_plus(A,C,[]), add(B,A,E,[]),
  m_minus(E,D,[]).
state(A,B,C,D) :-
  deriv(B,D), deriv(A,C), m_plus(A,C,[]), add(A,E,B,[]),
  m_plus(E,D,[]).

and I want it to be

state(A,B,C,D) :-
deriv(B,D),
deriv(A,C),
m_plus(A,C,[]),
add(B,A,E,[]),
m_plus(E,D,[]).

etc.

I think this may take more than one pass with a String Tokenizer but need help desperately (before I burst into tears and run away)

Please please please help me  

It would make me very happy  

The String Tokenizer splits up strings at a single given character, and if you specify several characters it splits at any one of them - you want to split at a multicharacter string.  (i.e. the tokenizer uses an OR operation and you want to do an AND)

Possible solutions?

a) Use another string handling class - is this Java2 1.4?  If it is, I would be very tempted to use a regular expression

or

b) "Roll your own" ... here's some code I've cut and pasted from our training notes and modified a bit

Code:

public static void main (String [] args) {      String you_said = "deriv(B,D), deriv(A,C), m_plus(A,C,[]), add(B,A,E,[]),";      int old_posn=0,posn=0;      while (posn>=0)         {         posn = you_said.indexOf("),",old_posn);         String next_word = (posn>0) ?               you_said.substring(old_posn,posn+2):               you_said.substring(old_posn);         System.out.println("Next word: "+next_word);         old_posn = posn + 2;         } } }

and here's what if output:

Code:

[localhost:~/jun03] graham% java Ic Next word: deriv(B,D), Next word:  deriv(A,C), Next word:  m_plus(A,C,[]), Next word:  add(B,A,E,[]), Next word: [localhost:~/jun03] graham%

---