Digg
del.icio.us
Newsvine
Facebook
Google
LinkedIn
MySpace
Reddit
Slashdot
StumbleUpon
Technorati
Twitter
Windows Live
YahooBuzz
FriendFeedA Simple CString Tokenizer
Posted
by Richard Case
on March 3rd, 1999
As the name suggests, this is a simple class to extract tokens from a CSting. I wrote this class becuase during the course of my final year project at University I needed a simple way to extract 'tokens' from a CString. This is what I came up with. Its probably not the best or most effeicent way to accomplish the task - but it works which in the main thing.
To use the CToken class:
-First include the CToken header file in your porgram:
#include "Token.h"
-Then you are free to create and use an instance of CToken. For example:
CString str = "A B C D"
CString newTok;
CToken tok(str);
tok.SetToken(" ");
while(tok.MoreTokens())
{
newTok = tok.GetToken()
}
Improvements:If people can suggest any improvements or there bugs please drop me an email
Comments
problem: blank tokens are trimmed!
Posted by pamar on 11/15/2007 11:35amWhit this string: "A|B|C| " and | separator, last token must be " " but it is trimmed.
Replyi rewrote it : shorter & better
Posted by Legacy on 03/10/2003 12:00amOriginally posted by: joeri
just for those in need, i thought this delimiter wasn't working well, so i rewrote it:
class CToken
{
public:
virtual CString GetNextToken(); // Returns the next token
virtual BOOL MoreTokens(); // Do we have any more tokens to retrieve
int m_nAmount;
int m_nCurrent;
CToken(CString str,CString token=" ");
virtual ~CToken();
protected:
CString m_strToToken; // The original string to tokenize
CString m_strLeft; // The string we have left after taking tokens out
CString m_strToken; // What is the value to tokenize on?
private:
int GetAmount(CString str);
};
CToken::CToken(CString str, CString token)
{
str.TrimLeft();
str.TrimRight();
m_strToToken = str;
m_strLeft = str;
m_strToken = token;
m_nAmount = GetAmount(str);
m_nCurrent = 0;
}
CToken::~CToken()
{
}
int CToken::GetAmount(CString str)
{
int count = 0;
int pos, len;
len = str.GetLength();
pos = str.Find(m_strToken,0);
while (pos != -1)
{
count++;
pos++;
str = str.Right(len - pos);
str.TrimLeft();
len = str.GetLength();
pos = str.Find(m_strToken,0);
}
if (!str.IsEmpty()) count++;
return count;
}
BOOL CToken::MoreTokens()
{
if (m_nCurrent < m_nAmount)
return TRUE;
else return FALSE;
}
ReplyCString CToken::GetNextToken()
{
if(!MoreTokens())
return "";
CString ret;
int pos, len;
len = m_strLeft.GetLength();
pos = m_strLeft.Find(m_strToken,0);
if (pos != -1)
{
ret = m_strLeft.Left(pos);
pos++;
m_strLeft = m_strLeft.Right(len - pos);
m_strLeft.TrimLeft();
}
else ret = m_strLeft;
m_nCurrent++;
return ret;}
lexiacal analyzing
Posted by Legacy on 01/12/2003 12:00amOriginally posted by: alan
ReplyNice class
Posted by Legacy on 07/31/2002 12:00amOriginally posted by: Annette
Hi,
I just wanted to say that even if it a simple class, I thought it was very nice and helpful. I like working with CString instead of "hacking" along with all these char * and other string types.
So, thanks!!
Annette Skaar
Replypointer to a pointer in a string class
Posted by Legacy on 05/06/2001 12:00amOriginally posted by: Cris Moore
I'm working on a project. My data member in the class is char **list. The parameter I'm being passed is char *alist. How do I convert?
ReplyI need help!!
Posted by Legacy on 04/18/2001 12:00amOriginally posted by: John
I am a beginner in visual C++ programming.I have to do a program that read a multi-command file . Each command have a fixed number of parameters .I have to make a string analysis and based on that analysis the program should fill some small records like matrix (two dimension vectors).
What would be the best way to do this?(in special the string analysis).
ReplyI wouldn't bother using this class....
Posted by Legacy on 09/06/2000 12:00amOriginally posted by: Paul Wendt
Hey,
ReplyThis class doesn't really offer anything above strtok so I'd personally stick with strtok. It's not like there is a huge learning curve with learning strtok either :)
--paul wendt
More easy way is to use AfxExtractSubString() function :))
Posted by Legacy on 02/11/2000 12:00amOriginally posted by: German Madyarov
Replyone single token not recognized?
Posted by Legacy on 11/09/1999 12:00amOriginally posted by: Hasch
When I have only one token in the string, say "B" without any delimiters, I would expect to get "B" on calling GetNextToken. But MoreTokens() returns "false".
ReplyI think, the "Check to see if there anymore tokens" from GetNextToken should also be done in the constructor. Then the flag "m_bFinalToken" would be set correctly.
What do you think of it?
Not compatible with strtok()
Posted by Legacy on 05/04/1999 12:00amOriginally posted by: Anders M Eriksson
ReplyLoading, Please Wait ...