pyre.parsing.Tokenizer.Tokenizer

Package pyre :: Package parsing :: Module Tokenizer :: Class Tokenizer

[show private | hide private]

Type Tokenizer

object --+
         |
        Tokenizer

Method Summary
	`__init__(self, file)`
	`fetch(self, scanner)` Return a token from the input stream
	`locator(self)`
	`unfetch(self, token)` Put a token back into the token stream
Inherited from object
	`__delattr__(...)` x.__delattr__('name') <==> del x.name
	`__getattribute__(...)` x.__getattribute__('name') <==> x.name
	`__hash__(x)` x.__hash__() <==> hash(x)
	`__new__(T, S, ...)` T.__new__(S, ...) -> a new object with type S, a subtype of T
	`__reduce__(...)` helper for pickle
	`__reduce_ex__(...)` helper for pickle
	`__repr__(x)` x.__repr__() <==> repr(x)
	`__setattr__(...)` x.__setattr__('name', value) <==> x.name = value
	`__str__(x)` x.__str__() <==> str(x)

Class Variable Summary
`classobj`	`EndOfFile` = `pyre.parsing.EndOfFile.EndOfFile`
`classobj`	`TokenizationException` = `pyre.parsing.TokenizationException.TokenizationException`

Method Details

Return a token from the input stream

Put a token back into the token stream

Generated by Epydoc 2.1 on Tue May 4 16:58:11 2004