|
Server : Apache/2.2.2 (Fedora) System : Linux App1.pathumtani.go.th 2.6.20-1.2320.fc5smp #1 SMP Tue Jun 12 19:40:16 EDT 2007 i686 User : apache ( 48) PHP Version : 5.2.9 Disable Function : NONE Directory : /proc/self/root/usr/lib/python2.4/site-packages/mx/TextTools/ |
Upload File : |
mò
H<c
@ sB d Z d k Z d k Z d k Td k l Z d k Td k Te i d „ Z d d „ Z d „ Z
d d „ Z d e i d
„ Z
e i d „ Z d d „ Z d
„ Z d e e e e e e i e e i d „ Z d e e e e e i e d „ Z e i e e i d „ Z e e e e e i d „ Z d e e e d „ Z d e e d „ Z d e e d „ Z d e e e d „ Z d „ Z d „ Z! d „ Z" d e e# e$ d „ Z% d e e& e' e( e) ƒ d „ Z* e e+ d d f e e+ d d f d e, e- e' d d ƒ d d! f e e. e/ d e0 f d" e1 e- d d d# f f Z2 e e2 d$ „ Z3 e3 Z4 e e+ d d f e e+ d d f d e, e5 e' d d ƒ d d! f e e. e/ d e0 f d" e1 e5 d d d# f f Z6 e6 d% „ Z7 e e, e8 d f d& e, e- e# d d f e e. e/ d e0 f f Z9 e& e8 d' „ Z: d k; Z; d( f d) „ ƒ YZ< d* d+ „ Z= e> d, j o e= ƒ n d S(- s< mxTextTools - A tools package for fast text processing.
Copyright (c) 2000, Marc-Andre Lemburg; mailto:mal@lemburg.com
Copyright (c) 2000-2001, eGenix.com Software GmbH; mailto:info@egenix.com
See the documentation for further information on copyrights,
or contact the author. All Rights Reserved.
N( t *( s __version__c C s | | } d } d }
| d \ } } } t | ƒ d j o | d } n t | ƒ d j o | d }
n t | d ƒ \ } } t | } t
| ƒ | j o | d j o
d } nI | d j o
d } n2 t | ƒ } t | ƒ d
j o | d
d } n d t | ƒ d
| | f | |
f S( s7 Returns a pp-formatted tag table entry as string
i i i i i t Tablet SubTables <table>s Here/Toi s ...s# %-15.15s : %-30s : jne=%+i : je=%+is %-.15s : %sN( R R ( t tablet it et jnet jet tt ct mt lent divmodt flagst cmdt id2cmdt typet TupleTypet Nonet repr( R R R R R R R
R
R R R ( ( t: /usr/lib/python2.4/site-packages/mx/TextTools/TextTools.pyt format_entry s&
iÿÿÿÿc C s| g } x_ t t | ƒ ƒ D]K } | | j o | i d t | | ƒ ƒ q | i d t | | ƒ ƒ q Wt i | d ƒ d S( s; Returns a pp-formatted version of the tag table as string s --> s s
N(
t lt rangeR R t jR t appendR t stringt join( R R R R ( ( R t format_table6 s
c C s t | ƒ GHd S( s Print the tag table
N( R R ( R ( ( R t print_tagtableB s i c C sõ xî | D]æ \ } } } } t | ƒ } t | ƒ d j o | d d } n t | | | !ƒ } t | ƒ d j o | d d } n | d j o' d | d G| Gd G| G| | f GHq d | d G| Gd G| G| | f GHt | | | d ƒ q Wd S( sC Print the taglist tags for text using the given indent level
i s ...i< t s |s : i N(
t tagst tagR t rt subtagsR t tagnameR t textt targetR t indentt
print_tags( R$ R R& R% R R R" R! R# ( ( R R' H s
'#c C sø xñ | D]é } t | ƒ | j oT t | ƒ } t | ƒ d j o | d d } n d | d G| Gd t | ƒ GHq | d } | d d !\ } } t | | | !ƒ } t | ƒ d j o | d d } n d | d G| Gd | | G| | f GHq Wd S(
s< Print the joinlist joins using the given indent level
i( s ...R s |s (len = %i)i i i N(
t joinsR R t
StringTypeR R$ R R& R R! ( R( R& R) R$ R R R! ( ( R t print_joinlistY s #
c C s| d g t | ƒ } xb t t | ƒ ƒ D]N } | | } t | ƒ | j o | | | <q& | d | d | d !| | <q& W| S( sÓ Return a normalized joinlist.
All tuples in the joinlist are turned into real strings. The
resulting list is a equivalent copy of the joinlist only
consisting of strings.
t i i i N( R t jlistR R R t entryR R) ( R, R) R R R- ( ( R t normlistm s
!c C sW h } xJ | D]B } | | } | i | ƒ o | | i | ƒ q
| g | | <q
W| S( N( t dR t wt indexR t has_keyR ( R R1 R R0 R/ ( ( R t _lookup_dict„ s
c C s4 g } t | ƒ } | i ƒ } t | ƒ d j oB | i d t t i
| i ƒ d ƒ f ƒ | i d t d f ƒ n xž | i ƒ D] \ } } | i d ƒ t | ƒ d } x. | D]& } | i d t | d d t f ƒ q° W| i d t t f ƒ d t | t | ƒ | f | | <q€ W| i d t t f ƒ t | ƒ S( s9 Creates a lookup table that matches the words in l
i R+ iÿÿÿÿi N( R R3 R R/ t keysR R R t IsInR R t Skipt itemsR t groupR R0 t Wordt MatchOkt Failt Heret Ist tuple( R R8 R/ R4 R R R R0 ( ( R t word_in_list s$ (
$$c C s÷ |
| ƒ | j o | | ƒ } n | } | i } | d j oD | d j o' t | ƒ d j o | | | | ƒ Sn t | ƒ } n | t
| d f | t t t | ƒ d d f | t t f f } | | | | | ƒ \ } } }
| p | Sn | | ƒ S( sì A fast replacement for string.replace.
what can be given as string or search object.
This function is a good example for the AppendTagobj-flag usage
(the taglist can be used directly as joinlist).
i i iÿÿÿÿN( R t whatR) t SearchObjectt sot matcht stopR t startR t string_replaceR$ t witht
sWordStartR6 t AppendTagobjt Movet ToEOFR R t foundt taglistt lastR ( R$ R@ RG RE RD RA R t joinlistR RF R R) RM RN RB R RL ( ( R t replaceª s
= c
C s˜ | d j o t | ƒ } n t | ƒ | j o | | ƒ } n | t | d d f f } | | | | | ƒ \ } }
}
| p | Sn | | | |
ƒ ƒ S( s Analogon to string.replace; returns a string with all occurences
of what in text[start:stop] replaced by with
- uses a one entry tag-table and a Boyer-Moore-Search-object
- what can be a string or a BMS/FS search object
- it's faster than string.replace in those cases, where
the what-string gets long and/or many replacements are found;
faster meaning from a few percent up to many times as fast
- start and stop define the slice of text to work in
- stop defaults to len(text)
i i N( RD R R R$ R R@ R) t BMSRG t sFindWordR R RE RL RM RN R RO ( R$ R@ RG RE RD R RO R R) RQ RN R RL RM ( ( R t _replace2Î s
c C s t | ƒ | j o | | ƒ } n | i | ƒ } | p | Sn g } d } x2 | D]* \ }
} | i | | |
!| ƒ | } qT W| i | | ƒ | | d ƒ S( Ni R+ ( R R@ R) t FSt findallR$ t slicesR t xt leftt rightR RG R ( R$ R@ RG R RT R) RV R RY RW RX ( ( R t _replace3ç s
c C s™ t | ƒ | j o | | ƒ } n | i | ƒ } | p | Sn d g t | ƒ }
x/ t
t | ƒ ƒ D] } | f | | |
| <qg W| | | |
ƒ ƒ S( N( R R@ R) RT RU R$ RV R R t replR R RG R RO ( R$ R@ RG R RO R RT R) RV R R[ ( ( R t _replace4ù s c C sD | d j o | | | | | | ƒ ƒ Sn | | | | | ƒ ƒ Sd S( s¨ Apply multiple replacement to a text at once.
replacements must be list of tuples (replacement, left,
right). It is used to replace the slice text[left:right] with
the string replacement.
Note that the replacements do not affect one another. Indices
always refer to the original text string.
Replacements must not overlap. Otherwise a ValueError is
raised.
N( RD R R RO R$ t replacementsRE ( R$ R] RE RD R RO ( ( R t multireplace s
c C sD | d j o | | ƒ i | | | ƒ Sn | | ƒ i | | ƒ Sd S( s A faster replacement for string.find().
Uses a search object for the task. Returns the position of the
first occurance of what in text[start:stop]. stop defaults to
len(text). Returns -1 in case no occurance was found.
N( RD R RA R@ t findR$ RE ( R$ R@ RE RD RA ( ( R R_ s
c C sD | d j o | | ƒ i | | | ƒ Sn | | ƒ i | | ƒ Sd S( s Find all occurances of what in text.
Uses a search object for the task. Returns a list of slice
tuples (l,r) marking the all occurances in
text[start:stop]. stop defaults to len(text). Returns an
empty list in case no occurance was found.
N( RD R RA R@ RU R$ RE ( R$ R@ RE RD RA ( ( R RU . s
c
C s | o | | | ƒ } n
| | ƒ } | o | i | | | ƒ }
n | i | | ƒ }
d } g } | i } x+ |
D]# \ } } | | | | !ƒ | } qt W| | | ƒ | S( s” A faster replacement for string.split().
Uses a search object for the task. Returns the result of
cutting the text[start:stop] string into snippets at every sep
occurance in form of a list of substrings. translate is passed
to the search object as translation string.
XXX convert to a C function... or even better, add as method
to search objects.
i N( t translateRA t sepRB RD RU R$ RE t cutsR t listR RX RY (
R$ Ra RE RD R` RA RY R Rc RB Rb R RX ( ( R t split? s
c C sd x] | D]U \ } } } } | t | ƒ } | | | !| | <| o t
| | | d | ƒ q q Wd S( Nt .( RM t oR R! t st prefixt strt pfxR$ t dictt _tagdict( R$ Rk Rh RM R Rf Rg R! Rj ( ( R Rl ` s c G s¬ t t | f | ƒ \ } } } | p | d | f Sn h } t } xY | D]Q \ } } } } t | ƒ }
| | | !| |
<| o | | t |
d | ƒ qJ qJ W| | | f S( sÝ Tag a text just like the function tag() and then convert
its output into a dictionary where the tagobjects reference
their respective strings
- this function emulates the interface of tag()
- in contrast to tag() this funtion *does* make copies
of the found stings
- returns a tuple (rc,tagdict,next) with the same meaning
of rc and next as tag(); tagdict is the new dictionary -
None in case rc is 0
Re N( t applyR R$ t argst rcRM t nextR R/ Rl t tagdictRf R R! Rg Ri Rj Rk ( R$ Rn R/ Rq R Rf Rp Rg Ro R! Rj RM ( ( R Rq h s c C s
t | d ƒ S( sB Return a set with all characters *except* the ones in chars.
i N( t sett chars( Rs ( ( R t invset€ s c C s< | d j o t | ƒ } n | | | | | ƒ } | d j S( s Return 1 iff text[start:stop] only contains whitespace
characters (as defined in Constants/Sets.py), 0 otherwise.
i N( RD R R R$ t setfindt
nonwhitespaceRE R ( R$ RE RD Rv Ru R ( ( R t
is_whitespace† s
R c C s | | | | ƒ | ƒ S( sá Eliminates newline characters and compresses whitespace
characters into one space.
The result is a one line text string. Tim Peters will like
this function called with '-' separator ;-)
N( R t setsplitR$ t collapse_sett separator( R$ Rz R Rx Ry ( ( R t collapse’ s
s
i s
t lines
iþÿÿÿs
empty lineiüÿÿÿc C s | | | ƒ d S( s Split text into a list of single lines.
The following combinations are considered to be line-ends:
'
', '
', '
'; they may be used in any combination. The
line-end indicators are removed from the strings prior to
adding them to the list.
This function allows dealing with text files from Macs, PCs
and Unix origins in a portable way.
i N( R R$ t linesplit_table( R$ R R} ( ( R t
splitlines§ s c C s t t | | ƒ d ƒ S( s Returns the number of lines in text.
Line ends are treated just like for splitlines() in a
portable way.
i N( R R R$ t linecount_table( R$ R ( ( R t
countlinesàs t wordc C s
| | | ƒ S( sª Split text into a list of single words.
Words are separated by whitespace. The whitespace is stripped
before adding the words to the list.
N( Rx R$ t whitespace_set( R$ Rx R‚ ( ( R t
splitwordsÔ s t _timerc B sw t Z d Z d Z d Z e i e i d „ Z e i e i d „ Z e i e i d „ Z e i e i d „ Z
d „ Z RS( s timer class with a quite obvious interface
- .start() starts a fairly accurate CPU-time timer plus an
absolute timer
- .stop() stops the timer and returns a tuple: the CPU-time in seconds
and the absolute time elapsed since .start() was called
i c C s | ƒ | _ | ƒ | _ d S( N( t timet selft atimet clockt utime( R† Rˆ R… ( ( R RE ò s c C s6 | ƒ | i | _ | ƒ | i | _ | i | i f S( N( Rˆ R† R‰ R… R‡ ( R† Rˆ R… ( ( R RD ÷ s c C s- | ƒ | i | _ | ƒ | i | _ | i S( N( Rˆ R† R‰ R… R‡ ( R† Rˆ R… ( ( R t usertimeý s c C s- | ƒ | i | _ | ƒ | i | _ | i S( N( Rˆ R† R‰ R… R‡ ( R† Rˆ R… ( ( R t abstime s c C s d | i | i f S( Ns %0.2fu %0.2fa sec.( R† R‰ R‡ ( R† ( ( R t __str__ s ( t __name__t
__module__t __doc__R‰ R‡ R… Rˆ RE RD RŠ R‹ RŒ ( ( ( R R„ æ s s mxTextTools/mxTextTools.cc C s@ d „ } t | ƒ i ƒ } d k } t ƒ } d t | ƒ GHd S( Nc C sú Hx£ t t | ƒ ƒ D] } | | | | j o Pq q Wd t | ƒ t | ƒ f GHt | ƒ t | ƒ j o d t | t | ƒ ƒ GHn d t | t | ƒ ƒ GHHd Sd | GH| | d | !d t | | ƒ t | | ƒ f | | d | d !GHHd S( Ns Length mismatch: orig=%i new=%is Missing chars:s
Excess chars:s Mismatch at offset %i:id s <- %s != %s ->i ( R R t origR t newR ( R R‘ R ( ( R t mismatch s Es Working on a %i byte string( R’ t opent filet readR$ R R„ R R ( R” R R$ R’ R ( ( R t _bench
s . ) $t __main__(? R R t typest mxTextToolst __version__t Constants.TagTablest Constants.SetsR R R R R' R) R* R. R3 R? R RQ R RO R RP R RS RT RZ R\ R^ R_ RU Rd Rl Rq Rt t nonwhitespace_setRu Rw Rx Rr t newlinet
whitespaceR{ R= t AllInSett AppendMatcht EOFR< R: R6 t _linesplit_tableR~ t linesplitRI t _linecount_tableR€ R‚ t _wordsplit_tableRƒ R… R„ R– R (" R3 Rl R RP R¥ R? R{ Rq R„ R R£ R_ R– Rd Rƒ Rš R' R R R. R¦ Rw R€ R¤ R˜ Rt RU R^ RS RZ R~ R\ R… R* ( ( R t ? sN
*$!!
kk = ' ÿ