首页 > 代码库 > 【译】itertools
【译】itertools
1、Itertools模块迭代器的种类
1.1 无限迭代器:
迭代器 | 参数 | 结果 | 示例 |
count() | start, [step] | start, start+step, start+2*step, ... | count(10) --> 10 11 12 13 14 ... |
cycle() | p | p0, p1, ... plast, p0, p1, ... | cycle(‘ABCD‘) --> A B C D A B C D ... |
repeat() | elem [,n] | elem, elem, elem, ... endlessly or up to n times | repeat(10, 3) --> 10 10 10 |
1.2 终止于最短输入序列的迭代器:
迭代器 | 参数 | 结果 | 示例 |
accumulate() | p [,func] | p0, p0+p1, p0+p1+p2, ... | accumulate([1,2,3,4,5]) --> 1 3 6 10 15 |
chain() | p, q, ... | p0, p1, ... plast, q0, q1, ... | chain(‘ABC‘, ‘DEF‘) --> A B C D E F |
chain.from_iterable() | iterable | p0, p1, ... plast, q0, q1, ... | chain.from_iterable([‘ABC‘, ‘DEF‘]) --> A B C D E F |
compress() | data, selectors | (d[0] if s[0]), (d[1] if s[1]), ... | compress(‘ABCDEF‘, [1,0,1,0,1,1]) --> A C E F |
dropwhile() | pred, seq | seq[n], seq[n+1], starting when pred fails | dropwhile(lambda x: x<5, [1,4,6,4,1]) --> 6 4 1 |
filterfalse() | pred, seq | elements of seq where pred(elem) is false | filterfalse(lambda x: x%2, range(10)) --> 0 2 4 6 8 |
groupby() | iterable[, keyfunc] | sub-iterators grouped by value of keyfunc(v) | |
islice() | seq, [start,] stop [, step] | elements from seq[start:stop:step] | islice(‘ABCDEFG‘, 2, None) --> C D E F G |
starmap() | func, seq | func(*seq[0]), func(*seq[1]), ... | starmap(pow, [(2,5), (3,2), (10,3)]) --> 32 9 1000 |
takewhile() | pred, seq | seq[0], seq[1], until pred fails | takewhile(lambda x: x<5, [1,4,6,4,1]) --> 1 4 |
tee() | it, n | it1, it2, ... itn splits one iterator into n | |
zip_longest() | p, q, ... | (p[0], q[0]), (p[1], q[1]), ... | zip_longest(‘ABCD‘, ‘xy‘, fillvalue=http://www.mamicode.com/‘-‘) --> Ax By C- D- |
1.3 组合产生器
迭代器 | 参数 | 结果 |
product() | p, q, ...[repeat=1] | 笛卡尔乘积,等价于for循环嵌套(乘法原理) |
permutations() | p[, r] | r长度元组,所有可能的排序,没有重复的元素(排列) |
combinations() | p, r | r长度元组,按排序顺序,没有重复元素(组合) |
combinations_with_replacement() | p, r | r长度元组,按排序顺序,存在重复元素 |
product(‘ABCD‘, repeat=2) | AA AB AC AD BA BB BC BD CA CB CC CD DA DB DC DD | |
permutations(‘ABCD‘, 2) | AB AC AD BA BC BD CA CB CD DA DB DC | |
combinations(‘ABCD‘, 2) | AB AC AD BC BD CD | |
combinations_with_replacement(‘ABCD‘, 2) | AA AB AC AD BB BC BD CC CD DD |
2、
repeat
(object[, times])
创建一个迭代器,它重复返回object对象,无穷尽地运行,除非指定了times参数。用作map()的参数,将不变参数映射到被调用函数。同时,用zip()来创建元组记录的不变部分。
def repeat(object, times=None): # repeat(10, 3) --> 10 10 10 if times is None: while True: yield object else: for i in range(times): yield object
cycle
(iterable)
创建一个迭代器,它返回可迭代对象中的元素,并且保存每个可迭代对象中元素的副本,当可迭代对象中的元素被耗尽时,返回保存在副本中的元素。无穷无尽地重复这一行为。近似等价于:
def cycle(iterable): # cycle(‘ABCD‘) --> A B C D A B C D A B C D ... saved = [] for element in iterable: yield element saved.append(element) while saved: for element in saved:
count
(start=0, step=1)
创建一个迭代器,它返回以start开始的均匀间隔的值。通常用作map()参数产生连续性的数据点。另外,用zip()来添加序列号,近似等价于:
def count(start=0, step=1): # count(10) --> 10 11 12 13 14 ... # count(2.5, 0.5) -> 2.5 3.0 3.5 ... n = start while True: yield n n += step
compress
(data, selectors)
创建一个迭代器,它过滤data的元素,返回仅当selecors为True时相应的data中的元素,当data或selectors可迭代对象中的元素被耗尽时停止。近似等价于:
def compress(data, selectors): # compress(‘ABCDEF‘, [1,0,1,0,1,1]) --> A C E F return (d for d, s in zip(data, selectors) if s)
dropwhile
(predicate, iterable)
创建一个迭代器,只要predicate为True就从可迭代对象中移除元素;然后返回每个元素。请注意,迭代器不产生任何输出,直到predicate第一次变成False,所以它可能有很长的启动时间。近似等价于:
def dropwhile(predicate, iterable): # dropwhile(lambda x: x<5, [1,4,6,4,1]) --> 6 4 1 iterable = iter(iterable) for x in iterable: if not predicate(x): yield x break for x in iterable: yield x
takewhile
(predicate, iterable)
创建一个迭代器,只要predicate为True就返回可迭代对象中的元素。近似等价于:
def takewhile(predicate, iterable): # takewhile(lambda x: x<5, [1,4,6,4,1]) --> 1 4 for x in iterable: if predicate(x): yield x else: break
tee(iterable, n=2)
从单个可迭代对象中返回n个独立的迭代器,近似等价于:
def tee(iterable, n=2): it = iter(iterable) deques = [collections.deque() for i in range(n)] def gen(mydeque): while True: if not mydeque: # when the local deque is empty try: newval = next(it) # fetch a new value and except StopIteration: return for d in deques: # load it to all the deques d.append(newval) yield mydeque.popleft() return tuple(gen(d) for d in deques)
filterfalse
(predicate, iterable) --->filter
创建一个迭代器,它过滤可迭代对象中的元素,返回仅当prediccate为False的元素,如果predicate为None,返回条目为False的元素。近似等价于:
def filterfalse(predicate, iterable): # filterfalse(lambda x: x%2, range(10)) --> 0 2 4 6 8 if predicate is None: predicate = bool for x in iterable: if not predicate(x): yield x
starmap
(function, iterable) --->map
创建一个迭代器,它使用从可迭代对象中获取的参数计算函数。当参数已经从单个可迭代对象(数据已经被预压缩)中分组到元组中时,而不是使用map()。map()和starmap()之间的区别与函数(a,b)和函数(* c)之间的区别相对应。 近似等价于:
def starmap(function, iterable): # starmap(pow, [(2,5), (3,2), (10,3)]) --> 32 9 1000 for args in iterable: yield function(*args)
zip_longest(*iterables, fillvalue=http://www.mamicode.com/None) --->zip
创建一个迭代器,它聚合每个可迭代对象的元素,如果可迭代对象长度不均匀,那么缺失值将填充为fillvalue。迭代继续直到最长的可迭代对象被耗尽,近似等价于:
class ZipExhausted(Exception): pass def zip_longest(*args, **kwds): # zip_longest(‘ABCD‘, ‘xy‘, fillvalue=http://www.mamicode.com/‘-‘) --> Ax By C- D- fillvalue = http://www.mamicode.com/kwds.get(‘fillvalue‘) counter = len(args) - 1 def sentinel(): nonlocal counter if not counter: raise ZipExhausted counter -= 1 yield fillvalue fillers = repeat(fillvalue) iterators = [chain(it, sentinel(), fillers) for it in args] try: while iterators: yield tuple(map(next, iterators)) except ZipExhausted: pass
如果其中一个可迭代对象可能是无限的,那么zip_longest()函数应该使用限制调用次数的东西(例如islice()或takewhile())来包装。如果未指定,则fillvalue默认为None。
3、Itertools模块的配方
def take(n, iterable): "Return first n items of the iterable as a list" return list(islice(iterable, n)) def tabulate(function, start=0): "Return function(0), function(1), ..." return map(function, count(start)) def tail(n, iterable): "Return an iterator over the last n items" # tail(3, ‘ABCDEFG‘) --> E F G return iter(collections.deque(iterable, maxlen=n)) def consume(iterator, n): "Advance the iterator n-steps ahead. If n is none, consume entirely." # Use functions that consume iterators at C speed. if n is None: # feed the entire iterator into a zero-length deque collections.deque(iterator, maxlen=0) else: # advance to the empty slice starting at position n next(islice(iterator, n, n), None) def nth(iterable, n, default=None): "Returns the nth item or a default value" return next(islice(iterable, n, None), default) def all_equal(iterable): "Returns True if all the elements are equal to each other" g = groupby(iterable) return next(g, True) and not next(g, False) def quantify(iterable, pred=bool): "Count how many times the predicate is true" return sum(map(pred, iterable)) def padnone(iterable): """Returns the sequence elements and then returns None indefinitely. Useful for emulating the behavior of the built-in map() function. """ return chain(iterable, repeat(None)) def ncycles(iterable, n): "Returns the sequence elements n times" return chain.from_iterable(repeat(tuple(iterable), n)) def dotproduct(vec1, vec2): return sum(map(operator.mul, vec1, vec2)) def flatten(listOfLists): "Flatten one level of nesting" return chain.from_iterable(listOfLists) def repeatfunc(func, times=None, *args): """Repeat calls to func with specified arguments. Example: repeatfunc(random.random) """ if times is None: return starmap(func, repeat(args)) return starmap(func, repeat(args, times)) def pairwise(iterable): "s -> (s0,s1), (s1,s2), (s2, s3), ..." a, b = tee(iterable) next(b, None) return zip(a, b) def grouper(iterable, n, fillvalue=http://www.mamicode.com/None): "Collect data into fixed-length chunks or blocks" # grouper(‘ABCDEFG‘, 3, ‘x‘) --> ABC DEF Gxx" args = [iter(iterable)] * n return zip_longest(*args, fillvalue=http://www.mamicode.com/fillvalue) def roundrobin(*iterables): "roundrobin(‘ABC‘, ‘D‘, ‘EF‘) --> A D E B F C" # Recipe credited to George Sakkis pending = len(iterables) nexts = cycle(iter(it).__next__ for it in iterables) while pending: try: for next in nexts: yield next() except StopIteration: pending -= 1 nexts = cycle(islice(nexts, pending)) def partition(pred, iterable): ‘Use a predicate to partition entries into false entries and true entries‘ # partition(is_odd, range(10)) --> 0 2 4 6 8 and 1 3 5 7 9 t1, t2 = tee(iterable) return filterfalse(pred, t1), filter(pred, t2) def powerset(iterable): "powerset([1,2,3]) --> () (1,) (2,) (3,) (1,2) (1,3) (2,3) (1,2,3)" s = list(iterable) return chain.from_iterable(combinations(s, r) for r in range(len(s)+1)) def unique_everseen(iterable, key=None): "List unique elements, preserving order. Remember all elements ever seen." # unique_everseen(‘AAAABBBCCDAABBB‘) --> A B C D # unique_everseen(‘ABBCcAD‘, str.lower) --> A B C D seen = set() seen_add = seen.add if key is None: for element in filterfalse(seen.__contains__, iterable): seen_add(element) yield element else: for element in iterable: k = key(element) if k not in seen: seen_add(k) yield element def unique_justseen(iterable, key=None): "List unique elements, preserving order. Remember only the element just seen." # unique_justseen(‘AAAABBBCCDAABBB‘) --> A B C D A B # unique_justseen(‘ABBCcAD‘, str.lower) --> A B C A D return map(next, map(itemgetter(1), groupby(iterable, key))) def iter_except(func, exception, first=None): """ Call a function repeatedly until an exception is raised. Converts a call-until-exception interface to an iterator interface. Like builtins.iter(func, sentinel) but uses an exception instead of a sentinel to end the loop. Examples: iter_except(functools.partial(heappop, h), IndexError) # priority queue iterator iter_except(d.popitem, KeyError) # non-blocking dict iterator iter_except(d.popleft, IndexError) # non-blocking deque iterator iter_except(q.get_nowait, Queue.Empty) # loop over a producer Queue iter_except(s.pop, KeyError) # non-blocking set iterator """ try: if first is not None: yield first() # For database APIs needing an initial cast to db.first() while True: yield func() except exception: pass def first_true(iterable, default=False, pred=None): """Returns the first true value in the iterable. If no true value is found, returns *default* If *pred* is not None, returns the first item for which pred(item) is true. """ # first_true([a,b,c], x) --> a or b or c or x # first_true([a,b], x, f) --> a if f(a) else b if f(b) else x return next(filter(pred, iterable), default) def random_product(*args, repeat=1): "Random selection from itertools.product(*args, **kwds)" pools = [tuple(pool) for pool in args] * repeat return tuple(random.choice(pool) for pool in pools) def random_permutation(iterable, r=None): "Random selection from itertools.permutations(iterable, r)" pool = tuple(iterable) r = len(pool) if r is None else r return tuple(random.sample(pool, r)) def random_combination(iterable, r): "Random selection from itertools.combinations(iterable, r)" pool = tuple(iterable) n = len(pool) indices = sorted(random.sample(range(n), r)) return tuple(pool[i] for i in indices) def random_combination_with_replacement(iterable, r): "Random selection from itertools.combinations_with_replacement(iterable, r)" pool = tuple(iterable) n = len(pool) indices = sorted(random.randrange(n) for i in range(r)) return tuple(pool[i] for i in indices)
【译】itertools